[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:10:30.025333 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:38757
I20260709 13:10:30.027196 21636 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:10:30.028376 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:30.049206 21642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:30.050321 21643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:30.050288 21645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:30.050814 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:31.308044 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:31.308344 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:31.308578 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602631308555 us; error 0 us; skew 500 ppm
I20260709 13:10:31.309556 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:31.317028 21636 webserver.cc:533] Webserver started at http://127.21.33.62:43727/ using document root <none> and password file <none>
I20260709 13:10:31.318080 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:31.318334 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:31.318851 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:31.325356 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "62f9e409586641608c8cceca1073a0ff"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-p8rg"
I20260709 13:10:31.334132 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:31.339998 21651 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.341539 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:31.341950 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "62f9e409586641608c8cceca1073a0ff"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-p8rg"
I20260709 13:10:31.342341 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:31.395613 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:31.397579 21636 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:10:31.398201 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:31.487596 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:38757
I20260709 13:10:31.487679 21702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:38757 every 8 connection(s)
I20260709 13:10:31.496973 21703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:31.522307 21703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff: Bootstrap starting.
I20260709 13:10:31.529202 21703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:31.531438 21703 log.cc:826] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff: Log is configured to *not* fsync() on all Append() calls
I20260709 13:10:31.536813 21703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff: No bootstrap required, opened a new log
I20260709 13:10:31.559458 21703 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f9e409586641608c8cceca1073a0ff" member_type: VOTER }
I20260709 13:10:31.560122 21703 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:31.560433 21703 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f9e409586641608c8cceca1073a0ff, State: Initialized, Role: FOLLOWER
I20260709 13:10:31.561681 21703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f9e409586641608c8cceca1073a0ff" member_type: VOTER }
I20260709 13:10:31.562261 21703 raft_consensus.cc:399] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:31.562577 21703 raft_consensus.cc:493] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:31.562973 21703 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:31.567902 21703 raft_consensus.cc:515] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f9e409586641608c8cceca1073a0ff" member_type: VOTER }
I20260709 13:10:31.568771 21703 leader_election.cc:304] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62f9e409586641608c8cceca1073a0ff; no voters: 
I20260709 13:10:31.570816 21703 leader_election.cc:290] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:31.571225 21706 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:31.573841 21706 raft_consensus.cc:697] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 1 LEADER]: Becoming Leader. State: Replica: 62f9e409586641608c8cceca1073a0ff, State: Running, Role: LEADER
I20260709 13:10:31.574960 21706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f9e409586641608c8cceca1073a0ff" member_type: VOTER }
I20260709 13:10:31.576155 21703 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:31.587013 21708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62f9e409586641608c8cceca1073a0ff. Latest consensus state: current_term: 1 leader_uuid: "62f9e409586641608c8cceca1073a0ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f9e409586641608c8cceca1073a0ff" member_type: VOTER } }
I20260709 13:10:31.587716 21707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62f9e409586641608c8cceca1073a0ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f9e409586641608c8cceca1073a0ff" member_type: VOTER } }
I20260709 13:10:31.588310 21707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:31.587976 21708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:31.592099 21714 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:31.606266 21714 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:31.610037 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:31.627738 21714 catalog_manager.cc:1383] Generated new cluster ID: 39fb63eeb9574792a12f55205a459195
I20260709 13:10:31.628142 21714 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:31.659153 21714 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:31.661041 21714 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:31.677943 21714 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff: Generated new TSK 0
I20260709 13:10:31.679232 21714 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:31.741474 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:31.750864 21725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.753340 21726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.756451 21728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:31.757352 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:31.758147 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:31.758337 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:31.758524 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602631758506 us; error 0 us; skew 500 ppm
I20260709 13:10:31.759183 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:31.764061 21636 webserver.cc:533] Webserver started at http://127.21.33.1:39691/ using document root <none> and password file <none>
I20260709 13:10:31.764678 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:31.764952 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:31.765268 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:31.766557 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "445da1eb4e864722a2e7648ab06210e0"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-p8rg"
I20260709 13:10:31.773833 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:31.779330 21733 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.780565 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:10:31.780951 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "445da1eb4e864722a2e7648ab06210e0"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-p8rg"
I20260709 13:10:31.781550 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:31.808063 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:31.810122 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:31.813316 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:31.819211 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:31.819559 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.819895 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:31.820127 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.928949 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:42151
I20260709 13:10:31.929013 21795 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:42151 every 8 connection(s)
I20260709 13:10:31.937626 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:31.951162 21800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.955713 21801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:31.962129 21804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:31.962422 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:31.963897 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:31.964201 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:31.964453 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602631964433 us; error 0 us; skew 500 ppm
I20260709 13:10:31.965401 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:31.967944 21796 heartbeater.cc:344] Connected to a master server at 127.21.33.62:38757
I20260709 13:10:31.968396 21796 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:31.969018 21636 webserver.cc:533] Webserver started at http://127.21.33.2:37631/ using document root <none> and password file <none>
I20260709 13:10:31.969691 21796 heartbeater.cc:507] Master 127.21.33.62:38757 requested a full tablet report, sending...
I20260709 13:10:31.969811 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:31.970196 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:31.970594 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:31.972175 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "e7a45570cd8647cfb086e131020776cf"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-p8rg"
I20260709 13:10:31.973068 21668 ts_manager.cc:194] Registered new tserver with Master: 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151)
I20260709 13:10:31.975710 21668 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32994
I20260709 13:10:31.980218 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:31.985391 21808 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:31.986357 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.002s
I20260709 13:10:31.986698 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "e7a45570cd8647cfb086e131020776cf"
format_stamp: "Formatted at 2026-07-09 13:10:31 on dist-test-slave-p8rg"
I20260709 13:10:31.987063 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:32.009814 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:32.011457 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:32.014303 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:32.018134 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:32.018430 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:32.018718 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:32.018971 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:32.131098 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:45401
I20260709 13:10:32.131230 21870 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:45401 every 8 connection(s)
I20260709 13:10:32.138816 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:32.150257 21874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:32.154750 21875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:32.154903 21871 heartbeater.cc:344] Connected to a master server at 127.21.33.62:38757
I20260709 13:10:32.155581 21871 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:32.156610 21871 heartbeater.cc:507] Master 127.21.33.62:38757 requested a full tablet report, sending...
I20260709 13:10:32.159185 21668 ts_manager.cc:194] Registered new tserver with Master: e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:32.161648 21668 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32998
I20260709 13:10:32.162690 21636 server_base.cc:1061] running on GCE node
W20260709 13:10:32.163642 21877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:32.164868 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:32.165184 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:32.165433 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602632165411 us; error 0 us; skew 500 ppm
I20260709 13:10:32.166240 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:32.169334 21636 webserver.cc:533] Webserver started at http://127.21.33.3:42755/ using document root <none> and password file <none>
I20260709 13:10:32.170003 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:32.170243 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:32.170565 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:32.172273 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "4096f29ab5894a7d9119444ce732db3b"
format_stamp: "Formatted at 2026-07-09 13:10:32 on dist-test-slave-p8rg"
I20260709 13:10:32.178666 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:10:32.182749 21882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:32.183771 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:10:32.184087 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "4096f29ab5894a7d9119444ce732db3b"
format_stamp: "Formatted at 2026-07-09 13:10:32 on dist-test-slave-p8rg"
I20260709 13:10:32.184415 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:32.195957 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:32.197368 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:32.199067 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:32.201900 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:32.202153 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:32.202407 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:32.202587 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:32.307382 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:36061
I20260709 13:10:32.308007 21944 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:36061 every 8 connection(s)
I20260709 13:10:32.326289 21945 heartbeater.cc:344] Connected to a master server at 127.21.33.62:38757
I20260709 13:10:32.326761 21945 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:32.327782 21945 heartbeater.cc:507] Master 127.21.33.62:38757 requested a full tablet report, sending...
I20260709 13:10:32.330135 21668 ts_manager.cc:194] Registered new tserver with Master: 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:32.331087 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017992294s
I20260709 13:10:32.331919 21668 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33002
I20260709 13:10:32.390810 21668 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33008:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:32.395421 21668 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:32.506505 21761 tablet_service.cc:1511] Processing CreateTablet for tablet 6441660788ea41a4ad81a0a4a2e68422 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:32.508613 21761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6441660788ea41a4ad81a0a4a2e68422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.512553 21759 tablet_service.cc:1511] Processing CreateTablet for tablet a443c6351ec94cb1ad4ab60d9ff96f1c (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:32.513373 21760 tablet_service.cc:1511] Processing CreateTablet for tablet c77dc53b15fb4f97905fe230f1954dd0 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:32.514214 21758 tablet_service.cc:1511] Processing CreateTablet for tablet 5a33ce30108d4326916d4119bb6d09e2 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:32.519667 21760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c77dc53b15fb4f97905fe230f1954dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.520783 21758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a33ce30108d4326916d4119bb6d09e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.524524 21757 tablet_service.cc:1511] Processing CreateTablet for tablet 76d790a743104ec3be0e678ee2e0ddfc (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:32.526996 21759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a443c6351ec94cb1ad4ab60d9ff96f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.535952 21833 tablet_service.cc:1511] Processing CreateTablet for tablet 5a33ce30108d4326916d4119bb6d09e2 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:32.537470 21833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a33ce30108d4326916d4119bb6d09e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.543157 21834 tablet_service.cc:1511] Processing CreateTablet for tablet a443c6351ec94cb1ad4ab60d9ff96f1c (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:32.544251 21836 tablet_service.cc:1511] Processing CreateTablet for tablet 6441660788ea41a4ad81a0a4a2e68422 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:32.546157 21835 tablet_service.cc:1511] Processing CreateTablet for tablet c77dc53b15fb4f97905fe230f1954dd0 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:32.548789 21757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d790a743104ec3be0e678ee2e0ddfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.555573 21834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a443c6351ec94cb1ad4ab60d9ff96f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.563674 21832 tablet_service.cc:1511] Processing CreateTablet for tablet 76d790a743104ec3be0e678ee2e0ddfc (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:32.563918 21756 tablet_service.cc:1511] Processing CreateTablet for tablet 943224133cb7443597a661a32f71c9ff (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:32.561488 21831 tablet_service.cc:1511] Processing CreateTablet for tablet 943224133cb7443597a661a32f71c9ff (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:32.570842 21755 tablet_service.cc:1511] Processing CreateTablet for tablet dcbe798d310f480b9a4f05494e867a2b (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:32.573195 21836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6441660788ea41a4ad81a0a4a2e68422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.575754 21831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943224133cb7443597a661a32f71c9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.578491 21756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943224133cb7443597a661a32f71c9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.578799 21754 tablet_service.cc:1511] Processing CreateTablet for tablet 4173eaab392c428a9a469cf5722c5b6f (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:32.583954 21830 tablet_service.cc:1511] Processing CreateTablet for tablet dcbe798d310f480b9a4f05494e867a2b (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:32.580180 21835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c77dc53b15fb4f97905fe230f1954dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.590277 21830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbe798d310f480b9a4f05494e867a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.589706 21754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4173eaab392c428a9a469cf5722c5b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.590735 21829 tablet_service.cc:1511] Processing CreateTablet for tablet 4173eaab392c428a9a469cf5722c5b6f (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:32.595366 21755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbe798d310f480b9a4f05494e867a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.604197 21832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d790a743104ec3be0e678ee2e0ddfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.609709 21910 tablet_service.cc:1511] Processing CreateTablet for tablet 6441660788ea41a4ad81a0a4a2e68422 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION VALUES < 238609294
W20260709 13:10:32.614328 21716 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:32.608276 21909 tablet_service.cc:1511] Processing CreateTablet for tablet c77dc53b15fb4f97905fe230f1954dd0 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:32.618891 21829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4173eaab392c428a9a469cf5722c5b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.616070 21908 tablet_service.cc:1511] Processing CreateTablet for tablet a443c6351ec94cb1ad4ab60d9ff96f1c (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:32.633224 21753 tablet_service.cc:1511] Processing CreateTablet for tablet 7dab60e1b775470cb89f2fa21d3eb6e8 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:32.632292 21910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6441660788ea41a4ad81a0a4a2e68422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.651643 21904 tablet_service.cc:1511] Processing CreateTablet for tablet dcbe798d310f480b9a4f05494e867a2b (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:32.626261 21907 tablet_service.cc:1511] Processing CreateTablet for tablet 5a33ce30108d4326916d4119bb6d09e2 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:32.656245 21906 tablet_service.cc:1511] Processing CreateTablet for tablet 76d790a743104ec3be0e678ee2e0ddfc (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:32.656378 21828 tablet_service.cc:1511] Processing CreateTablet for tablet 7dab60e1b775470cb89f2fa21d3eb6e8 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:32.662439 21909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c77dc53b15fb4f97905fe230f1954dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.658200 21753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab60e1b775470cb89f2fa21d3eb6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.663335 21905 tablet_service.cc:1511] Processing CreateTablet for tablet 943224133cb7443597a661a32f71c9ff (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:32.665136 21908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a443c6351ec94cb1ad4ab60d9ff96f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.675460 21903 tablet_service.cc:1511] Processing CreateTablet for tablet 4173eaab392c428a9a469cf5722c5b6f (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:32.690416 21906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d790a743104ec3be0e678ee2e0ddfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.713568 21902 tablet_service.cc:1511] Processing CreateTablet for tablet 7dab60e1b775470cb89f2fa21d3eb6e8 (DEFAULT_TABLE table=test-workload [id=29fb7928ab914a2d9f389fd3ae301673]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:32.713068 21828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab60e1b775470cb89f2fa21d3eb6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.731961 21903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4173eaab392c428a9a469cf5722c5b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.757800 21904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbe798d310f480b9a4f05494e867a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.781776 21907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a33ce30108d4326916d4119bb6d09e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.808477 21963 tablet_bootstrap.cc:492] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:32.808449 21905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 943224133cb7443597a661a32f71c9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.820623 21902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab60e1b775470cb89f2fa21d3eb6e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:32.882982 21964 tablet_bootstrap.cc:492] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:32.909340 21963 tablet_bootstrap.cc:654] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:32.917405 21965 tablet_bootstrap.cc:492] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:32.927665 21964 tablet_bootstrap.cc:654] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:32.947120 21965 tablet_bootstrap.cc:654] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:32.977312 21964 tablet_bootstrap.cc:492] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:32.979027 21964 ts_tablet_manager.cc:1403] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.030s
I20260709 13:10:32.980516 21796 heartbeater.cc:499] Master 127.21.33.62:38757 was elected leader, sending a full tablet report...
I20260709 13:10:32.983400 21965 tablet_bootstrap.cc:492] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:32.983814 21963 tablet_bootstrap.cc:492] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:32.983897 21965 ts_tablet_manager.cc:1403] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.068s	user 0.017s	sys 0.003s
I20260709 13:10:32.987284 21963 ts_tablet_manager.cc:1403] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.187s	user 0.027s	sys 0.033s
I20260709 13:10:32.989228 21964 raft_consensus.cc:359] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:32.990116 21964 raft_consensus.cc:385] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:32.990425 21964 raft_consensus.cc:740] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:32.991149 21964 consensus_queue.cc:260] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:32.999655 21965 raft_consensus.cc:359] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.004114 21965 raft_consensus.cc:385] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.004735 21965 raft_consensus.cc:740] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.002830 21963 raft_consensus.cc:359] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.006140 21963 raft_consensus.cc:385] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.006690 21963 raft_consensus.cc:740] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.007902 21963 consensus_queue.cc:260] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.010200 21965 consensus_queue.cc:260] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.013345 21871 heartbeater.cc:499] Master 127.21.33.62:38757 was elected leader, sending a full tablet report...
I20260709 13:10:33.014595 21963 ts_tablet_manager.cc:1434] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.003s
I20260709 13:10:33.015834 21963 tablet_bootstrap.cc:492] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.023959 21963 tablet_bootstrap.cc:654] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.025835 21945 heartbeater.cc:499] Master 127.21.33.62:38757 was elected leader, sending a full tablet report...
I20260709 13:10:33.026845 21965 ts_tablet_manager.cc:1434] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.001s
I20260709 13:10:33.033368 21964 ts_tablet_manager.cc:1434] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.022s
I20260709 13:10:33.034505 21964 tablet_bootstrap.cc:492] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.034607 21965 tablet_bootstrap.cc:492] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.041884 21964 tablet_bootstrap.cc:654] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.042095 21965 tablet_bootstrap.cc:654] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:33.065588 21946 tablet.cc:2406] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:33.066848 21963 tablet_bootstrap.cc:492] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.067435 21963 ts_tablet_manager.cc:1403] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.007s
I20260709 13:10:33.067654 21964 tablet_bootstrap.cc:492] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.068138 21964 ts_tablet_manager.cc:1403] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.015s
I20260709 13:10:33.071240 21965 tablet_bootstrap.cc:492] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.071694 21965 ts_tablet_manager.cc:1403] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.008s
I20260709 13:10:33.074095 21963 raft_consensus.cc:359] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.074405 21964 raft_consensus.cc:359] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.074906 21963 raft_consensus.cc:385] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.074656 21965 raft_consensus.cc:359] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.075361 21964 raft_consensus.cc:385] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.075362 21963 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.075928 21964 raft_consensus.cc:740] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.076406 21963 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.076761 21964 consensus_queue.cc:260] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.075423 21965 raft_consensus.cc:385] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.078218 21965 raft_consensus.cc:740] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.079264 21964 ts_tablet_manager.cc:1434] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:10:33.078989 21965 consensus_queue.cc:260] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.080595 21964 tablet_bootstrap.cc:492] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.081203 21965 ts_tablet_manager.cc:1434] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:10:33.082252 21965 tablet_bootstrap.cc:492] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.085039 21963 ts_tablet_manager.cc:1434] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:10:33.086531 21971 raft_consensus.cc:493] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.087033 21971 raft_consensus.cc:515] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.087560 21969 raft_consensus.cc:493] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.088091 21969 raft_consensus.cc:515] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.090298 21971 leader_election.cc:290] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:33.097553 21963 tablet_bootstrap.cc:492] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.118600 21971 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.090921 21969 leader_election.cc:290] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.119297 21971 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.122026 21971 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:33.117825 21964 tablet_bootstrap.cc:654] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.126219 21963 tablet_bootstrap.cc:654] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.129949 21965 tablet_bootstrap.cc:654] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.138288 21964 tablet_bootstrap.cc:492] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.138818 21964 ts_tablet_manager.cc:1403] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.008s
I20260709 13:10:33.151350 21964 raft_consensus.cc:359] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.152194 21964 raft_consensus.cc:385] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.152504 21964 raft_consensus.cc:740] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.153343 21964 consensus_queue.cc:260] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.164929 21964 ts_tablet_manager.cc:1434] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:10:33.171280 21964 tablet_bootstrap.cc:492] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.176964 21963 tablet_bootstrap.cc:492] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.177712 21963 ts_tablet_manager.cc:1403] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.092s	user 0.016s	sys 0.011s
I20260709 13:10:33.181529 21963 raft_consensus.cc:359] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.182374 21963 raft_consensus.cc:385] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.182703 21963 raft_consensus.cc:740] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.183485 21964 tablet_bootstrap.cc:654] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.183548 21963 consensus_queue.cc:260] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.185976 21963 ts_tablet_manager.cc:1434] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:33.187047 21963 tablet_bootstrap.cc:492] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.208612 21963 tablet_bootstrap.cc:654] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.218642 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.218443 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4173eaab392c428a9a469cf5722c5b6f" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.219604 21770 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 0.
W20260709 13:10:33.221320 21883 leader_election.cc:343] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.222345 21883 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:33.223538 21975 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.224083 21975 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.224553 21975 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.226188 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.226941 21770 raft_consensus.cc:2468] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.230644 21965 tablet_bootstrap.cc:492] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.230938 21809 leader_election.cc:304] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.231253 21965 ts_tablet_manager.cc:1403] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.149s	user 0.011s	sys 0.003s
I20260709 13:10:33.231837 21969 raft_consensus.cc:2804] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.232220 21969 raft_consensus.cc:493] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.232537 21969 raft_consensus.cc:3060] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.235821 21963 tablet_bootstrap.cc:492] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.236351 21963 ts_tablet_manager.cc:1403] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.000s
I20260709 13:10:33.237663 21965 raft_consensus.cc:359] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.238490 21965 raft_consensus.cc:385] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.238821 21965 raft_consensus.cc:740] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.239441 21963 raft_consensus.cc:359] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.240190 21963 raft_consensus.cc:385] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.239715 21965 consensus_queue.cc:260] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.240530 21963 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.240417 21969 raft_consensus.cc:515] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.241319 21963 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.242753 21969 leader_election.cc:290] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.243826 21963 ts_tablet_manager.cc:1434] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:10:33.245179 21963 tablet_bootstrap.cc:492] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.250123 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4173eaab392c428a9a469cf5722c5b6f" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
I20260709 13:10:33.251834 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
W20260709 13:10:33.253618 21885 leader_election.cc:343] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.254077 21885 leader_election.cc:304] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b; no voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf
I20260709 13:10:33.254938 21971 raft_consensus.cc:2749] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:33.255420 21885 leader_election.cc:343] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.257601 21975 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.258940 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.259112 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.263058 21770 raft_consensus.cc:3060] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.260618 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.264691 21963 tablet_bootstrap.cc:654] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:33.264611 21810 leader_election.cc:343] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.265801 21964 tablet_bootstrap.cc:492] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
W20260709 13:10:33.265929 21810 leader_election.cc:343] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.266397 21964 ts_tablet_manager.cc:1403] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.095s	user 0.044s	sys 0.021s
I20260709 13:10:33.270113 21964 raft_consensus.cc:359] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.270948 21964 raft_consensus.cc:385] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.271283 21964 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.261320 21965 ts_tablet_manager.cc:1434] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 13:10:33.272789 21965 tablet_bootstrap.cc:492] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.272296 21964 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.275019 21964 ts_tablet_manager.cc:1434] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:10:33.276028 21964 tablet_bootstrap.cc:492] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.280542 21770 raft_consensus.cc:2468] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.283488 21964 tablet_bootstrap.cc:654] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.284026 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.284423 21809 leader_election.cc:304] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:33.284873 21771 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.285379 21969 raft_consensus.cc:2804] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.286450 21975 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:33.288383 21969 raft_consensus.cc:697] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:33.289014 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:33.289366 21969 consensus_queue.cc:237] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
W20260709 13:10:33.290535 21885 leader_election.cc:343] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.292665 21771 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 1.
I20260709 13:10:33.292969 21963 tablet_bootstrap.cc:492] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.293460 21963 ts_tablet_manager.cc:1403] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.009s
I20260709 13:10:33.293748 21883 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: e7a45570cd8647cfb086e131020776cf
I20260709 13:10:33.295121 21970 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.295662 21970 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.296977 21963 raft_consensus.cc:359] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.297920 21963 raft_consensus.cc:385] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.298291 21963 raft_consensus.cc:740] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.299413 21963 consensus_queue.cc:260] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.304539 21975 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.298899 21970 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.308074 21965 tablet_bootstrap.cc:654] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.309571 21990 raft_consensus.cc:493] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.305593 21975 raft_consensus.cc:697] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:33.310271 21990 raft_consensus.cc:515] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.311306 21975 consensus_queue.cc:237] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.313835 21990 leader_election.cc:290] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.315105 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
W20260709 13:10:33.316800 21810 leader_election.cc:343] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.319731 21963 ts_tablet_manager.cc:1434] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.002s
I20260709 13:10:33.321082 21963 tablet_bootstrap.cc:492] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.317397 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.322993 21771 raft_consensus.cc:2468] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.326527 21809 leader_election.cc:304] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:33.327620 21969 raft_consensus.cc:2804] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.328114 21969 raft_consensus.cc:493] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.328545 21969 raft_consensus.cc:3060] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.349644 21668 catalog_manager.cc:5697] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 1 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:33.348978 21667 catalog_manager.cc:5697] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 0 to 1, leader changed from <none> to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.356133 21969 raft_consensus.cc:515] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.359755 21969 leader_election.cc:290] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.361479 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.362265 21771 raft_consensus.cc:3060] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.362083 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
W20260709 13:10:33.369925 21810 leader_election.cc:343] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.375422 21771 raft_consensus.cc:2468] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.376891 21964 tablet_bootstrap.cc:492] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.377156 21963 tablet_bootstrap.cc:654] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.377032 21809 leader_election.cc:304] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:33.377481 21964 ts_tablet_manager.cc:1403] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.007s
I20260709 13:10:33.379272 21969 raft_consensus.cc:2804] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.379804 21969 raft_consensus.cc:697] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:33.383725 21969 consensus_queue.cc:237] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.389739 21990 raft_consensus.cc:493] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.393019 21990 raft_consensus.cc:515] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.399345 21995 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.400241 21990 leader_election.cc:290] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.400004 21995 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.406728 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.408527 21964 raft_consensus.cc:359] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.408668 21995 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.406155 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
W20260709 13:10:33.410235 21809 leader_election.cc:343] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.410501 21964 raft_consensus.cc:385] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.411636 21964 raft_consensus.cc:740] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.411943 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
I20260709 13:10:33.412355 21964 consensus_queue.cc:260] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.414324 21964 ts_tablet_manager.cc:1434] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
W20260709 13:10:33.414458 21810 leader_election.cc:343] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.415252 21964 tablet_bootstrap.cc:492] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.415086 21810 leader_election.cc:304] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:33.411631 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
I20260709 13:10:33.416513 21990 raft_consensus.cc:2749] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:33.416687 21846 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 0.
W20260709 13:10:33.418879 21736 leader_election.cc:343] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.419602 21736 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.420850 21995 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.421204 21995 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.423700 21995 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.436261 21965 tablet_bootstrap.cc:492] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.436873 21963 tablet_bootstrap.cc:492] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.437455 21963 ts_tablet_manager.cc:1403] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.117s	user 0.025s	sys 0.018s
I20260709 13:10:33.436841 21965 ts_tablet_manager.cc:1403] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.164s	user 0.005s	sys 0.011s
I20260709 13:10:33.439042 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.439272 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.440862 21963 raft_consensus.cc:359] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
W20260709 13:10:33.444681 21735 leader_election.cc:343] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.442719 21963 raft_consensus.cc:385] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.445489 21735 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0; no voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf
I20260709 13:10:33.447501 21970 raft_consensus.cc:2749] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:33.448019 21735 leader_election.cc:343] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.447016 21963 raft_consensus.cc:740] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.449349 21965 raft_consensus.cc:359] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.450145 21965 raft_consensus.cc:385] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.450457 21965 raft_consensus.cc:740] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.449767 21963 consensus_queue.cc:260] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.456136 21995 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.457535 21963 ts_tablet_manager.cc:1434] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:33.456996 21965 consensus_queue.cc:260] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.458499 21995 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.459115 21963 tablet_bootstrap.cc:492] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.459226 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:33.459729 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.459839 21964 tablet_bootstrap.cc:654] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.459884 21846 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:33.461445 21735 leader_election.cc:343] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.470920 21965 ts_tablet_manager.cc:1434] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.007s
I20260709 13:10:33.472179 21965 tablet_bootstrap.cc:492] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.473379 21964 tablet_bootstrap.cc:492] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.473961 21964 ts_tablet_manager.cc:1403] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.005s
I20260709 13:10:33.478384 21964 raft_consensus.cc:359] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.479226 21964 raft_consensus.cc:385] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.481011 21963 tablet_bootstrap.cc:654] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.479527 21964 raft_consensus.cc:740] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.484133 21846 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 1.
I20260709 13:10:33.483325 21964 consensus_queue.cc:260] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.486045 21736 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:33.486682 21964 ts_tablet_manager.cc:1434] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:10:33.487440 21995 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.488252 21995 raft_consensus.cc:697] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:33.488890 21964 tablet_bootstrap.cc:492] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.487722 21965 tablet_bootstrap.cc:654] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.489641 21995 consensus_queue.cc:237] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.502235 21964 tablet_bootstrap.cc:654] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.502892 21668 catalog_manager.cc:5697] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 0 to 1, leader changed from <none> to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.516449 21963 tablet_bootstrap.cc:492] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.517038 21963 ts_tablet_manager.cc:1403] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.010s
I20260709 13:10:33.520193 21665 catalog_manager.cc:5697] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.523415 21963 raft_consensus.cc:359] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.525029 21963 raft_consensus.cc:385] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.525432 21963 raft_consensus.cc:740] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.526465 21963 consensus_queue.cc:260] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.527199 21965 tablet_bootstrap.cc:492] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.528074 21965 ts_tablet_manager.cc:1403] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.011s
I20260709 13:10:33.529295 21963 ts_tablet_manager.cc:1434] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:10:33.530509 21963 tablet_bootstrap.cc:492] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.531605 21965 raft_consensus.cc:359] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.532536 21965 raft_consensus.cc:385] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.532653 21975 raft_consensus.cc:493] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.532886 21965 raft_consensus.cc:740] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.533185 21975 raft_consensus.cc:515] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.533928 21965 consensus_queue.cc:260] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.535449 21975 leader_election.cc:290] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:33.536824 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
W20260709 13:10:33.538463 21883 leader_election.cc:343] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.539517 21963 tablet_bootstrap.cc:654] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.539839 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
W20260709 13:10:33.541513 21885 leader_election.cc:343] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:33.541981 21885 leader_election.cc:304] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b; no voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf
I20260709 13:10:33.543529 21965 ts_tablet_manager.cc:1434] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.005s
I20260709 13:10:33.543545 21975 raft_consensus.cc:2749] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:33.551062 21965 tablet_bootstrap.cc:492] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.556906 21964 tablet_bootstrap.cc:492] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.557469 21964 ts_tablet_manager.cc:1403] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.007s
I20260709 13:10:33.560665 21964 raft_consensus.cc:359] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.561452 21964 raft_consensus.cc:385] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.561806 21964 raft_consensus.cc:740] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.562857 21965 tablet_bootstrap.cc:654] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.562623 21964 consensus_queue.cc:260] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.566049 21963 tablet_bootstrap.cc:492] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.566789 21963 ts_tablet_manager.cc:1403] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.004s
I20260709 13:10:33.567377 21964 ts_tablet_manager.cc:1434] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:10:33.568449 21964 tablet_bootstrap.cc:492] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.573560 21963 raft_consensus.cc:359] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.574620 21963 raft_consensus.cc:385] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.575002 21963 raft_consensus.cc:740] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.575927 21963 consensus_queue.cc:260] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.578985 21963 ts_tablet_manager.cc:1434] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:10:33.580479 21963 tablet_bootstrap.cc:492] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0: Bootstrap starting.
I20260709 13:10:33.589422 21963 tablet_bootstrap.cc:654] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.590081 21965 tablet_bootstrap.cc:492] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.590723 21965 ts_tablet_manager.cc:1403] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.015s
I20260709 13:10:33.593878 21964 tablet_bootstrap.cc:654] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.595149 21965 raft_consensus.cc:359] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.596258 21965 raft_consensus.cc:385] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.596688 21965 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.598753 21965 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.602246 21965 ts_tablet_manager.cc:1434] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.003s
I20260709 13:10:33.603705 21965 tablet_bootstrap.cc:492] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.608573 21963 tablet_bootstrap.cc:492] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0: No bootstrap required, opened a new log
I20260709 13:10:33.609153 21963 ts_tablet_manager.cc:1403] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.006s
I20260709 13:10:33.613024 21963 raft_consensus.cc:359] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.613885 21963 raft_consensus.cc:385] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.614143 21965 tablet_bootstrap.cc:654] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.614187 21963 raft_consensus.cc:740] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.614997 21963 consensus_queue.cc:260] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
W20260709 13:10:33.616487 21716 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:33.617671 21963 ts_tablet_manager.cc:1434] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:33.624010 21965 tablet_bootstrap.cc:492] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.624696 21965 ts_tablet_manager.cc:1403] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:10:33.630348 21965 raft_consensus.cc:359] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.631400 21965 raft_consensus.cc:385] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.631788 21965 raft_consensus.cc:740] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.632779 21965 consensus_queue.cc:260] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.635876 21965 ts_tablet_manager.cc:1434] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:10:33.636799 21964 tablet_bootstrap.cc:492] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.637036 21965 tablet_bootstrap.cc:492] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.637534 21964 ts_tablet_manager.cc:1403] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.069s	user 0.020s	sys 0.020s
I20260709 13:10:33.639752 21964 raft_consensus.cc:359] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.640277 21964 raft_consensus.cc:385] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.640552 21964 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.641244 21964 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.643311 21964 ts_tablet_manager.cc:1434] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:33.644294 21964 tablet_bootstrap.cc:492] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: Bootstrap starting.
I20260709 13:10:33.645040 21965 tablet_bootstrap.cc:654] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.650282 21964 tablet_bootstrap.cc:654] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.673759 21969 raft_consensus.cc:493] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.674314 21969 raft_consensus.cc:515] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.675611 21965 tablet_bootstrap.cc:492] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.676121 21965 ts_tablet_manager.cc:1403] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.020s
I20260709 13:10:33.676628 21969 leader_election.cc:290] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.677986 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.679211 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.679842 21771 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.678722 21920 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.680629 21964 tablet_bootstrap.cc:492] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: No bootstrap required, opened a new log
I20260709 13:10:33.681149 21964 ts_tablet_manager.cc:1403] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: Time spent bootstrapping tablet: real 0.037s	user 0.023s	sys 0.011s
I20260709 13:10:33.681020 21809 leader_election.cc:304] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.682008 21969 raft_consensus.cc:2804] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.682370 21969 raft_consensus.cc:493] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.682675 21969 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.684871 21965 raft_consensus.cc:359] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.685037 21964 raft_consensus.cc:359] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.685592 21965 raft_consensus.cc:385] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.685917 21964 raft_consensus.cc:385] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.686018 21965 raft_consensus.cc:740] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.686357 21964 raft_consensus.cc:740] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.687127 21964 consensus_queue.cc:260] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.687145 21965 consensus_queue.cc:260] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.689770 21964 ts_tablet_manager.cc:1434] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:33.692034 21965 ts_tablet_manager.cc:1434] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:10:33.693130 21965 tablet_bootstrap.cc:492] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: Bootstrap starting.
I20260709 13:10:33.702488 21965 tablet_bootstrap.cc:654] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:33.707916 21920 raft_consensus.cc:3060] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.711102 21969 raft_consensus.cc:515] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.714375 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.715052 21771 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.714591 21969 leader_election.cc:290] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.717939 21975 raft_consensus.cc:493] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.718423 21975 raft_consensus.cc:515] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.719283 21965 tablet_bootstrap.cc:492] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: No bootstrap required, opened a new log
I20260709 13:10:33.719751 21965 ts_tablet_manager.cc:1403] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.008s
I20260709 13:10:33.720528 21975 leader_election.cc:290] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:33.721894 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
I20260709 13:10:33.722766 21965 raft_consensus.cc:359] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.723443 21965 raft_consensus.cc:385] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:33.723735 21965 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Initialized, Role: FOLLOWER
I20260709 13:10:33.723796 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.724680 21770 raft_consensus.cc:2393] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4096f29ab5894a7d9119444ce732db3b in current term 1: Already voted for candidate e7a45570cd8647cfb086e131020776cf in this term.
I20260709 13:10:33.724604 21965 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.726260 21885 leader_election.cc:304] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b; no voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf
I20260709 13:10:33.727098 21965 ts_tablet_manager.cc:1434] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:33.727311 21971 raft_consensus.cc:3060] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.730324 21969 consensus_queue.cc:1048] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:33.732947 21771 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.734222 21809 leader_election.cc:304] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.735303 21969 raft_consensus.cc:2804] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.735984 21969 raft_consensus.cc:697] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:33.736784 21969 consensus_queue.cc:237] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:33.740412 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.741021 21920 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.750064 21975 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:33.752107 21920 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.759585 21971 raft_consensus.cc:2749] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:33.773557 21995 raft_consensus.cc:493] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.774299 21995 raft_consensus.cc:515] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.777089 21995 leader_election.cc:290] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.779875 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
I20260709 13:10:33.780635 21846 raft_consensus.cc:2468] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 0.
I20260709 13:10:33.782155 21736 leader_election.cc:304] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.782995 21995 raft_consensus.cc:2804] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.779845 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.783643 21995 raft_consensus.cc:493] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.784045 21995 raft_consensus.cc:3060] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.785199 21920 raft_consensus.cc:2468] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 0.
I20260709 13:10:33.794441 21995 raft_consensus.cc:515] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.796487 21846 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.797154 21995 leader_election.cc:290] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.797991 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:33.798591 21845 raft_consensus.cc:3060] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.803054 22023 raft_consensus.cc:493] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.803537 22023 raft_consensus.cc:515] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.806135 21845 raft_consensus.cc:2468] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 1.
I20260709 13:10:33.806926 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4173eaab392c428a9a469cf5722c5b6f" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.807217 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.807837 21920 raft_consensus.cc:3060] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.807943 21770 raft_consensus.cc:2468] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.811379 22019 raft_consensus.cc:493] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.811883 22019 raft_consensus.cc:515] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.813874 21969 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.814337 21969 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.816551 21969 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.818120 21736 leader_election.cc:304] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.819164 21995 raft_consensus.cc:2804] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.819621 21995 raft_consensus.cc:697] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:33.819979 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.820508 21995 consensus_queue.cc:237] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.821894 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.820777 21920 raft_consensus.cc:2468] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 1.
I20260709 13:10:33.822501 21919 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.824627 21810 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.825692 21990 raft_consensus.cc:2804] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.826053 21990 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.827005 21990 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.822471 21770 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.830654 21969 consensus_queue.cc:1048] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:33.833900 21667 catalog_manager.cc:5697] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 0 to 1, leader changed from <none> to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:33.843737 22023 leader_election.cc:290] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.846155 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4173eaab392c428a9a469cf5722c5b6f" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.847119 21919 raft_consensus.cc:2468] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.847245 21809 leader_election.cc:304] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.848552 22023 raft_consensus.cc:2804] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.848976 22023 raft_consensus.cc:493] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.849400 22023 raft_consensus.cc:3060] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.851444 22024 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:33.854758 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0" is_pre_election: true
I20260709 13:10:33.854470 21990 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.855384 21770 raft_consensus.cc:2468] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.858739 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.859519 21919 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.863425 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.863996 21770 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.866174 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:33.867060 21920 raft_consensus.cc:2468] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 0.
I20260709 13:10:33.868583 21810 leader_election.cc:304] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.869714 21969 raft_consensus.cc:2804] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:33.870122 21969 raft_consensus.cc:493] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.870517 21969 raft_consensus.cc:3060] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.872361 21770 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.828795 22019 leader_election.cc:290] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.894074 21665 catalog_manager.cc:5697] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.879127 21969 raft_consensus.cc:515] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.897238 21969 leader_election.cc:290] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.887524 21990 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.890569 22027 consensus_queue.cc:1048] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:33.887107 21919 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.898401 22019 consensus_queue.cc:1048] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:33.901481 21810 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.902683 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.902998 22019 raft_consensus.cc:2804] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.903381 21920 raft_consensus.cc:3060] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.904008 22019 raft_consensus.cc:697] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:33.898823 22023 raft_consensus.cc:515] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.904945 22019 consensus_queue.cc:237] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.915369 21919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4173eaab392c428a9a469cf5722c5b6f" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:33.913214 22023 leader_election.cc:290] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:33.919167 21768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4173eaab392c428a9a469cf5722c5b6f" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.922952 21768 raft_consensus.cc:3060] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.919126 21919 raft_consensus.cc:3060] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.921301 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:33.930188 21770 raft_consensus.cc:3060] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.955868 21918 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:33.955943 21768 raft_consensus.cc:2468] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.962800 21809 leader_election.cc:304] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:33.962299 21668 catalog_manager.cc:5697] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 0 to 1, leader changed from <none> to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.963822 22019 raft_consensus.cc:2804] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:33.964484 22019 raft_consensus.cc:697] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:33.965615 22019 consensus_queue.cc:237] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:33.966885 21995 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:33.968950 21919 raft_consensus.cc:2468] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.979182 21920 raft_consensus.cc:2468] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:33.991699 21668 catalog_manager.cc:5697] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 0 to 1, leader changed from <none> to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.013494 21770 raft_consensus.cc:2468] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 1.
I20260709 13:10:34.015815 21810 leader_election.cc:304] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:34.017822 22019 raft_consensus.cc:2804] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:34.018347 22019 raft_consensus.cc:697] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:34.019179 22019 consensus_queue.cc:237] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.034053 21995 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.054275 21665 catalog_manager.cc:5697] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 0 to 1, leader changed from <none> to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.147289 22019 consensus_queue.cc:1048] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.159472 22027 consensus_queue.cc:1048] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.206648 21970 consensus_queue.cc:1048] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.219522 22040 consensus_queue.cc:1048] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:34.344172 21846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:54704
I20260709 13:10:34.345057 21846 raft_consensus.cc:606] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Received request to transfer leadership to TS 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:34.362440 21918 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
mode: GRACEFUL
new_leader_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:40784
I20260709 13:10:34.363130 21918 raft_consensus.cc:606] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 LEADER]: Received request to transfer leadership to TS 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:34.369851 21846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:54704
I20260709 13:10:34.370493 21846 raft_consensus.cc:606] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Received request to transfer leadership to TS 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:34.391424 21771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
mode: GRACEFUL
new_leader_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:56660
I20260709 13:10:34.392026 21771 raft_consensus.cc:606] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Received request to transfer leadership to TS e7a45570cd8647cfb086e131020776cf
I20260709 13:10:34.407739 21846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:54704
I20260709 13:10:34.408375 21846 raft_consensus.cc:606] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Received request to transfer leadership to TS 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:34.442399 22025 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.450418 22027 raft_consensus.cc:993] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: : Instructing follower 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:34.451020 22023 raft_consensus.cc:1081] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Signalling peer 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:34.453153 21770 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:10:34.457458 21770 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:34.457919 21770 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.464005 22027 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.465940 21770 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.473927 21770 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:34.476823 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:34.478437 22023 consensus_queue.cc:1048] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:34.481134 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:34.481812 21846 raft_consensus.cc:3055] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:34.482122 21846 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:34.483479 22027 consensus_queue.cc:1048] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.483625 21846 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.485566 21846 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.493794 22023 consensus_queue.cc:1048] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:34.506688 22027 raft_consensus.cc:993] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: : Instructing follower 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:34.507253 22019 raft_consensus.cc:1081] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Signalling peer 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:34.513334 21917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:40760
I20260709 13:10:34.514014 21917 raft_consensus.cc:493] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:34.514389 21917 raft_consensus.cc:3060] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.527575 22019 consensus_queue.cc:1048] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:34.527855 21846 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:34.537781 21736 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:34.538793 22040 raft_consensus.cc:2804] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.539448 22040 raft_consensus.cc:697] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:34.540316 22040 consensus_queue.cc:237] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.547961 21917 raft_consensus.cc:515] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.554476 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:34.561760 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab60e1b775470cb89f2fa21d3eb6e8" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:34.562580 21846 raft_consensus.cc:3055] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:34.562975 21846 raft_consensus.cc:740] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:34.563858 21846 consensus_queue.cc:260] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.564900 21846 raft_consensus.cc:3060] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.574259 21846 raft_consensus.cc:2468] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 2.
I20260709 13:10:34.575498 21885 leader_election.cc:304] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:34.576418 22049 raft_consensus.cc:2804] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.576951 22049 raft_consensus.cc:697] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 2 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:34.576658 21668 catalog_manager.cc:5697] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 1 to 2, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.578020 22049 consensus_queue.cc:237] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.581408 21917 leader_election.cc:290] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 2 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:34.588263 21667 catalog_manager.cc:5697] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 1 to 2, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 2 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.681102 21995 raft_consensus.cc:993] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: : Instructing follower e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:34.681659 21995 raft_consensus.cc:1081] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Signalling peer e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:34.683517 21846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:54692
I20260709 13:10:34.684230 21846 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:34.684676 21846 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.692770 21846 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.695014 21846 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 2 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:34.696113 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:34.696286 21917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:34.696748 21771 raft_consensus.cc:3055] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:34.696920 21917 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.697108 21771 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:34.698006 21771 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.699321 21771 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.702428 21917 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 2.
I20260709 13:10:34.703591 21810 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:34.704324 22019 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.704783 22019 raft_consensus.cc:697] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:34.705860 22019 consensus_queue.cc:237] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.706251 21771 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 2.
I20260709 13:10:34.714282 21668 catalog_manager.cc:5697] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 1 to 2, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.734673 22023 raft_consensus.cc:993] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: : Instructing follower 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:34.735178 22023 raft_consensus.cc:1081] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Signalling peer 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:34.736841 21771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:10:34.737561 21771 raft_consensus.cc:493] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:34.737934 21771 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.745182 21771 raft_consensus.cc:515] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:34.747388 21771 leader_election.cc:290] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Requested vote from peers 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:34.748822 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:34.749094 21917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:34.749437 21846 raft_consensus.cc:3055] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:34.749768 21917 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.749801 21846 raft_consensus.cc:740] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:34.750864 21846 consensus_queue.cc:260] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:34.752115 21846 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.755216 21917 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:34.756273 21735 leader_election.cc:304] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:34.757011 21995 raft_consensus.cc:2804] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.757453 21995 raft_consensus.cc:697] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:34.758338 21995 consensus_queue.cc:237] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:34.759027 21846 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:34.767086 21668 catalog_manager.cc:5697] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 1 to 2, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.883215 22050 raft_consensus.cc:993] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: : Instructing follower 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:34.883706 22049 raft_consensus.cc:1081] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 LEADER]: Signalling peer 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:34.885680 21771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:56644
I20260709 13:10:34.886994 21771 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:34.887547 21771 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.896687 21771 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.898969 21771 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:34.900437 21917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:34.900393 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:34.901201 21917 raft_consensus.cc:3055] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:34.901304 21846 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.901607 21917 raft_consensus.cc:740] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:34.902516 21917 consensus_queue.cc:260] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.903571 21917 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.908972 21917 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:34.909646 21846 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:34.910202 21735 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:34.911090 21995 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:34.911593 21995 raft_consensus.cc:697] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:34.912446 21995 consensus_queue.cc:237] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:34.920545 21668 catalog_manager.cc:5697] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 1 to 2, leader changed from 4096f29ab5894a7d9119444ce732db3b (127.21.33.3) to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.958142 21846 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.959733 21995 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:34.969036 21917 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:34.978559 21917 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:34.980095 22038 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.057307 21771 raft_consensus.cc:3060] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:35.062718 21771 raft_consensus.cc:1275] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.064075 22049 consensus_queue.cc:1048] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:35.074182 21846 raft_consensus.cc:1275] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.075798 22049 consensus_queue.cc:1048] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.157902 21917 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.159341 22023 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.159615 21846 raft_consensus.cc:1275] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.161026 22038 consensus_queue.cc:1048] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.169720 21771 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.171339 22027 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:35.179816 21917 raft_consensus.cc:1275] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.181667 21995 consensus_queue.cc:1048] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.367952 21845 raft_consensus.cc:1275] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.369796 21995 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:35.380321 21917 raft_consensus.cc:1275] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:35.382974 21995 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.449887 21636 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:10:37.451064 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:37.477960 21846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:54710
I20260709 13:10:37.478700 21846 raft_consensus.cc:606] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Received request to transfer leadership to TS 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:37.492930 21771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
mode: GRACEFUL
new_leader_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:56662
I20260709 13:10:37.493669 21771 raft_consensus.cc:606] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Received request to transfer leadership to TS 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:37.495292 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:37.495859 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:37.784035 22038 raft_consensus.cc:993] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: : Instructing follower 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:37.784577 22072 raft_consensus.cc:1081] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Signalling peer 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:37.786768 21917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:40768
I20260709 13:10:37.787806 21917 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:37.788331 21917 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.791086 22073 raft_consensus.cc:993] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: : Instructing follower 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:37.791568 22027 raft_consensus.cc:1081] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Signalling peer 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:37.793145 21920 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:40760
I20260709 13:10:37.793702 21920 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:37.794107 21920 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.797573 21917 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:37.799921 21917 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:37.800803 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:37.801597 21771 raft_consensus.cc:3055] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:37.801448 21920 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:37.801970 21771 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:37.802831 21771 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:37.802935 21846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:37.803771 21771 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.804113 21846 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.808944 21771 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 3.
I20260709 13:10:37.810166 21846 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 3.
I20260709 13:10:37.810969 21920 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:37.811385 21883 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:37.812023 21771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:37.812184 22049 raft_consensus.cc:2804] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:37.812728 21771 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.812615 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:37.814024 21845 raft_consensus.cc:3055] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:37.814357 21845 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:37.815191 21845 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:37.816522 21845 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:37.816445 22049 raft_consensus.cc:697] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:37.817703 22049 consensus_queue.cc:237] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:37.821192 21771 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 3.
I20260709 13:10:37.822438 21770 raft_consensus.cc:1240] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Rejecting Update request from peer e7a45570cd8647cfb086e131020776cf for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:37.824294 21883 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:37.824685 21845 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 3.
I20260709 13:10:37.825292 22049 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:37.826082 22049 raft_consensus.cc:697] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:37.825766 21668 catalog_manager.cc:5697] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 2 to 3, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 3 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:37.826938 22049 consensus_queue.cc:237] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:37.844796 21668 catalog_manager.cc:5697] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 2 to 3, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 3 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:38.235569 21770 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:38.237483 22049 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:38.250211 21845 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:38.251856 22049 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:38.291350 21845 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:38.292917 22049 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:38.304515 21770 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:38.307726 22083 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:38.496907 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:38.508072 21770 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
mode: GRACEFUL
new_leader_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:56660
I20260709 13:10:38.508690 21770 raft_consensus.cc:606] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Received request to transfer leadership to TS e7a45570cd8647cfb086e131020776cf
I20260709 13:10:38.514612 21770 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
mode: GRACEFUL
new_leader_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:56660
I20260709 13:10:38.515236 21770 raft_consensus.cc:606] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Received request to transfer leadership to TS e7a45570cd8647cfb086e131020776cf
I20260709 13:10:38.521489 21920 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
mode: GRACEFUL
new_leader_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:40784
I20260709 13:10:38.522341 21920 raft_consensus.cc:606] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Received request to transfer leadership to TS e7a45570cd8647cfb086e131020776cf
I20260709 13:10:38.528115 21770 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
mode: GRACEFUL
new_leader_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:56660
I20260709 13:10:38.528662 21770 raft_consensus.cc:606] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Received request to transfer leadership to TS e7a45570cd8647cfb086e131020776cf
I20260709 13:10:38.534632 21920 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
mode: GRACEFUL
new_leader_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:40784
I20260709 13:10:38.535277 21920 raft_consensus.cc:606] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Received request to transfer leadership to TS e7a45570cd8647cfb086e131020776cf
I20260709 13:10:38.581084 22090 raft_consensus.cc:993] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: : Instructing follower e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.581727 22090 raft_consensus.cc:1081] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Signalling peer e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.583564 21845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:54692
I20260709 13:10:38.584092 21845 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:38.584506 21845 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.589990 21845 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.592041 21845 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:38.593103 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:38.593289 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:38.593946 21920 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.593901 21770 raft_consensus.cc:3055] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:38.594558 21770 raft_consensus.cc:740] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:38.595531 21770 consensus_queue.cc:260] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.597229 21770 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.602843 21920 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 3.
I20260709 13:10:38.604388 21810 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:38.605052 21770 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 3.
I20260709 13:10:38.605458 22073 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:38.606303 22073 raft_consensus.cc:697] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:38.607223 22073 consensus_queue.cc:237] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.615625 21665 catalog_manager.cc:5697] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 2 to 3, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 3 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:38.690840 22074 raft_consensus.cc:993] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: : Instructing follower e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.691357 22074 raft_consensus.cc:1081] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Signalling peer e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.692890 21845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:54678
I20260709 13:10:38.693403 21845 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:38.693766 21845 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:38.699589 21845 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.701679 21845 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 4 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:38.702551 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:38.702919 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:38.703197 21770 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:38.703469 21920 raft_consensus.cc:3055] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:38.703784 21920 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:38.704460 21920 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.705425 21920 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:38.708847 21770 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 4.
I20260709 13:10:38.710100 21809 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:38.710924 22073 raft_consensus.cc:2804] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:38.711063 21920 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 4.
I20260709 13:10:38.711413 22073 raft_consensus.cc:697] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:38.713558 22073 consensus_queue.cc:237] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.720085 21665 catalog_manager.cc:5697] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 3 to 4, leader changed from 4096f29ab5894a7d9119444ce732db3b (127.21.33.3) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 4 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:38.776718 22088 raft_consensus.cc:993] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: : Instructing follower e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.777248 22088 raft_consensus.cc:1081] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Signalling peer e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.778923 21845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:54678
I20260709 13:10:38.779435 21845 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:38.779768 21845 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:38.785157 21845 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.787505 21845 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 4 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:38.788345 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:38.788679 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:38.789058 21770 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:38.789208 21920 raft_consensus.cc:3055] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:38.789728 21920 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:38.790567 21920 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.791860 21920 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:38.797873 21770 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 4.
I20260709 13:10:38.799593 21809 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:38.800165 21920 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 4.
I20260709 13:10:38.800642 22073 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:38.801194 22073 raft_consensus.cc:697] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:38.802095 22073 consensus_queue.cc:237] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.808660 21665 catalog_manager.cc:5697] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 3 to 4, leader changed from 4096f29ab5894a7d9119444ce732db3b (127.21.33.3) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 4 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:38.846733 22072 raft_consensus.cc:993] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: : Instructing follower e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.847239 22072 raft_consensus.cc:1081] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Signalling peer e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.848795 21845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:54692
I20260709 13:10:38.849327 21845 raft_consensus.cc:493] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:38.849668 21845 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.854869 21845 raft_consensus.cc:515] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:38.856698 21845 leader_election.cc:290] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:38.857971 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:38.857992 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6441660788ea41a4ad81a0a4a2e68422" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:38.858794 21770 raft_consensus.cc:3055] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:38.859146 21920 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.859613 21770 raft_consensus.cc:740] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:38.860517 21770 consensus_queue.cc:260] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:38.861848 21770 raft_consensus.cc:3060] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:38.862915 22090 raft_consensus.cc:993] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: : Instructing follower e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.863364 22072 raft_consensus.cc:1081] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Signalling peer e7a45570cd8647cfb086e131020776cf to start an election
I20260709 13:10:38.864840 21845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
 from {username='slave'} at 127.0.0.1:54692
I20260709 13:10:38.865351 21845 raft_consensus.cc:493] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:38.865733 21845 raft_consensus.cc:3060] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:38.867460 21920 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 3.
I20260709 13:10:38.868696 21810 leader_election.cc:304] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:38.869593 22073 raft_consensus.cc:2804] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:38.869642 21770 raft_consensus.cc:2468] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 3.
I20260709 13:10:38.870213 22073 raft_consensus.cc:697] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:38.871121 22073 consensus_queue.cc:237] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } }
I20260709 13:10:38.871356 21845 raft_consensus.cc:515] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.873544 21845 leader_election.cc:290] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 2 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:38.874289 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:38.874853 21770 raft_consensus.cc:3055] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:38.875082 21770 raft_consensus.cc:740] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:38.876379 21920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a33ce30108d4326916d4119bb6d09e2" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:38.875864 21770 consensus_queue.cc:260] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.877015 21920 raft_consensus.cc:3060] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:38.877738 21770 raft_consensus.cc:3060] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:38.881795 21665 catalog_manager.cc:5697] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 2 to 3, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 3 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:38.883437 21920 raft_consensus.cc:2468] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 2.
I20260709 13:10:38.884681 21810 leader_election.cc:304] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:38.885296 21770 raft_consensus.cc:2468] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 2.
I20260709 13:10:38.885408 22073 raft_consensus.cc:2804] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:38.885991 22073 raft_consensus.cc:697] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:38.886935 22073 consensus_queue.cc:237] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:38.898089 21665 catalog_manager.cc:5697] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 1 to 2, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:39.078790 21920 raft_consensus.cc:1275] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:39.080350 22073 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:39.092718 21770 raft_consensus.cc:1275] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:39.095026 22073 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:39.141525 21770 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:39.143481 22073 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:39.151656 21920 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:39.153007 22103 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:39.246502 21771 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:39.247846 22099 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:39.255604 21920 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:39.257129 22099 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:39.290220 21920 raft_consensus.cc:1275] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:39.291465 22099 consensus_queue.cc:1048] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:39.301591 21771 raft_consensus.cc:1275] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:39.303493 22103 consensus_queue.cc:1048] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:39.331135 21920 raft_consensus.cc:1275] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:39.332626 22103 consensus_queue.cc:1048] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.342540 21771 raft_consensus.cc:1275] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Refusing update from remote peer e7a45570cd8647cfb086e131020776cf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:39.344694 22073 consensus_queue.cc:1048] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:10:41.219156 21792 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 21639
W20260709 13:10:41.220170 21792 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 21702
W20260709 13:10:41.221040 21792 debug-util.cc:398] Leaking SignalData structure 0x7b08001a3be0 after lost signal to thread 21795
W20260709 13:10:41.221869 21792 debug-util.cc:398] Leaking SignalData structure 0x7b08001a1ea0 after lost signal to thread 21870
W20260709 13:10:41.222625 21792 debug-util.cc:398] Leaking SignalData structure 0x7b08001da8c0 after lost signal to thread 21944
I20260709 13:10:41.638906 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:41.650848 21845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:54710
I20260709 13:10:41.651892 21845 raft_consensus.cc:606] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Received request to transfer leadership to TS 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:41.655952 21845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "943224133cb7443597a661a32f71c9ff"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:54710
I20260709 13:10:41.656601 21845 raft_consensus.cc:606] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Received request to transfer leadership to TS 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:41.659744 21845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:54710
I20260709 13:10:41.660265 21845 raft_consensus.cc:606] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Received request to transfer leadership to TS 4096f29ab5894a7d9119444ce732db3b
I20260709 13:10:41.663007 21845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:54710
I20260709 13:10:41.663555 21845 raft_consensus.cc:606] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Received request to transfer leadership to TS 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:41.666070 21845 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b"
dest_uuid: "e7a45570cd8647cfb086e131020776cf"
mode: GRACEFUL
new_leader_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:54710
I20260709 13:10:41.666589 21845 raft_consensus.cc:606] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Received request to transfer leadership to TS 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:41.667888 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:41.668586 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:41.706851 22099 raft_consensus.cc:993] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: : Instructing follower 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:41.708457 22111 raft_consensus.cc:1081] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Signalling peer 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:41.710580 21771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:10:41.711230 21771 raft_consensus.cc:493] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:41.711663 21771 raft_consensus.cc:3060] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.722180 21771 raft_consensus.cc:515] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.726500 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:41.727540 21845 raft_consensus.cc:3055] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:41.727991 21771 leader_election.cc:290] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:41.727931 21845 raft_consensus.cc:740] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:41.729880 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:41.729946 21845 consensus_queue.cc:260] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.730764 21918 raft_consensus.cc:3060] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.732008 21845 raft_consensus.cc:3060] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.736884 21918 raft_consensus.cc:2468] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:41.738783 21735 leader_election.cc:304] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:41.740720 22113 raft_consensus.cc:2804] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:41.740922 21845 raft_consensus.cc:2468] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 2.
I20260709 13:10:41.742120 22113 raft_consensus.cc:697] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:41.743330 22113 consensus_queue.cc:237] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.759608 21668 catalog_manager.cc:5697] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 1 to 2, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:41.793084 22099 raft_consensus.cc:993] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: : Instructing follower 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:41.793810 22112 raft_consensus.cc:1081] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Signalling peer 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:41.795557 21918 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "943224133cb7443597a661a32f71c9ff"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:40760
I20260709 13:10:41.796087 21918 raft_consensus.cc:493] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:41.796391 21918 raft_consensus.cc:3060] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.803225 21918 raft_consensus.cc:515] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.805672 21918 leader_election.cc:290] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 2 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:41.813025 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:41.814056 21770 raft_consensus.cc:3060] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.815192 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "943224133cb7443597a661a32f71c9ff" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:41.815858 21845 raft_consensus.cc:3055] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:41.816181 21845 raft_consensus.cc:740] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:41.817121 21845 consensus_queue.cc:260] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.818679 21845 raft_consensus.cc:3060] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.823558 21770 raft_consensus.cc:2468] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 2.
I20260709 13:10:41.825454 21883 leader_election.cc:304] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:41.826737 22083 raft_consensus.cc:2804] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:41.827641 21845 raft_consensus.cc:2468] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 2.
I20260709 13:10:41.829398 22083 raft_consensus.cc:697] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 2 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:41.830892 22083 consensus_queue.cc:237] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.840497 21668 catalog_manager.cc:5697] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 1 to 2, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 2 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:41.871855 22099 raft_consensus.cc:993] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: : Instructing follower 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:41.872412 22112 raft_consensus.cc:1081] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Signalling peer 4096f29ab5894a7d9119444ce732db3b to start an election
I20260709 13:10:41.874116 21918 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c"
dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
 from {username='slave'} at 127.0.0.1:40760
I20260709 13:10:41.874653 21918 raft_consensus.cc:493] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:41.874997 21918 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.880812 21918 raft_consensus.cc:515] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.882844 21918 leader_election.cc:290] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 5 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:41.884466 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:41.885054 21770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a443c6351ec94cb1ad4ab60d9ff96f1c" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
I20260709 13:10:41.885247 21845 raft_consensus.cc:3055] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:41.885826 21845 raft_consensus.cc:740] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:41.885957 21770 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.887012 21845 consensus_queue.cc:260] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.888715 21845 raft_consensus.cc:3060] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.891538 21770 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 5.
I20260709 13:10:41.892849 21883 leader_election.cc:304] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0; no voters: 
I20260709 13:10:41.893656 22083 raft_consensus.cc:2804] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:41.894130 22083 raft_consensus.cc:697] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 5 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:41.895107 22083 consensus_queue.cc:237] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:41.897857 21845 raft_consensus.cc:2468] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 5.
I20260709 13:10:41.904531 21668 catalog_manager.cc:5697] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 4 to 5, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 5 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.039500 22099 raft_consensus.cc:993] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: : Instructing follower 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:42.039997 22112 raft_consensus.cc:1081] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Signalling peer 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:42.041636 21770 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:10:42.042117 21770 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:42.042428 21770 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:42.046859 21770 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:42.048687 21770 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 4 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:42.049618 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:42.050385 21845 raft_consensus.cc:3055] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:42.050673 21845 raft_consensus.cc:740] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:42.051020 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:42.051607 21918 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:42.051414 21845 consensus_queue.cc:260] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:42.052520 21845 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:42.057237 21845 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 4.
I20260709 13:10:42.058795 21736 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:42.059568 22113 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:42.059899 21918 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 4.
I20260709 13:10:42.060005 22113 raft_consensus.cc:697] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 4 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:42.061129 22113 consensus_queue.cc:237] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:42.070775 21668 catalog_manager.cc:5697] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 3 to 4, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 4 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:42.115756 22099 raft_consensus.cc:993] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: : Instructing follower 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:42.116258 22099 raft_consensus.cc:1081] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Signalling peer 445da1eb4e864722a2e7648ab06210e0 to start an election
I20260709 13:10:42.118098 21770 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc"
dest_uuid: "445da1eb4e864722a2e7648ab06210e0"
 from {username='slave'} at 127.0.0.1:56634
I20260709 13:10:42.118602 21770 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:42.119002 21770 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:42.123785 21770 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:42.125764 21770 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 5 election: Requested vote from peers e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:42.126703 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:42.127318 21845 raft_consensus.cc:3055] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:42.127614 21845 raft_consensus.cc:740] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:42.128417 21845 consensus_queue.cc:260] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:42.130103 21845 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:42.129988 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "445da1eb4e864722a2e7648ab06210e0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:42.131083 21918 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:42.138186 21845 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 5.
I20260709 13:10:42.139706 21736 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf; no voters: 
I20260709 13:10:42.140841 21918 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 445da1eb4e864722a2e7648ab06210e0 in term 5.
I20260709 13:10:42.140793 22113 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:42.141659 22113 raft_consensus.cc:697] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 5 LEADER]: Becoming Leader. State: Replica: 445da1eb4e864722a2e7648ab06210e0, State: Running, Role: LEADER
I20260709 13:10:42.142606 22113 consensus_queue.cc:237] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:42.149562 21845 raft_consensus.cc:1275] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:42.151153 22113 consensus_queue.cc:1048] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:42.151333 21668 catalog_manager.cc:5697] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 reported cstate change: term changed from 4 to 5, leader changed from e7a45570cd8647cfb086e131020776cf (127.21.33.2) to 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1). New cstate: current_term: 5 leader_uuid: "445da1eb4e864722a2e7648ab06210e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:42.180354 21918 raft_consensus.cc:1275] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:42.182065 22113 consensus_queue.cc:1048] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:42.384727 21845 raft_consensus.cc:1275] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:42.386732 22083 consensus_queue.cc:1048] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:42.391393 21846 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:42.393260 22083 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:42.410737 21770 raft_consensus.cc:1275] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:42.420765 22083 consensus_queue.cc:1048] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:42.428301 21771 raft_consensus.cc:1275] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Refusing update from remote peer 4096f29ab5894a7d9119444ce732db3b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:42.430255 22083 consensus_queue.cc:1048] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:10:42.469359 21918 raft_consensus.cc:1275] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:42.471190 22114 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:42.491916 21845 raft_consensus.cc:1275] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:42.494819 22113 consensus_queue.cc:1048] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:42.562862 21845 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:42.564251 22114 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:42.576248 21918 raft_consensus.cc:1275] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Refusing update from remote peer 445da1eb4e864722a2e7648ab06210e0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:42.578315 22138 consensus_queue.cc:1048] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:42.670145 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:42.686132 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
I20260709 13:10:42.745656 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:42.746738 21636 tablet_replica.cc:333] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.748096 21636 raft_consensus.cc:2243] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.748888 21636 raft_consensus.cc:2272] T 4173eaab392c428a9a469cf5722c5b6f P 445da1eb4e864722a2e7648ab06210e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.754053 21636 tablet_replica.cc:333] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.754979 21636 raft_consensus.cc:2243] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.755626 21636 raft_consensus.cc:2272] T 6441660788ea41a4ad81a0a4a2e68422 P 445da1eb4e864722a2e7648ab06210e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.759452 21636 tablet_replica.cc:333] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.760926 21636 raft_consensus.cc:2243] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.761653 21636 raft_consensus.cc:2272] T 5a33ce30108d4326916d4119bb6d09e2 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.765396 21636 tablet_replica.cc:333] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.766297 21636 raft_consensus.cc:2243] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.766937 21636 raft_consensus.cc:2272] T 943224133cb7443597a661a32f71c9ff P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.770677 21636 tablet_replica.cc:333] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.771626 21636 raft_consensus.cc:2243] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:42.772992 21636 raft_consensus.cc:2272] T dcbe798d310f480b9a4f05494e867a2b P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.776664 21636 tablet_replica.cc:333] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.777571 21636 raft_consensus.cc:2243] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.778790 21636 raft_consensus.cc:2272] T a443c6351ec94cb1ad4ab60d9ff96f1c P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.782079 21636 tablet_replica.cc:333] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.783182 21636 raft_consensus.cc:2243] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:42.783941 21636 raft_consensus.cc:2272] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 445da1eb4e864722a2e7648ab06210e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.787500 21636 tablet_replica.cc:333] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.788563 21636 raft_consensus.cc:2243] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:42.789790 21636 raft_consensus.cc:2272] T 76d790a743104ec3be0e678ee2e0ddfc P 445da1eb4e864722a2e7648ab06210e0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:42.793355 21636 tablet_replica.cc:333] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0: stopping tablet replica
I20260709 13:10:42.794627 21636 raft_consensus.cc:2243] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:42.796212 21636 raft_consensus.cc:2272] T c77dc53b15fb4f97905fe230f1954dd0 P 445da1eb4e864722a2e7648ab06210e0 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:42.959854 21883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
W20260709 13:10:42.972117 21809 consensus_peers.cc:597] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf -> Peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Couldn't send request to peer 445da1eb4e864722a2e7648ab06210e0. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:42.977298 21883 consensus_peers.cc:597] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b -> Peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Couldn't send request to peer 445da1eb4e864722a2e7648ab06210e0. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:42.977910 21883 consensus_peers.cc:597] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b -> Peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Couldn't send request to peer 445da1eb4e864722a2e7648ab06210e0. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:42.985749 21883 consensus_peers.cc:597] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b -> Peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Couldn't send request to peer 445da1eb4e864722a2e7648ab06210e0. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:43.007421 21809 consensus_peers.cc:597] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf -> Peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Couldn't send request to peer 445da1eb4e864722a2e7648ab06210e0. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:43.102736 21809 consensus_peers.cc:597] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf -> Peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Couldn't send request to peer 445da1eb4e864722a2e7648ab06210e0. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:43.969353 21636 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21792)
I20260709 13:10:43.989670 22134 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:43.990264 22134 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:43.992341 22134 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:43.993709 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
W20260709 13:10:43.996832 21883 leader_election.cc:336] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:43.997340 21883 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b; no voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf
I20260709 13:10:43.998363 22134 raft_consensus.cc:2749] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:43.998872 22099 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:43.999440 22099 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.002038 22099 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:44.003386 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:44.004175 21918 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 4.
I20260709 13:10:44.005946 21810 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 
W20260709 13:10:44.007175 21809 leader_election.cc:336] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.007438 22099 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:44.008131 22099 raft_consensus.cc:493] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Starting leader election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:44.008710 22099 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:44.015578 22099 raft_consensus.cc:515] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.017405 22099 leader_election.cc:290] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 5 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:44.018762 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77dc53b15fb4f97905fe230f1954dd0" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b"
I20260709 13:10:44.019516 21918 raft_consensus.cc:3060] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:44.021363 21809 leader_election.cc:336] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.028436 21918 raft_consensus.cc:2468] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7a45570cd8647cfb086e131020776cf in term 5.
I20260709 13:10:44.030167 21810 leader_election.cc:304] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:44.031147 22099 raft_consensus.cc:2804] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:44.031687 22099 raft_consensus.cc:697] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 5 LEADER]: Becoming Leader. State: Replica: e7a45570cd8647cfb086e131020776cf, State: Running, Role: LEADER
I20260709 13:10:44.032523 22099 consensus_queue.cc:237] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.041034 21668 catalog_manager.cc:5697] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf reported cstate change: term changed from 4 to 5, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to e7a45570cd8647cfb086e131020776cf (127.21.33.2). New cstate: current_term: 5 leader_uuid: "e7a45570cd8647cfb086e131020776cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:44.076159 22099 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:44.076785 22099 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.078461 22112 raft_consensus.cc:493] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:44.079164 22099 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:44.079067 22112 raft_consensus.cc:515] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.081544 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
I20260709 13:10:44.081733 22112 leader_election.cc:290] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), 4096f29ab5894a7d9119444ce732db3b (127.21.33.3:36061)
I20260709 13:10:44.084060 21918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe798d310f480b9a4f05494e867a2b" candidate_uuid: "e7a45570cd8647cfb086e131020776cf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4096f29ab5894a7d9119444ce732db3b" is_pre_election: true
W20260709 13:10:44.084230 21809 leader_election.cc:336] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.084856 21809 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:44.085103 22134 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:44.085708 22134 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.086612 22112 raft_consensus.cc:2749] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:44.088328 22134 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
W20260709 13:10:44.088950 21809 leader_election.cc:336] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.089704 21809 leader_election.cc:304] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7a45570cd8647cfb086e131020776cf; no voters: 4096f29ab5894a7d9119444ce732db3b, 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:44.090718 22112 raft_consensus.cc:2749] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:44.091610 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf" is_pre_election: true
I20260709 13:10:44.092447 21845 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 5.
W20260709 13:10:44.092902 21883 leader_election.cc:336] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.093708 21885 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:44.094676 22134 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:44.095186 22134 raft_consensus.cc:493] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:44.095934 22134 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:44.105013 22134 raft_consensus.cc:515] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.107579 22134 leader_election.cc:290] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 6 election: Requested vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:44.111018 21845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d790a743104ec3be0e678ee2e0ddfc" candidate_uuid: "4096f29ab5894a7d9119444ce732db3b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7a45570cd8647cfb086e131020776cf"
I20260709 13:10:44.111682 21845 raft_consensus.cc:3060] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:44.114358 21883 leader_election.cc:336] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.120400 21845 raft_consensus.cc:2468] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4096f29ab5894a7d9119444ce732db3b in term 6.
I20260709 13:10:44.121939 21885 leader_election.cc:304] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b, e7a45570cd8647cfb086e131020776cf; no voters: 445da1eb4e864722a2e7648ab06210e0
I20260709 13:10:44.122906 22154 raft_consensus.cc:2804] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:44.123431 22154 raft_consensus.cc:697] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 6 LEADER]: Becoming Leader. State: Replica: 4096f29ab5894a7d9119444ce732db3b, State: Running, Role: LEADER
I20260709 13:10:44.124312 22154 consensus_queue.cc:237] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.132055 21668 catalog_manager.cc:5697] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b reported cstate change: term changed from 5 to 6, leader changed from 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1) to 4096f29ab5894a7d9119444ce732db3b (127.21.33.3). New cstate: current_term: 6 leader_uuid: "4096f29ab5894a7d9119444ce732db3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.144994 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
I20260709 13:10:44.167706 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
I20260709 13:10:44.195438 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:44.196285 21636 tablet_replica.cc:333] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.196985 21636 raft_consensus.cc:2243] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.197765 21636 pending_rounds.cc:70] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: Trying to abort 1 pending ops.
I20260709 13:10:44.198081 21636 pending_rounds.cc:77] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf: Aborting op as it isn't in flight: id { term: 5 index: 5 } timestamp: 7305636429965828096 op_type: NO_OP noop_request { }
I20260709 13:10:44.198439 21636 raft_consensus.cc:2889] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 5 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:44.198753 21636 raft_consensus.cc:2272] T c77dc53b15fb4f97905fe230f1954dd0 P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.200910 21636 tablet_replica.cc:333] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.201460 21636 raft_consensus.cc:2243] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.201939 21636 raft_consensus.cc:2272] T a443c6351ec94cb1ad4ab60d9ff96f1c P e7a45570cd8647cfb086e131020776cf [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.204653 21636 tablet_replica.cc:333] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.205197 21636 raft_consensus.cc:2243] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.206065 21636 raft_consensus.cc:2272] T 6441660788ea41a4ad81a0a4a2e68422 P e7a45570cd8647cfb086e131020776cf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.207799 21636 tablet_replica.cc:333] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.208355 21636 raft_consensus.cc:2243] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.209028 21636 raft_consensus.cc:2272] T 4173eaab392c428a9a469cf5722c5b6f P e7a45570cd8647cfb086e131020776cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.211028 21636 tablet_replica.cc:333] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.211474 21636 raft_consensus.cc:2243] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.211833 21636 raft_consensus.cc:2272] T 7dab60e1b775470cb89f2fa21d3eb6e8 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.213558 21636 tablet_replica.cc:333] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.213995 21636 raft_consensus.cc:2243] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.214326 21636 raft_consensus.cc:2272] T 76d790a743104ec3be0e678ee2e0ddfc P e7a45570cd8647cfb086e131020776cf [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.215783 21636 tablet_replica.cc:333] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.216194 21636 raft_consensus.cc:2243] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.216521 21636 raft_consensus.cc:2272] T 943224133cb7443597a661a32f71c9ff P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.217952 22154 raft_consensus.cc:493] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 445da1eb4e864722a2e7648ab06210e0)
I20260709 13:10:44.218163 21636 tablet_replica.cc:333] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.218820 21636 raft_consensus.cc:2243] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.218490 22154 raft_consensus.cc:515] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445da1eb4e864722a2e7648ab06210e0" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 42151 } } peers { permanent_uuid: "e7a45570cd8647cfb086e131020776cf" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 45401 } } peers { permanent_uuid: "4096f29ab5894a7d9119444ce732db3b" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 36061 } }
I20260709 13:10:44.219785 21636 raft_consensus.cc:2272] T 5a33ce30108d4326916d4119bb6d09e2 P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.220850 22154 leader_election.cc:290] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151), e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401)
I20260709 13:10:44.222561 21636 tablet_replica.cc:333] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf: stopping tablet replica
I20260709 13:10:44.223309 21636 raft_consensus.cc:2243] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.223883 21636 raft_consensus.cc:2272] T dcbe798d310f480b9a4f05494e867a2b P e7a45570cd8647cfb086e131020776cf [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:44.224903 21885 leader_election.cc:336] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:44.225354 21883 leader_election.cc:336] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 445da1eb4e864722a2e7648ab06210e0 (127.21.33.1:42151): Network error: Client connection negotiation failed: client connection to 127.21.33.1:42151: connect: Connection refused (error 111)
I20260709 13:10:44.225822 21883 leader_election.cc:304] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4096f29ab5894a7d9119444ce732db3b; no voters: 445da1eb4e864722a2e7648ab06210e0, e7a45570cd8647cfb086e131020776cf
I20260709 13:10:44.226497 22154 raft_consensus.cc:2749] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:44.256326 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:10:44.278259 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
W20260709 13:10:44.298478 21885 consensus_peers.cc:597] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b -> Peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Couldn't send request to peer e7a45570cd8647cfb086e131020776cf. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.313246 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:44.314260 21636 tablet_replica.cc:333] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
W20260709 13:10:44.314560 21885 consensus_peers.cc:597] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b -> Peer e7a45570cd8647cfb086e131020776cf (127.21.33.2:45401): Couldn't send request to peer e7a45570cd8647cfb086e131020776cf. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:45401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.315335 21636 raft_consensus.cc:2243] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.316686 21636 raft_consensus.cc:2272] T 943224133cb7443597a661a32f71c9ff P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.318673 21636 tablet_replica.cc:333] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.319234 21636 raft_consensus.cc:2243] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.319640 21636 raft_consensus.cc:2272] T dcbe798d310f480b9a4f05494e867a2b P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.321439 21636 tablet_replica.cc:333] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.322057 21636 raft_consensus.cc:2243] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.322803 21636 raft_consensus.cc:2272] T a443c6351ec94cb1ad4ab60d9ff96f1c P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.324544 21636 tablet_replica.cc:333] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.325044 21636 raft_consensus.cc:2243] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.325438 21636 raft_consensus.cc:2272] T 5a33ce30108d4326916d4119bb6d09e2 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.327235 21636 tablet_replica.cc:333] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.327714 21636 raft_consensus.cc:2243] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.328118 21636 raft_consensus.cc:2272] T 6441660788ea41a4ad81a0a4a2e68422 P 4096f29ab5894a7d9119444ce732db3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.329921 21636 tablet_replica.cc:333] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.330504 21636 raft_consensus.cc:2243] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.330969 21636 raft_consensus.cc:2272] T c77dc53b15fb4f97905fe230f1954dd0 P 4096f29ab5894a7d9119444ce732db3b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.332712 21636 tablet_replica.cc:333] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.333225 21636 raft_consensus.cc:2243] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 6 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.333998 21636 pending_rounds.cc:70] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: Trying to abort 1 pending ops.
I20260709 13:10:44.334167 21636 pending_rounds.cc:77] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b: Aborting op as it isn't in flight: id { term: 6 index: 6 } timestamp: 7305636430341980160 op_type: NO_OP noop_request { }
I20260709 13:10:44.334515 21636 raft_consensus.cc:2889] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 6 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:44.334807 21636 raft_consensus.cc:2272] T 76d790a743104ec3be0e678ee2e0ddfc P 4096f29ab5894a7d9119444ce732db3b [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.337056 21636 tablet_replica.cc:333] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.337692 21636 raft_consensus.cc:2243] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.338464 21636 raft_consensus.cc:2272] T 7dab60e1b775470cb89f2fa21d3eb6e8 P 4096f29ab5894a7d9119444ce732db3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.340121 21636 tablet_replica.cc:333] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b: stopping tablet replica
I20260709 13:10:44.340595 21636 raft_consensus.cc:2243] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.341029 21636 raft_consensus.cc:2272] T 4173eaab392c428a9a469cf5722c5b6f P 4096f29ab5894a7d9119444ce732db3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.371265 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:10:44.391860 21636 master.cc:562] Master@127.21.33.62:38757 shutting down...
I20260709 13:10:44.539642 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.540343 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.540706 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 62f9e409586641608c8cceca1073a0ff: stopping tablet replica
I20260709 13:10:44.561115 21636 master.cc:584] Master@127.21.33.62:38757 shutdown complete.
4096f29ab5894a7d9119444ce732db3b  1
445da1eb4e864722a2e7648ab06210e0  4
e7a45570cd8647cfb086e131020776cf  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (14586 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:10:44.609138 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:37807
I20260709 13:10:44.610735 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:44.617174 22156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:44.618546 22159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:44.619186 22157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:44.619439 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:44.620497 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:44.620720 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:44.620903 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602644620881 us; error 0 us; skew 500 ppm
I20260709 13:10:44.621636 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:44.624914 21636 webserver.cc:533] Webserver started at http://127.21.33.62:45345/ using document root <none> and password file <none>
I20260709 13:10:44.625579 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:44.625800 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:44.626120 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:44.627326 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "4fa5ade16aa14d1098b97ad7a8bc938c"
format_stamp: "Formatted at 2026-07-09 13:10:44 on dist-test-slave-p8rg"
I20260709 13:10:44.631947 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:44.635494 22164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:44.636318 21636 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:44.636672 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "4fa5ade16aa14d1098b97ad7a8bc938c"
format_stamp: "Formatted at 2026-07-09 13:10:44 on dist-test-slave-p8rg"
I20260709 13:10:44.636981 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:44.673262 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:44.674728 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:44.722381 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:37807
I20260709 13:10:44.722482 22215 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:37807 every 8 connection(s)
I20260709 13:10:44.726897 22216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:44.737213 22216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c: Bootstrap starting.
I20260709 13:10:44.742574 22216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:44.747063 22216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c: No bootstrap required, opened a new log
I20260709 13:10:44.749243 22216 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" member_type: VOTER }
I20260709 13:10:44.749714 22216 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:44.749930 22216 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa5ade16aa14d1098b97ad7a8bc938c, State: Initialized, Role: FOLLOWER
I20260709 13:10:44.750560 22216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" member_type: VOTER }
I20260709 13:10:44.751010 22216 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:44.751240 22216 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:44.751523 22216 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:44.755877 22216 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" member_type: VOTER }
I20260709 13:10:44.756402 22216 leader_election.cc:304] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fa5ade16aa14d1098b97ad7a8bc938c; no voters: 
I20260709 13:10:44.757902 22216 leader_election.cc:290] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:44.758337 22219 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:44.759826 22219 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 1 LEADER]: Becoming Leader. State: Replica: 4fa5ade16aa14d1098b97ad7a8bc938c, State: Running, Role: LEADER
I20260709 13:10:44.760556 22219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" member_type: VOTER }
I20260709 13:10:44.761332 22216 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:44.765318 22220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" member_type: VOTER } }
I20260709 13:10:44.765493 22221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fa5ade16aa14d1098b97ad7a8bc938c. Latest consensus state: current_term: 1 leader_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa5ade16aa14d1098b97ad7a8bc938c" member_type: VOTER } }
I20260709 13:10:44.767068 22221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:44.766108 22220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:44.770507 22227 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:44.779224 22227 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:44.780334 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:44.788700 22227 catalog_manager.cc:1383] Generated new cluster ID: b87656195fe442a09dcebb0da65ebeda
I20260709 13:10:44.788987 22227 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:44.822877 22227 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:44.824388 22227 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:44.839872 22227 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c: Generated new TSK 0
I20260709 13:10:44.840557 22227 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:44.847055 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:44.853734 22237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:44.855683 22238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:44.859026 22240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:44.863029 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:44.864281 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:44.864526 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:44.864764 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602644864717 us; error 0 us; skew 500 ppm
I20260709 13:10:44.865356 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:44.868506 21636 webserver.cc:533] Webserver started at http://127.21.33.1:41221/ using document root <none> and password file <none>
I20260709 13:10:44.869096 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:44.869303 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:44.869664 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:44.870957 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
format_stamp: "Formatted at 2026-07-09 13:10:44 on dist-test-slave-p8rg"
I20260709 13:10:44.876220 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:10:44.879954 22245 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:44.880941 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:10:44.881342 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
format_stamp: "Formatted at 2026-07-09 13:10:44 on dist-test-slave-p8rg"
I20260709 13:10:44.881728 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:44.904868 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:44.907020 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:44.909631 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:44.912320 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:44.912569 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:44.912793 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:44.912951 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:44.962802 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:38863
I20260709 13:10:44.963028 22307 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:38863 every 8 connection(s)
I20260709 13:10:44.968348 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:44.976130 22312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:44.979555 22313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:44.982369 21636 server_base.cc:1061] running on GCE node
W20260709 13:10:44.982525 22315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:44.984041 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:44.984261 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:44.984517 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602644984473 us; error 0 us; skew 500 ppm
I20260709 13:10:44.985199 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:44.988674 21636 webserver.cc:533] Webserver started at http://127.21.33.2:45205/ using document root <none> and password file <none>
I20260709 13:10:44.989408 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:44.989743 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:44.989796 22308 heartbeater.cc:344] Connected to a master server at 127.21.33.62:37807
I20260709 13:10:44.990180 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:44.990263 22308 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:44.991318 22308 heartbeater.cc:507] Master 127.21.33.62:37807 requested a full tablet report, sending...
I20260709 13:10:44.992036 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "4c791639cce64ab489670508e36f682a"
format_stamp: "Formatted at 2026-07-09 13:10:44 on dist-test-slave-p8rg"
I20260709 13:10:44.994377 22181 ts_manager.cc:194] Registered new tserver with Master: 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863)
I20260709 13:10:44.997421 22181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55512
I20260709 13:10:44.999408 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:10:45.003733 22320 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.004498 21636 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:10:45.004835 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "4c791639cce64ab489670508e36f682a"
format_stamp: "Formatted at 2026-07-09 13:10:44 on dist-test-slave-p8rg"
I20260709 13:10:45.005254 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:45.026378 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:45.027930 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:45.029754 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:45.032377 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:45.032635 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.032919 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:45.033119 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.090209 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:36473
I20260709 13:10:45.090342 22382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:36473 every 8 connection(s)
I20260709 13:10:45.098553 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:45.108649 22386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.112115 22387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.113030 22383 heartbeater.cc:344] Connected to a master server at 127.21.33.62:37807
I20260709 13:10:45.113487 22383 heartbeater.cc:461] Registering TS with master...
W20260709 13:10:45.114128 22389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.114646 22383 heartbeater.cc:507] Master 127.21.33.62:37807 requested a full tablet report, sending...
I20260709 13:10:45.115528 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:45.116454 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:45.116750 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:45.116953 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602645116933 us; error 0 us; skew 500 ppm
I20260709 13:10:45.117276 22181 ts_manager.cc:194] Registered new tserver with Master: 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:45.117725 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:45.119390 22181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55522
I20260709 13:10:45.121212 21636 webserver.cc:533] Webserver started at http://127.21.33.3:34167/ using document root <none> and password file <none>
I20260709 13:10:45.121975 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:45.122236 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:45.122504 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:45.123721 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "f26944a219514330933cbf63a91527d1"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-p8rg"
I20260709 13:10:45.128935 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:10:45.133605 22394 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.134908 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:10:45.135457 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "f26944a219514330933cbf63a91527d1"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-p8rg"
I20260709 13:10:45.135743 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:45.163652 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:45.165086 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:45.166743 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:45.169646 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:45.169858 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.170159 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:45.170428 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.229550 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:38091
I20260709 13:10:45.229779 22456 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:38091 every 8 connection(s)
I20260709 13:10:45.246070 22457 heartbeater.cc:344] Connected to a master server at 127.21.33.62:37807
I20260709 13:10:45.246564 22457 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:45.247363 22457 heartbeater.cc:507] Master 127.21.33.62:37807 requested a full tablet report, sending...
I20260709 13:10:45.249567 22181 ts_manager.cc:194] Registered new tserver with Master: f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:45.250555 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017110637s
I20260709 13:10:45.251557 22181 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55536
I20260709 13:10:45.283324 22181 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55550:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:45.286549 22181 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:45.375913 22422 tablet_service.cc:1511] Processing CreateTablet for tablet 34c135a95e524c95954daed5cce26c4b (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:45.377799 22422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c135a95e524c95954daed5cce26c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.381211 22421 tablet_service.cc:1511] Processing CreateTablet for tablet f02b85578a6140ab8d3bb048afec6c95 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:45.383010 22421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02b85578a6140ab8d3bb048afec6c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.398473 22273 tablet_service.cc:1511] Processing CreateTablet for tablet 34c135a95e524c95954daed5cce26c4b (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:45.398473 22420 tablet_service.cc:1511] Processing CreateTablet for tablet b3f31fd90b28408bb19233cd0d171cae (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:45.401959 22420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3f31fd90b28408bb19233cd0d171cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.403095 22418 tablet_service.cc:1511] Processing CreateTablet for tablet ce2b70e6efcb4ce383dcb4288dbb3c03 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:45.405192 22418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce2b70e6efcb4ce383dcb4288dbb3c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.400266 22419 tablet_service.cc:1511] Processing CreateTablet for tablet 4385d13d29c04839b94a9c984bef179c (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:45.406981 22419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4385d13d29c04839b94a9c984bef179c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.400266 22273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c135a95e524c95954daed5cce26c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.425333 22472 tablet_bootstrap.cc:492] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:45.425536 22417 tablet_service.cc:1511] Processing CreateTablet for tablet acaac9b2b1b94d16bd89a6696dd3de46 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:45.427193 22417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acaac9b2b1b94d16bd89a6696dd3de46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.426945 22272 tablet_service.cc:1511] Processing CreateTablet for tablet f02b85578a6140ab8d3bb048afec6c95 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:45.430987 22416 tablet_service.cc:1511] Processing CreateTablet for tablet 17b472ae65ee42e2bfed4a439cf4b88c (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:45.432723 22416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b472ae65ee42e2bfed4a439cf4b88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.434324 22344 tablet_service.cc:1511] Processing CreateTablet for tablet ce2b70e6efcb4ce383dcb4288dbb3c03 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:45.434828 22414 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfba1fdc47d42b796048719a3ec7488 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:45.435923 22344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce2b70e6efcb4ce383dcb4288dbb3c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.435981 22268 tablet_service.cc:1511] Processing CreateTablet for tablet 46386a22c8cf46acae0712ef631824c3 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:45.437534 22348 tablet_service.cc:1511] Processing CreateTablet for tablet 34c135a95e524c95954daed5cce26c4b (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:45.438666 22343 tablet_service.cc:1511] Processing CreateTablet for tablet 46386a22c8cf46acae0712ef631824c3 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:45.440115 22347 tablet_service.cc:1511] Processing CreateTablet for tablet f02b85578a6140ab8d3bb048afec6c95 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:45.441140 22342 tablet_service.cc:1511] Processing CreateTablet for tablet acaac9b2b1b94d16bd89a6696dd3de46 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:45.442198 22264 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfba1fdc47d42b796048719a3ec7488 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:45.442653 22345 tablet_service.cc:1511] Processing CreateTablet for tablet 4385d13d29c04839b94a9c984bef179c (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:45.428488 22272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02b85578a6140ab8d3bb048afec6c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.443468 22341 tablet_service.cc:1511] Processing CreateTablet for tablet 17b472ae65ee42e2bfed4a439cf4b88c (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:45.445441 22415 tablet_service.cc:1511] Processing CreateTablet for tablet 1221f647ca7c40a3b50c4d3414a33e89 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:45.446190 22340 tablet_service.cc:1511] Processing CreateTablet for tablet 1221f647ca7c40a3b50c4d3414a33e89 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:45.431046 22271 tablet_service.cc:1511] Processing CreateTablet for tablet b3f31fd90b28408bb19233cd0d171cae (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:45.447770 22472 tablet_bootstrap.cc:654] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.433184 22270 tablet_service.cc:1511] Processing CreateTablet for tablet 4385d13d29c04839b94a9c984bef179c (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:45.449788 22267 tablet_service.cc:1511] Processing CreateTablet for tablet acaac9b2b1b94d16bd89a6696dd3de46 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:45.450130 22269 tablet_service.cc:1511] Processing CreateTablet for tablet ce2b70e6efcb4ce383dcb4288dbb3c03 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:45.451805 22346 tablet_service.cc:1511] Processing CreateTablet for tablet b3f31fd90b28408bb19233cd0d171cae (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:45.452164 22339 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfba1fdc47d42b796048719a3ec7488 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:45.440124 22265 tablet_service.cc:1511] Processing CreateTablet for tablet 1221f647ca7c40a3b50c4d3414a33e89 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:45.438380 22266 tablet_service.cc:1511] Processing CreateTablet for tablet 17b472ae65ee42e2bfed4a439cf4b88c (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:45.456626 22422 tablet_service.cc:1511] Processing CreateTablet for tablet 46386a22c8cf46acae0712ef631824c3 (DEFAULT_TABLE table=test-workload [id=867cbfaef51245bbb1bb2af62c252f62]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:45.457191 22264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfba1fdc47d42b796048719a3ec7488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.458148 22267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acaac9b2b1b94d16bd89a6696dd3de46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.459111 22266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b472ae65ee42e2bfed4a439cf4b88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.459947 22270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4385d13d29c04839b94a9c984bef179c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.461948 22271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3f31fd90b28408bb19233cd0d171cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.463466 22265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1221f647ca7c40a3b50c4d3414a33e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.464567 22269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce2b70e6efcb4ce383dcb4288dbb3c03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.464815 22348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c135a95e524c95954daed5cce26c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.465893 22347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02b85578a6140ab8d3bb048afec6c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.468667 22340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1221f647ca7c40a3b50c4d3414a33e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.469549 22268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46386a22c8cf46acae0712ef631824c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.469880 22342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acaac9b2b1b94d16bd89a6696dd3de46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.436143 22414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfba1fdc47d42b796048719a3ec7488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.472031 22341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b472ae65ee42e2bfed4a439cf4b88c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.477928 22345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4385d13d29c04839b94a9c984bef179c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.481616 22415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1221f647ca7c40a3b50c4d3414a33e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.483065 22343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46386a22c8cf46acae0712ef631824c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.485900 22339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfba1fdc47d42b796048719a3ec7488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.497268 22422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46386a22c8cf46acae0712ef631824c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.550876 22346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3f31fd90b28408bb19233cd0d171cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.611851 22472 tablet_bootstrap.cc:492] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:45.612675 22472 ts_tablet_manager.cc:1403] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.188s	user 0.025s	sys 0.026s
I20260709 13:10:45.618304 22474 tablet_bootstrap.cc:492] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:45.624086 22475 tablet_bootstrap.cc:492] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:45.638154 22474 tablet_bootstrap.cc:654] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.647159 22472 raft_consensus.cc:359] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.648151 22472 raft_consensus.cc:385] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.648535 22472 raft_consensus.cc:740] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.661230 22472 consensus_queue.cc:260] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.663431 22474 tablet_bootstrap.cc:492] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:45.663714 22475 tablet_bootstrap.cc:654] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.664181 22474 ts_tablet_manager.cc:1403] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.000s
I20260709 13:10:45.668972 22457 heartbeater.cc:499] Master 127.21.33.62:37807 was elected leader, sending a full tablet report...
I20260709 13:10:45.668823 22474 raft_consensus.cc:359] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.669757 22474 raft_consensus.cc:385] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.670156 22475 tablet_bootstrap.cc:492] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:45.670118 22474 raft_consensus.cc:740] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.670770 22475 ts_tablet_manager.cc:1403] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.005s
I20260709 13:10:45.670725 22472 ts_tablet_manager.cc:1434] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.058s	user 0.013s	sys 0.005s
I20260709 13:10:45.671128 22474 consensus_queue.cc:260] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.672888 22472 tablet_bootstrap.cc:492] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:45.674526 22475 raft_consensus.cc:359] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.675283 22475 raft_consensus.cc:385] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.675590 22475 raft_consensus.cc:740] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.676400 22475 consensus_queue.cc:260] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.682082 22472 tablet_bootstrap.cc:654] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.696959 22383 heartbeater.cc:499] Master 127.21.33.62:37807 was elected leader, sending a full tablet report...
I20260709 13:10:45.698845 22308 heartbeater.cc:499] Master 127.21.33.62:37807 was elected leader, sending a full tablet report...
I20260709 13:10:45.704762 22481 raft_consensus.cc:493] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.705406 22481 raft_consensus.cc:515] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.708539 22474 ts_tablet_manager.cc:1434] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.044s	user 0.012s	sys 0.010s
I20260709 13:10:45.709867 22474 tablet_bootstrap.cc:492] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:45.721524 22481 leader_election.cc:290] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:45.724089 22474 tablet_bootstrap.cc:654] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.726243 22475 ts_tablet_manager.cc:1434] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.055s	user 0.017s	sys 0.024s
I20260709 13:10:45.731804 22472 tablet_bootstrap.cc:492] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:45.732606 22472 ts_tablet_manager.cc:1403] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.018s
I20260709 13:10:45.737118 22472 raft_consensus.cc:359] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.738085 22472 raft_consensus.cc:385] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.738425 22472 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.739385 22472 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.740167 22475 tablet_bootstrap.cc:492] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:45.740810 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:45.742259 22472 ts_tablet_manager.cc:1434] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
W20260709 13:10:45.742673 22322 leader_election.cc:343] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:45.743392 22472 tablet_bootstrap.cc:492] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:45.747064 22474 tablet_bootstrap.cc:492] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:45.747766 22474 ts_tablet_manager.cc:1403] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.005s
I20260709 13:10:45.748549 22475 tablet_bootstrap.cc:654] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.751482 22474 raft_consensus.cc:359] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.752363 22474 raft_consensus.cc:385] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.752723 22474 raft_consensus.cc:740] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.752669 22472 tablet_bootstrap.cc:654] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.753787 22474 consensus_queue.cc:260] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.756364 22474 ts_tablet_manager.cc:1434] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:10:45.757413 22474 tablet_bootstrap.cc:492] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:45.762403 22475 tablet_bootstrap.cc:492] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:45.763257 22475 ts_tablet_manager.cc:1403] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.011s
I20260709 13:10:45.765836 22474 tablet_bootstrap.cc:654] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.766822 22472 tablet_bootstrap.cc:492] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:45.766906 22475 raft_consensus.cc:359] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.767416 22472 ts_tablet_manager.cc:1403] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 13:10:45.767737 22475 raft_consensus.cc:385] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.768115 22475 raft_consensus.cc:740] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.768934 22475 consensus_queue.cc:260] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.770787 22472 raft_consensus.cc:359] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.771248 22475 ts_tablet_manager.cc:1434] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:45.771564 22472 raft_consensus.cc:385] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.771863 22472 raft_consensus.cc:740] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.772279 22475 tablet_bootstrap.cc:492] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
W20260709 13:10:45.773592 22229 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:45.773568 22472 consensus_queue.cc:260] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.779759 22475 tablet_bootstrap.cc:654] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.780903 22472 ts_tablet_manager.cc:1434] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 13:10:45.782127 22472 tablet_bootstrap.cc:492] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:45.783342 22474 tablet_bootstrap.cc:492] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:45.784096 22474 ts_tablet_manager.cc:1403] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.015s
I20260709 13:10:45.786985 22479 raft_consensus.cc:493] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.787545 22479 raft_consensus.cc:515] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.790463 22479 leader_election.cc:290] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f26944a219514330933cbf63a91527d1 (127.21.33.3:38091), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:45.795787 22477 raft_consensus.cc:493] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.796902 22477 raft_consensus.cc:515] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.803218 22477 leader_election.cc:290] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:45.794697 22474 raft_consensus.cc:359] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.808609 22474 raft_consensus.cc:385] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.809108 22474 raft_consensus.cc:740] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.810191 22474 consensus_queue.cc:260] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.818408 22472 tablet_bootstrap.cc:654] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.824489 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:45.813421 22474 ts_tablet_manager.cc:1434] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:10:45.826923 22474 tablet_bootstrap.cc:492] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
W20260709 13:10:45.827342 22321 leader_election.cc:343] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f26944a219514330933cbf63a91527d1 (127.21.33.3:38091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:45.827996 22321 leader_election.cc:304] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c791639cce64ab489670508e36f682a; no voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1
I20260709 13:10:45.829650 22481 raft_consensus.cc:2749] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:45.833595 22475 tablet_bootstrap.cc:492] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:45.834399 22475 ts_tablet_manager.cc:1403] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.062s	user 0.027s	sys 0.035s
I20260709 13:10:45.844375 22477 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.845315 22477 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.848322 22477 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:45.844583 22475 raft_consensus.cc:359] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.853480 22475 raft_consensus.cc:385] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.861241 22472 tablet_bootstrap.cc:492] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:45.862162 22472 ts_tablet_manager.cc:1403] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.006s
I20260709 13:10:45.865932 22481 raft_consensus.cc:493] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.866652 22481 raft_consensus.cc:515] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.871425 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:45.872316 22432 raft_consensus.cc:2468] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:45.874429 22321 leader_election.cc:304] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:45.875537 22480 raft_consensus.cc:2804] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:45.875962 22480 raft_consensus.cc:493] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.876194 22481 leader_election.cc:290] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f26944a219514330933cbf63a91527d1 (127.21.33.3:38091), 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863)
I20260709 13:10:45.877599 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:45.878366 22283 raft_consensus.cc:2468] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:45.881386 22479 raft_consensus.cc:493] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.876302 22480 raft_consensus.cc:3060] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:45.881994 22479 raft_consensus.cc:515] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.886159 22479 leader_election.cc:290] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:45.887518 22474 tablet_bootstrap.cc:654] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.893370 22480 raft_consensus.cc:515] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.897540 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:45.898180 22432 raft_consensus.cc:3060] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:45.898727 22472 raft_consensus.cc:359] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.899466 22472 raft_consensus.cc:385] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.899775 22472 raft_consensus.cc:740] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.856819 22475 raft_consensus.cc:740] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.897557 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:45.904395 22475 consensus_queue.cc:260] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.910918 22475 ts_tablet_manager.cc:1434] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.000s
I20260709 13:10:45.912155 22475 tablet_bootstrap.cc:492] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:45.915561 22358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:45.906322 22283 raft_consensus.cc:3060] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:45.924844 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:45.935458 22355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:45.936367 22355 raft_consensus.cc:2468] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 0.
I20260709 13:10:45.937551 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:45.938498 22356 raft_consensus.cc:2393] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in current term 1: Already voted for candidate 4c791639cce64ab489670508e36f682a in this term.
W20260709 13:10:45.941613 22395 leader_election.cc:343] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:45.942395 22395 leader_election.cc:343] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:45.944568 22475 tablet_bootstrap.cc:654] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.944654 22432 raft_consensus.cc:2468] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:45.903537 22480 leader_election.cc:290] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Requested vote from peers f26944a219514330933cbf63a91527d1 (127.21.33.3:38091), 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863)
I20260709 13:10:45.908588 22474 tablet_bootstrap.cc:492] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:45.904103 22472 consensus_queue.cc:260] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.950562 22246 leader_election.cc:304] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:45.951570 22479 raft_consensus.cc:2804] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:45.952004 22479 raft_consensus.cc:493] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.952386 22479 raft_consensus.cc:3060] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:45.952798 22283 raft_consensus.cc:2468] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:45.954310 22472 ts_tablet_manager.cc:1434] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.001s
I20260709 13:10:45.954264 22322 leader_election.cc:304] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:45.955464 22472 tablet_bootstrap.cc:492] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:45.957808 22474 ts_tablet_manager.cc:1403] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.131s	user 0.017s	sys 0.003s
I20260709 13:10:45.960397 22479 raft_consensus.cc:515] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.961288 22480 raft_consensus.cc:2804] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:45.961880 22480 raft_consensus.cc:697] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:45.962256 22472 tablet_bootstrap.cc:654] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.963039 22480 consensus_queue.cc:237] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:45.963482 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:45.963745 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:45.964520 22432 raft_consensus.cc:2393] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in current term 1: Already voted for candidate 4c791639cce64ab489670508e36f682a in this term.
I20260709 13:10:45.965090 22474 raft_consensus.cc:359] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.972002 22474 raft_consensus.cc:385] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.968114 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:45.973708 22474 raft_consensus.cc:740] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
W20260709 13:10:45.969485 22396 leader_election.cc:343] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:45.974715 22472 tablet_bootstrap.cc:492] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:45.974625 22396 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:45.975420 22472 ts_tablet_manager.cc:1403] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 13:10:45.974658 22474 consensus_queue.cc:260] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.976388 22477 raft_consensus.cc:2749] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:45.970793 22479 leader_election.cc:290] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:45.971235 22475 tablet_bootstrap.cc:492] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:45.969314 22246 leader_election.cc:304] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1
I20260709 13:10:45.965950 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:45.979409 22475 ts_tablet_manager.cc:1403] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.001s
I20260709 13:10:45.979352 22472 raft_consensus.cc:359] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.980319 22472 raft_consensus.cc:385] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.980705 22472 raft_consensus.cc:740] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.981909 22472 consensus_queue.cc:260] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.983664 22475 raft_consensus.cc:359] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.984658 22475 raft_consensus.cc:385] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.985045 22475 raft_consensus.cc:740] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.989213 22472 ts_tablet_manager.cc:1434] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:10:45.991920 22477 raft_consensus.cc:493] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.992652 22477 raft_consensus.cc:515] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.986423 22475 consensus_queue.cc:260] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:45.999662 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfba1fdc47d42b796048719a3ec7488" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.002839 22477 leader_election.cc:290] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
W20260709 13:10:46.006043 22395 leader_election.cc:343] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.003867 22479 raft_consensus.cc:2749] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:46.013460 22472 tablet_bootstrap.cc:492] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:46.007921 22475 ts_tablet_manager.cc:1434] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.002s
I20260709 13:10:46.019158 22475 tablet_bootstrap.cc:492] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
W20260709 13:10:46.022725 22246 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:38091, user_credentials={real_user=slave}} blocked reactor thread for 53435.4us
I20260709 13:10:46.025084 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:46.025748 22357 raft_consensus.cc:3060] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.027287 22474 ts_tablet_manager.cc:1434] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.069s	user 0.025s	sys 0.024s
I20260709 13:10:46.028265 22475 tablet_bootstrap.cc:654] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.028539 22474 tablet_bootstrap.cc:492] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:46.029745 22178 catalog_manager.cc:5697] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 1 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.033744 22357 raft_consensus.cc:2468] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 1.
I20260709 13:10:46.036093 22472 tablet_bootstrap.cc:654] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.036855 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
W20260709 13:10:46.039115 22396 leader_election.cc:343] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.039830 22396 leader_election.cc:304] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.036090 22474 tablet_bootstrap.cc:654] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.042053 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfba1fdc47d42b796048719a3ec7488" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:46.042754 22283 raft_consensus.cc:2468] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 0.
W20260709 13:10:46.043980 22246 leader_election.cc:343] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f26944a219514330933cbf63a91527d1 (127.21.33.3:38091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.045169 22477 raft_consensus.cc:2749] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:46.045480 22246 leader_election.cc:343] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f26944a219514330933cbf63a91527d1 (127.21.33.3:38091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.048086 22396 leader_election.cc:304] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a
I20260709 13:10:46.049435 22477 raft_consensus.cc:2804] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.049612 22475 tablet_bootstrap.cc:492] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:46.049871 22477 raft_consensus.cc:493] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.050163 22475 ts_tablet_manager.cc:1403] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.004s
I20260709 13:10:46.050221 22477 raft_consensus.cc:3060] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.056882 22472 tablet_bootstrap.cc:492] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:46.057561 22472 ts_tablet_manager.cc:1403] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.010s
I20260709 13:10:46.057107 22475 raft_consensus.cc:359] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.058327 22475 raft_consensus.cc:385] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.058771 22475 raft_consensus.cc:740] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.059827 22475 consensus_queue.cc:260] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.061550 22472 raft_consensus.cc:359] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.063403 22472 raft_consensus.cc:385] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.063727 22472 raft_consensus.cc:740] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.064628 22472 consensus_queue.cc:260] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.065428 22474 tablet_bootstrap.cc:492] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:46.066761 22477 raft_consensus.cc:515] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.067466 22475 ts_tablet_manager.cc:1434] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:10:46.069419 22475 tablet_bootstrap.cc:492] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:46.070240 22477 leader_election.cc:290] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:46.066536 22474 ts_tablet_manager.cc:1403] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.010s
I20260709 13:10:46.070926 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfba1fdc47d42b796048719a3ec7488" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:46.072271 22357 raft_consensus.cc:3060] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.077237 22474 raft_consensus.cc:359] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.078166 22474 raft_consensus.cc:385] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.078529 22474 raft_consensus.cc:740] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.079859 22474 consensus_queue.cc:260] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.080610 22475 tablet_bootstrap.cc:654] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.071796 22246 leader_election.cc:304] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: f26944a219514330933cbf63a91527d1
I20260709 13:10:46.084736 22474 ts_tablet_manager.cc:1434] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:10:46.085132 22472 ts_tablet_manager.cc:1434] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.013s
I20260709 13:10:46.085902 22474 tablet_bootstrap.cc:492] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:46.087180 22357 raft_consensus.cc:2468] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 1.
I20260709 13:10:46.088706 22395 leader_election.cc:304] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:46.090268 22472 tablet_bootstrap.cc:492] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:46.090497 22501 raft_consensus.cc:2804] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.091053 22501 raft_consensus.cc:697] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:46.092062 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfba1fdc47d42b796048719a3ec7488" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:46.092799 22283 raft_consensus.cc:3060] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.098052 22508 raft_consensus.cc:2804] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.098667 22508 raft_consensus.cc:697] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 1 LEADER]: Becoming Leader. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:46.099615 22508 consensus_queue.cc:237] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.092082 22501 consensus_queue.cc:237] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.104976 22472 tablet_bootstrap.cc:654] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.110821 22474 tablet_bootstrap.cc:654] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.128002 22474 tablet_bootstrap.cc:492] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:46.128813 22474 ts_tablet_manager.cc:1403] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.006s
I20260709 13:10:46.141850 22474 raft_consensus.cc:359] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.147716 22475 tablet_bootstrap.cc:492] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:46.148447 22475 ts_tablet_manager.cc:1403] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.042s
I20260709 13:10:46.125293 22283 raft_consensus.cc:2468] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 1.
I20260709 13:10:46.143002 22474 raft_consensus.cc:385] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.150667 22474 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.156778 22475 raft_consensus.cc:359] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.157693 22472 tablet_bootstrap.cc:492] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:46.157887 22475 raft_consensus.cc:385] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.156780 22179 catalog_manager.cc:5697] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f26944a219514330933cbf63a91527d1 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f26944a219514330933cbf63a91527d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:46.158239 22472 ts_tablet_manager.cc:1403] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.068s	user 0.034s	sys 0.018s
I20260709 13:10:46.158309 22475 raft_consensus.cc:740] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.159588 22475 consensus_queue.cc:260] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.162392 22475 ts_tablet_manager.cc:1434] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.006s
I20260709 13:10:46.161890 22474 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.162623 22472 raft_consensus.cc:359] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.163399 22472 raft_consensus.cc:385] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.163579 22475 tablet_bootstrap.cc:492] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:46.163705 22472 raft_consensus.cc:740] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.164760 22474 ts_tablet_manager.cc:1434] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.035s	user 0.002s	sys 0.010s
I20260709 13:10:46.164517 22472 consensus_queue.cc:260] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.165900 22474 tablet_bootstrap.cc:492] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:46.166885 22472 ts_tablet_manager.cc:1434] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:46.167981 22472 tablet_bootstrap.cc:492] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:46.171943 22475 tablet_bootstrap.cc:654] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.173826 22474 tablet_bootstrap.cc:654] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.176581 22472 tablet_bootstrap.cc:654] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.185686 22474 tablet_bootstrap.cc:492] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:46.186410 22474 ts_tablet_manager.cc:1403] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.007s
I20260709 13:10:46.190910 22474 raft_consensus.cc:359] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.191720 22474 raft_consensus.cc:385] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.192034 22474 raft_consensus.cc:740] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.194069 22475 tablet_bootstrap.cc:492] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:46.194633 22475 ts_tablet_manager.cc:1403] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.018s
I20260709 13:10:46.194281 22178 catalog_manager.cc:5697] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.195412 22474 consensus_queue.cc:260] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.196704 22472 tablet_bootstrap.cc:492] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:46.197230 22472 ts_tablet_manager.cc:1403] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.016s
I20260709 13:10:46.197834 22475 raft_consensus.cc:359] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.198369 22474 ts_tablet_manager.cc:1434] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:10:46.198581 22475 raft_consensus.cc:385] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.198931 22475 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.199509 22474 tablet_bootstrap.cc:492] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:46.199846 22475 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.200493 22472 raft_consensus.cc:359] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.201272 22472 raft_consensus.cc:385] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.201629 22472 raft_consensus.cc:740] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.202440 22475 ts_tablet_manager.cc:1434] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:10:46.202486 22472 consensus_queue.cc:260] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.204041 22475 tablet_bootstrap.cc:492] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:46.204931 22472 ts_tablet_manager.cc:1434] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:10:46.205921 22472 tablet_bootstrap.cc:492] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:46.207201 22501 raft_consensus.cc:493] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.207721 22501 raft_consensus.cc:515] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.209090 22480 raft_consensus.cc:493] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.209663 22480 raft_consensus.cc:515] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.210379 22501 leader_election.cc:290] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.212626 22472 tablet_bootstrap.cc:654] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.212777 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.213650 22432 raft_consensus.cc:2468] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 0.
I20260709 13:10:46.211760 22480 leader_election.cc:290] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.212069 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.214905 22474 tablet_bootstrap.cc:654] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.214684 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
W20260709 13:10:46.216084 22246 leader_election.cc:343] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:46.216348 22322 leader_election.cc:343] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.216826 22246 leader_election.cc:304] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a
I20260709 13:10:46.217103 22475 tablet_bootstrap.cc:654] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.218129 22501 raft_consensus.cc:2804] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.218634 22501 raft_consensus.cc:493] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.219086 22501 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.224152 22508 raft_consensus.cc:493] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.224671 22508 raft_consensus.cc:515] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.225051 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.225688 22432 raft_consensus.cc:2468] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.225921 22477 raft_consensus.cc:493] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.226473 22477 raft_consensus.cc:515] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.226950 22508 leader_election.cc:290] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:46.227314 22321 leader_election.cc:304] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.228391 22480 raft_consensus.cc:2804] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.228765 22480 raft_consensus.cc:493] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.229281 22480 raft_consensus.cc:3060] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.230913 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.231812 22356 raft_consensus.cc:2393] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f26944a219514330933cbf63a91527d1 in current term 1: Already voted for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in this term.
I20260709 13:10:46.232457 22481 raft_consensus.cc:493] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.232959 22481 raft_consensus.cc:515] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.228812 22477 leader_election.cc:290] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:46.234964 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:46.235755 22481 leader_election.cc:290] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
W20260709 13:10:46.236356 22396 leader_election.cc:343] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.236771 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:46.239814 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:46.240736 22282 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.241248 22480 raft_consensus.cc:515] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.241542 22396 leader_election.cc:304] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.242362 22477 raft_consensus.cc:3060] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.242249 22322 leader_election.cc:304] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:46.229213 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.245294 22472 tablet_bootstrap.cc:492] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:46.245877 22472 ts_tablet_manager.cc:1403] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.040s	user 0.003s	sys 0.015s
I20260709 13:10:46.246157 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
W20260709 13:10:46.247777 22395 leader_election.cc:343] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:46.248324 22321 leader_election.cc:343] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f26944a219514330933cbf63a91527d1 (127.21.33.3:38091): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.248284 22395 leader_election.cc:304] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.249312 22481 raft_consensus.cc:2804] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.249581 22508 raft_consensus.cc:2749] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:46.249224 22472 raft_consensus.cc:359] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.250222 22472 raft_consensus.cc:385] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.250568 22472 raft_consensus.cc:740] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.249716 22477 raft_consensus.cc:2749] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:46.251441 22481 raft_consensus.cc:493] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.251397 22472 consensus_queue.cc:260] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.252100 22481 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.253856 22480 leader_election.cc:290] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.253979 22472 ts_tablet_manager.cc:1434] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:10:46.255149 22474 tablet_bootstrap.cc:492] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:46.255542 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:46.255834 22472 tablet_bootstrap.cc:492] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1: Bootstrap starting.
I20260709 13:10:46.255851 22474 ts_tablet_manager.cc:1403] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.057s	user 0.028s	sys 0.009s
I20260709 13:10:46.256729 22475 tablet_bootstrap.cc:492] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:46.257225 22475 ts_tablet_manager.cc:1403] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.054s	user 0.007s	sys 0.009s
W20260709 13:10:46.258428 22322 leader_election.cc:343] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.257252 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.258891 22474 raft_consensus.cc:359] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.259506 22432 raft_consensus.cc:3060] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.259651 22474 raft_consensus.cc:385] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.259945 22474 raft_consensus.cc:740] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.260856 22475 raft_consensus.cc:359] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.260810 22474 consensus_queue.cc:260] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.261771 22475 raft_consensus.cc:385] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.262430 22475 raft_consensus.cc:740] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.263089 22472 tablet_bootstrap.cc:654] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.263375 22474 ts_tablet_manager.cc:1434] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:46.263550 22475 consensus_queue.cc:260] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.264449 22474 tablet_bootstrap.cc:492] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:46.266521 22475 ts_tablet_manager.cc:1434] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:10:46.267513 22432 raft_consensus.cc:2468] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.267843 22475 tablet_bootstrap.cc:492] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:46.269011 22321 leader_election.cc:304] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.269889 22480 raft_consensus.cc:2804] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.271284 22474 tablet_bootstrap.cc:654] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.274039 22480 raft_consensus.cc:697] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.275012 22480 consensus_queue.cc:237] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.276772 22475 tablet_bootstrap.cc:654] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.278076 22472 tablet_bootstrap.cc:492] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1: No bootstrap required, opened a new log
I20260709 13:10:46.278576 22472 ts_tablet_manager.cc:1403] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 13:10:46.279156 22474 tablet_bootstrap.cc:492] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:46.279629 22474 ts_tablet_manager.cc:1403] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:10:46.281322 22522 raft_consensus.cc:493] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.281831 22522 raft_consensus.cc:515] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.282104 22472 raft_consensus.cc:359] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.283005 22472 raft_consensus.cc:385] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.283329 22472 raft_consensus.cc:740] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.284379 22472 consensus_queue.cc:260] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.285105 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1221f647ca7c40a3b50c4d3414a33e89" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
W20260709 13:10:46.286525 22322 leader_election.cc:343] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.287537 22472 ts_tablet_manager.cc:1434] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:46.288046 22522 leader_election.cc:290] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.290006 22474 raft_consensus.cc:359] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.290745 22474 raft_consensus.cc:385] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.291059 22474 raft_consensus.cc:740] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.291620 22475 tablet_bootstrap.cc:492] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:46.291756 22474 consensus_queue.cc:260] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.292176 22475 ts_tablet_manager.cc:1403] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.013s
I20260709 13:10:46.293999 22474 ts_tablet_manager.cc:1434] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:10:46.294996 22474 tablet_bootstrap.cc:492] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9: Bootstrap starting.
I20260709 13:10:46.295176 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1221f647ca7c40a3b50c4d3414a33e89" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.299538 22431 raft_consensus.cc:2468] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.300822 22321 leader_election.cc:304] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.302639 22481 raft_consensus.cc:515] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.305644 22481 leader_election.cc:290] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.306331 22481 raft_consensus.cc:2804] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.306375 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:46.301087 22501 raft_consensus.cc:515] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.306743 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.307250 22282 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.307425 22431 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.309016 22501 leader_election.cc:290] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.306929 22481 raft_consensus.cc:493] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.310163 22481 raft_consensus.cc:3060] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.310333 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.310441 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:46.311075 22432 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:46.312021 22246 leader_election.cc:343] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.316812 22282 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.318614 22474 tablet_bootstrap.cc:654] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.318805 22481 raft_consensus.cc:515] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.319795 22432 raft_consensus.cc:2468] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 1.
I20260709 13:10:46.295373 22475 raft_consensus.cc:359] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.321254 22246 leader_election.cc:304] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 4c791639cce64ab489670508e36f682a
I20260709 13:10:46.321704 22481 leader_election.cc:290] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.322172 22501 raft_consensus.cc:2804] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.322710 22501 raft_consensus.cc:697] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:46.322886 22322 leader_election.cc:304] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:46.323173 22430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1221f647ca7c40a3b50c4d3414a33e89" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.323550 22501 consensus_queue.cc:237] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.323918 22430 raft_consensus.cc:3060] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.324146 22481 raft_consensus.cc:2804] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.325443 22481 raft_consensus.cc:697] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.325908 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1221f647ca7c40a3b50c4d3414a33e89" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:46.326435 22481 consensus_queue.cc:237] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
W20260709 13:10:46.328807 22322 leader_election.cc:343] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:46.334398 22430 raft_consensus.cc:2468] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.336376 22321 leader_election.cc:304] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.338347 22480 raft_consensus.cc:2804] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.339398 22480 raft_consensus.cc:697] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.306710 22180 catalog_manager.cc:5697] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 1 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.340698 22480 consensus_queue.cc:237] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.321475 22475 raft_consensus.cc:385] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.343588 22475 raft_consensus.cc:740] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.346226 22475 consensus_queue.cc:260] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.351351 22475 ts_tablet_manager.cc:1434] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 13:10:46.353333 22475 tablet_bootstrap.cc:492] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: Bootstrap starting.
I20260709 13:10:46.357025 22431 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.376029 22475 tablet_bootstrap.cc:654] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.378437 22474 tablet_bootstrap.cc:492] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9: No bootstrap required, opened a new log
I20260709 13:10:46.378969 22474 ts_tablet_manager.cc:1403] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent bootstrapping tablet: real 0.084s	user 0.029s	sys 0.031s
I20260709 13:10:46.381747 22475 tablet_bootstrap.cc:492] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: No bootstrap required, opened a new log
I20260709 13:10:46.382246 22475 ts_tablet_manager.cc:1403] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 13:10:46.382162 22474 raft_consensus.cc:359] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.383021 22474 raft_consensus.cc:385] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.382320 22179 catalog_manager.cc:5697] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.383354 22474 raft_consensus.cc:740] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.384928 22474 consensus_queue.cc:260] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.386626 22475 raft_consensus.cc:359] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.387337 22475 raft_consensus.cc:385] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.387638 22474 ts_tablet_manager.cc:1434] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 13:10:46.387606 22475 raft_consensus.cc:740] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.388494 22475 consensus_queue.cc:260] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.391192 22475 ts_tablet_manager.cc:1434] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:10:46.394809 22180 catalog_manager.cc:5697] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 1 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.396548 22180 catalog_manager.cc:5697] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 1 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.399295 22480 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.399808 22480 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.402007 22480 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.403007 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:46.403169 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.403733 22431 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.403775 22283 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.404963 22322 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:46.406109 22480 raft_consensus.cc:2804] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.406440 22480 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.406719 22480 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.413558 22480 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.415956 22480 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.416293 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:46.417133 22283 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.417133 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.417884 22431 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.423173 22480 raft_consensus.cc:493] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.423698 22480 raft_consensus.cc:515] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.424847 22283 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.424851 22431 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.426221 22322 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:46.427055 22481 raft_consensus.cc:2804] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.427179 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.427523 22480 leader_election.cc:290] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.426934 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" is_pre_election: true
I20260709 13:10:46.427670 22481 raft_consensus.cc:697] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.427798 22431 raft_consensus.cc:2468] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.428294 22283 raft_consensus.cc:2468] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 0.
I20260709 13:10:46.428925 22321 leader_election.cc:304] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:46.428771 22481 consensus_queue.cc:237] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.429843 22480 raft_consensus.cc:2804] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.430223 22480 raft_consensus.cc:493] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.430523 22480 raft_consensus.cc:3060] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.437569 22480 raft_consensus.cc:515] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.439749 22480 leader_election.cc:290] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.440999 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:46.441723 22283 raft_consensus.cc:3060] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.441970 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.442536 22431 raft_consensus.cc:3060] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.443305 22179 catalog_manager.cc:5697] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 1 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.449172 22283 raft_consensus.cc:2468] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.449337 22431 raft_consensus.cc:2468] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 1.
I20260709 13:10:46.450433 22321 leader_election.cc:304] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:46.451328 22480 raft_consensus.cc:2804] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.451819 22480 raft_consensus.cc:697] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.452467 22480 consensus_queue.cc:237] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.460258 22179 catalog_manager.cc:5697] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 1 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.461720 22480 consensus_queue.cc:1048] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.462291 22501 raft_consensus.cc:493] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.462802 22501 raft_consensus.cc:515] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.464895 22501 leader_election.cc:290] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.466761 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1221f647ca7c40a3b50c4d3414a33e89" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.469113 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1221f647ca7c40a3b50c4d3414a33e89" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.470052 22431 raft_consensus.cc:2393] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in current term 1: Already voted for candidate 4c791639cce64ab489670508e36f682a in this term.
I20260709 13:10:46.471263 22246 leader_election.cc:304] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1
I20260709 13:10:46.472579 22501 raft_consensus.cc:3060] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.482281 22522 consensus_queue.cc:1048] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.482766 22501 raft_consensus.cc:2749] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:46.493675 22477 consensus_queue.cc:1048] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.507972 22477 consensus_queue.cc:1048] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.514351 22501 raft_consensus.cc:493] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.514917 22501 raft_consensus.cc:515] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.517195 22501 leader_election.cc:290] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.518237 22430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.518347 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.518990 22430 raft_consensus.cc:2393] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in current term 1: Already voted for candidate 4c791639cce64ab489670508e36f682a in this term.
I20260709 13:10:46.520681 22246 leader_election.cc:304] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1
I20260709 13:10:46.521581 22501 raft_consensus.cc:3060] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.530298 22501 raft_consensus.cc:2749] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:46.693567 22501 consensus_queue.cc:1048] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.703076 22501 raft_consensus.cc:493] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.703711 22501 raft_consensus.cc:515] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.706823 22501 leader_election.cc:290] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.710204 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a" is_pre_election: true
I20260709 13:10:46.710744 22430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1" is_pre_election: true
I20260709 13:10:46.710945 22356 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 0.
I20260709 13:10:46.711377 22430 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 0.
I20260709 13:10:46.712894 22543 consensus_queue.cc:1048] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.713058 22246 leader_election.cc:304] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:46.714604 22543 raft_consensus.cc:2804] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:46.715039 22543 raft_consensus.cc:493] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.715466 22543 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.724117 22543 raft_consensus.cc:515] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.726462 22543 leader_election.cc:290] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.727424 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:46.727782 22430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.728082 22357 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.728384 22430 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.734531 22481 consensus_queue.cc:1048] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.736155 22357 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 1.
I20260709 13:10:46.736143 22430 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 1.
I20260709 13:10:46.737555 22246 leader_election.cc:304] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:46.738421 22543 raft_consensus.cc:2804] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:46.738915 22543 raft_consensus.cc:697] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:46.739293 22522 consensus_queue.cc:1048] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.739786 22543 consensus_queue.cc:237] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.754499 22180 catalog_manager.cc:5697] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:46.758191 22522 consensus_queue.cc:1048] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.761580 22357 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:46.789357 22481 consensus_queue.cc:1048] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.790179 22543 consensus_queue.cc:1048] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.814311 22502 consensus_queue.cc:1048] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.851141 22357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b"
dest_uuid: "4c791639cce64ab489670508e36f682a"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:56022
I20260709 13:10:46.851842 22357 raft_consensus.cc:606] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.858492 22548 consensus_queue.cc:1048] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.859511 22357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "4c791639cce64ab489670508e36f682a"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:56022
I20260709 13:10:46.860239 22357 raft_consensus.cc:606] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.866861 22548 consensus_queue.cc:1048] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.874557 22357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c"
dest_uuid: "4c791639cce64ab489670508e36f682a"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:56022
I20260709 13:10:46.875291 22357 raft_consensus.cc:606] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:46.880508 22522 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.909191 22480 raft_consensus.cc:993] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:46.909915 22480 raft_consensus.cc:1081] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:46.913367 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40016
I20260709 13:10:46.915215 22281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "4c791639cce64ab489670508e36f682a"
 from {username='slave'} at 127.0.0.1:40044
I20260709 13:10:46.915318 22282 raft_consensus.cc:493] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.916075 22281 raft_consensus.cc:606] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Received request to transfer leadership to TS 4c791639cce64ab489670508e36f682a
I20260709 13:10:46.916139 22282 raft_consensus.cc:3060] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.922660 22480 raft_consensus.cc:993] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:46.923164 22480 raft_consensus.cc:1081] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:46.924904 22282 raft_consensus.cc:515] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:46.928316 22522 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:46.930821 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.931478 22431 raft_consensus.cc:3060] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.931828 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c135a95e524c95954daed5cce26c4b" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:46.932454 22357 raft_consensus.cc:3055] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.932763 22357 raft_consensus.cc:740] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.934084 22357 consensus_queue.cc:260] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:46.935415 22357 raft_consensus.cc:3060] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.937991 22281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40016
I20260709 13:10:46.938938 22281 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.939505 22281 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.945770 22481 consensus_queue.cc:1048] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.949704 22281 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.955830 22481 consensus_queue.cc:1048] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:46.958164 22431 raft_consensus.cc:2468] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:46.958350 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:46.959072 22356 raft_consensus.cc:3055] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:46.959580 22356 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:46.960726 22356 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.929438 22282 leader_election.cc:290] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Requested vote from peers f26944a219514330933cbf63a91527d1 (127.21.33.3:38091), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:46.965296 22246 leader_election.cc:304] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:46.965922 22356 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.966456 22502 raft_consensus.cc:2804] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.966992 22502 raft_consensus.cc:697] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:46.968170 22502 consensus_queue.cc:237] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } }
I20260709 13:10:46.969343 22357 raft_consensus.cc:2468] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:46.971810 22522 raft_consensus.cc:993] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:46.972254 22522 raft_consensus.cc:1081] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:46.974427 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:46.975136 22432 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.978192 22356 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:46.979622 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40016
I20260709 13:10:46.979655 22246 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:46.980721 22501 raft_consensus.cc:2804] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:46.981401 22282 raft_consensus.cc:493] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:46.981243 22501 raft_consensus.cc:697] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:46.981855 22282 raft_consensus.cc:3060] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:46.982609 22501 consensus_queue.cc:237] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.979192 22281 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:46.991504 22282 raft_consensus.cc:515] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:46.994745 22432 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:46.997179 22282 leader_election.cc:290] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:47.000001 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:47.000658 22356 raft_consensus.cc:3055] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:47.001001 22356 raft_consensus.cc:740] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:47.001113 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4385d13d29c04839b94a9c984bef179c" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:47.001973 22432 raft_consensus.cc:3060] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.002172 22356 consensus_queue.cc:260] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:47.005347 22356 raft_consensus.cc:3060] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.012878 22432 raft_consensus.cc:2468] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:47.014186 22356 raft_consensus.cc:2468] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:47.014636 22246 leader_election.cc:304] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:47.015587 22501 raft_consensus.cc:2804] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:47.016263 22501 raft_consensus.cc:697] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:47.017204 22501 consensus_queue.cc:237] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:47.020972 22178 catalog_manager.cc:5697] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 1 to 2, leader changed from 4c791639cce64ab489670508e36f682a (127.21.33.2) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.022993 22178 catalog_manager.cc:5697] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 1 to 2, leader changed from 4c791639cce64ab489670508e36f682a (127.21.33.2) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.041033 22181 catalog_manager.cc:5697] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 1 to 2, leader changed from 4c791639cce64ab489670508e36f682a (127.21.33.2) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.254357 22501 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:47.268440 22543 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:47.373068 22431 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.374727 22543 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.386071 22356 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.388182 22501 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:47.435014 22563 raft_consensus.cc:993] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower 4c791639cce64ab489670508e36f682a to start an election
I20260709 13:10:47.435546 22501 raft_consensus.cc:1081] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Signalling peer 4c791639cce64ab489670508e36f682a to start an election
I20260709 13:10:47.437323 22356 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46"
dest_uuid: "4c791639cce64ab489670508e36f682a"
 from {username='slave'} at 127.0.0.1:56008
I20260709 13:10:47.437963 22356 raft_consensus.cc:493] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:47.438324 22356 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.444705 22356 raft_consensus.cc:515] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:47.446817 22356 leader_election.cc:290] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 2 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:47.447757 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:47.448170 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:47.448452 22282 raft_consensus.cc:3055] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:47.448778 22432 raft_consensus.cc:1275] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.448817 22282 raft_consensus.cc:740] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:47.448836 22431 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.450335 22501 consensus_queue.cc:1048] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.450001 22282 consensus_queue.cc:260] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:47.451503 22282 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:47.458195 22431 raft_consensus.cc:2468] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 2.
I20260709 13:10:47.460270 22321 leader_election.cc:304] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:47.462754 22282 raft_consensus.cc:2468] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 2.
I20260709 13:10:47.463434 22481 raft_consensus.cc:2804] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:47.463945 22481 raft_consensus.cc:697] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:47.464785 22481 consensus_queue.cc:237] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:47.466401 22356 raft_consensus.cc:1275] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.468137 22543 consensus_queue.cc:1048] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.478322 22181 catalog_manager.cc:5697] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 1 to 2, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 2 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.579353 22356 raft_consensus.cc:1275] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.581324 22543 consensus_queue.cc:1048] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.590726 22431 raft_consensus.cc:1275] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.592784 22543 consensus_queue.cc:1048] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.844264 22282 raft_consensus.cc:1275] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Refusing update from remote peer 4c791639cce64ab489670508e36f682a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.845729 22481 consensus_queue.cc:1048] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:47.857043 22431 raft_consensus.cc:1275] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 4c791639cce64ab489670508e36f682a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:47.859041 22566 consensus_queue.cc:1048] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:49.974865 21636 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:10:49.975733 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:49.998699 22282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:40048
I20260709 13:10:49.999274 22282 raft_consensus.cc:606] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Received request to transfer leadership to TS f26944a219514330933cbf63a91527d1
I20260709 13:10:50.002419 22282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:40048
I20260709 13:10:50.003022 22282 raft_consensus.cc:606] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Received request to transfer leadership to TS f26944a219514330933cbf63a91527d1
I20260709 13:10:50.004343 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:50.004912 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:50.379813 22568 raft_consensus.cc:993] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:50.380486 22568 raft_consensus.cc:1081] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Signalling peer f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:50.382910 22431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae"
dest_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:39548
I20260709 13:10:50.383728 22431 raft_consensus.cc:493] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:50.384140 22431 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.392108 22431 raft_consensus.cc:515] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:50.394469 22431 leader_election.cc:290] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 2 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:50.395991 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:50.395885 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:50.396974 22357 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.396859 22282 raft_consensus.cc:3055] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:50.397545 22282 raft_consensus.cc:740] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:50.398670 22282 consensus_queue.cc:260] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:50.400628 22282 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.407193 22357 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 2.
I20260709 13:10:50.409138 22395 leader_election.cc:304] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:50.410159 22282 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 2.
I20260709 13:10:50.410358 22477 raft_consensus.cc:2804] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:50.412389 22477 raft_consensus.cc:697] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 LEADER]: Becoming Leader. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:50.413703 22477 consensus_queue.cc:237] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:50.427109 22181 catalog_manager.cc:5697] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 reported cstate change: term changed from 1 to 2, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to f26944a219514330933cbf63a91527d1 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "f26944a219514330933cbf63a91527d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.509150 22573 raft_consensus.cc:993] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:50.509805 22573 raft_consensus.cc:1081] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Signalling peer f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:50.512228 22431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:39548
I20260709 13:10:50.512807 22431 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:50.513208 22431 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:50.520828 22431 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:50.522522 22431 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 3 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:50.523756 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:50.523756 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:50.524610 22357 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:50.524610 22282 raft_consensus.cc:3055] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:50.525185 22282 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:50.526132 22282 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:50.527482 22282 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:50.532265 22357 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 3.
I20260709 13:10:50.534305 22395 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:50.535339 22477 raft_consensus.cc:2804] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:50.535380 22282 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 3.
I20260709 13:10:50.536077 22477 raft_consensus.cc:697] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 LEADER]: Becoming Leader. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:50.537062 22477 consensus_queue.cc:237] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:50.544876 22181 catalog_manager.cc:5697] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 reported cstate change: term changed from 2 to 3, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to f26944a219514330933cbf63a91527d1 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "f26944a219514330933cbf63a91527d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.847808 22357 raft_consensus.cc:1275] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:50.849263 22477 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:50.864801 22282 raft_consensus.cc:1275] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:50.866730 22574 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:50.938100 22357 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:50.939895 22574 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:50.950933 22282 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:50.952765 22583 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:51.006361 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:51.036739 22431 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "f26944a219514330933cbf63a91527d1"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:53656
I20260709 13:10:51.037361 22431 raft_consensus.cc:606] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:51.044420 22431 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae"
dest_uuid: "f26944a219514330933cbf63a91527d1"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:53656
I20260709 13:10:51.044939 22431 raft_consensus.cc:606] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:51.057991 22357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3"
dest_uuid: "4c791639cce64ab489670508e36f682a"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:56022
I20260709 13:10:51.058760 22357 raft_consensus.cc:606] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:51.066126 22357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46"
dest_uuid: "4c791639cce64ab489670508e36f682a"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:56022
I20260709 13:10:51.066684 22357 raft_consensus.cc:606] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:51.073387 22357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c"
dest_uuid: "4c791639cce64ab489670508e36f682a"
mode: GRACEFUL
new_leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:56022
I20260709 13:10:51.074067 22357 raft_consensus.cc:606] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Received request to transfer leadership to TS 56bd3251a0c944dcb474c7f5ec3a11f9
I20260709 13:10:51.213886 22587 raft_consensus.cc:993] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.214507 22587 raft_consensus.cc:1081] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.216568 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40016
I20260709 13:10:51.217204 22282 raft_consensus.cc:493] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.217644 22282 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.223094 22282 raft_consensus.cc:515] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.224856 22282 leader_election.cc:290] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:51.226598 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:51.226405 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:51.227182 22431 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.227421 22357 raft_consensus.cc:3055] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.227790 22357 raft_consensus.cc:740] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:51.228595 22357 consensus_queue.cc:260] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.229769 22357 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.232151 22431 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:51.233685 22246 leader_election.cc:304] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd3251a0c944dcb474c7f5ec3a11f9, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:51.234812 22568 raft_consensus.cc:2804] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.235993 22568 raft_consensus.cc:697] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:51.236948 22357 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:51.237061 22568 consensus_queue.cc:237] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.244850 22181 catalog_manager.cc:5697] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 1 to 2, leader changed from 4c791639cce64ab489670508e36f682a (127.21.33.2) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.359581 22477 raft_consensus.cc:993] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.360134 22477 raft_consensus.cc:1081] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.361989 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:10:51.362556 22282 raft_consensus.cc:493] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.362943 22282 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:51.368072 22282 raft_consensus.cc:515] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.369879 22282 leader_election.cc:290] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 3 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:51.371002 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:51.370743 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:51.371809 22431 raft_consensus.cc:3055] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:51.371924 22357 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:51.372259 22431 raft_consensus.cc:740] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:51.373425 22431 consensus_queue.cc:260] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.374504 22431 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:51.377562 22357 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 3.
I20260709 13:10:51.378793 22246 leader_election.cc:304] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:51.379501 22568 raft_consensus.cc:2804] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:51.380055 22568 raft_consensus.cc:697] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:51.380030 22431 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 3.
I20260709 13:10:51.381189 22568 consensus_queue.cc:237] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.387833 22181 catalog_manager.cc:5697] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 2 to 3, leader changed from f26944a219514330933cbf63a91527d1 (127.21.33.3) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 3 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.440316 22480 raft_consensus.cc:993] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.440855 22587 raft_consensus.cc:1081] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.442965 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40016
I20260709 13:10:51.443535 22282 raft_consensus.cc:493] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.443791 22282 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:51.448026 22282 raft_consensus.cc:515] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.449862 22282 leader_election.cc:290] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 3 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:51.450798 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:51.451349 22357 raft_consensus.cc:3055] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:51.451639 22357 raft_consensus.cc:740] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:51.451411 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acaac9b2b1b94d16bd89a6696dd3de46" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:51.452132 22432 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:51.452381 22357 consensus_queue.cc:260] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.453398 22357 raft_consensus.cc:3060] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:51.458250 22357 raft_consensus.cc:2468] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 3.
I20260709 13:10:51.458782 22432 raft_consensus.cc:2468] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 3.
I20260709 13:10:51.459376 22246 leader_election.cc:304] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:51.460366 22568 raft_consensus.cc:2804] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:51.460843 22568 raft_consensus.cc:697] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:51.461843 22568 consensus_queue.cc:237] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.470587 22583 raft_consensus.cc:993] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.470185 22181 catalog_manager.cc:5697] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 2 to 3, leader changed from 4c791639cce64ab489670508e36f682a (127.21.33.2) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 3 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.471239 22574 raft_consensus.cc:1081] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.476735 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40032
I20260709 13:10:51.478274 22282 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.478596 22282 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:51.484442 22282 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.486584 22282 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 4 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:51.487463 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:51.488090 22357 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:51.487978 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:51.488585 22432 raft_consensus.cc:3055] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:51.488899 22432 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:51.489751 22432 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.490993 22432 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:51.494777 22357 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 4.
I20260709 13:10:51.496289 22246 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:51.497292 22568 raft_consensus.cc:2804] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:51.497877 22568 raft_consensus.cc:697] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:51.497890 22432 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 4.
I20260709 13:10:51.499552 22568 consensus_queue.cc:237] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.509088 22181 catalog_manager.cc:5697] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 3 to 4, leader changed from f26944a219514330933cbf63a91527d1 (127.21.33.3) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 4 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.530041 22480 raft_consensus.cc:993] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: : Instructing follower 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.530524 22480 raft_consensus.cc:1081] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Signalling peer 56bd3251a0c944dcb474c7f5ec3a11f9 to start an election
I20260709 13:10:51.532439 22282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
 from {username='slave'} at 127.0.0.1:40016
I20260709 13:10:51.533027 22282 raft_consensus.cc:493] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:51.533403 22282 raft_consensus.cc:3060] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.540166 22282 raft_consensus.cc:515] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.542034 22282 leader_election.cc:290] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Requested vote from peers 4c791639cce64ab489670508e36f682a (127.21.33.2:36473), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:51.542954 22357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:51.543350 22432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b472ae65ee42e2bfed4a439cf4b88c" candidate_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:51.543602 22357 raft_consensus.cc:3055] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:51.543944 22432 raft_consensus.cc:3060] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.543949 22357 raft_consensus.cc:740] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:51.544924 22357 consensus_queue.cc:260] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.545845 22357 raft_consensus.cc:3060] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:51.550225 22357 raft_consensus.cc:2468] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:51.551376 22246 leader_election.cc:304] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:51.551606 22432 raft_consensus.cc:2468] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56bd3251a0c944dcb474c7f5ec3a11f9 in term 2.
I20260709 13:10:51.552254 22568 raft_consensus.cc:2804] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:51.552709 22568 raft_consensus.cc:697] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Leader. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:51.553627 22568 consensus_queue.cc:237] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:51.561610 22181 catalog_manager.cc:5697] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 reported cstate change: term changed from 1 to 2, leader changed from 4c791639cce64ab489670508e36f682a (127.21.33.2) to 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.794380 22432 raft_consensus.cc:1275] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:51.796540 22568 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:51.808378 22357 raft_consensus.cc:1275] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:51.810456 22568 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:51.820729 22432 raft_consensus.cc:1275] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:51.822362 22592 consensus_queue.cc:1048] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:51.841179 22357 raft_consensus.cc:1275] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:51.844949 22357 raft_consensus.cc:1275] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:51.845531 22603 consensus_queue.cc:1048] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:51.846364 22592 consensus_queue.cc:1048] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:51.862556 22432 raft_consensus.cc:1275] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:51.864277 22603 consensus_queue.cc:1048] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:51.919224 22356 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:51.921037 22603 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:51.931285 22432 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:51.934701 22568 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:51.970981 22356 raft_consensus.cc:1275] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:51.972719 22568 consensus_queue.cc:1048] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:51.984083 22432 raft_consensus.cc:1275] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 56bd3251a0c944dcb474c7f5ec3a11f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:51.986778 22603 consensus_queue.cc:1048] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:54.084690 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:54.094677 22282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "4c791639cce64ab489670508e36f682a"
 from {username='slave'} at 127.0.0.1:40048
I20260709 13:10:54.095192 22282 raft_consensus.cc:606] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Received request to transfer leadership to TS 4c791639cce64ab489670508e36f682a
I20260709 13:10:54.098045 22282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:40048
I20260709 13:10:54.098539 22282 raft_consensus.cc:606] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Received request to transfer leadership to TS f26944a219514330933cbf63a91527d1
I20260709 13:10:54.101577 22282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "4c791639cce64ab489670508e36f682a"
 from {username='slave'} at 127.0.0.1:40048
I20260709 13:10:54.102029 22282 raft_consensus.cc:606] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Received request to transfer leadership to TS 4c791639cce64ab489670508e36f682a
I20260709 13:10:54.104827 22282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
mode: GRACEFUL
new_leader_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:40048
I20260709 13:10:54.105309 22282 raft_consensus.cc:606] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 LEADER]: Received request to transfer leadership to TS f26944a219514330933cbf63a91527d1
I20260709 13:10:54.106592 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:10:54.107199 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:54.142568 22610 raft_consensus.cc:993] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:54.143323 22568 raft_consensus.cc:1081] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 LEADER]: Signalling peer f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:54.144853 22431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95"
dest_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:39548
I20260709 13:10:54.145363 22431 raft_consensus.cc:493] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:54.145718 22431 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:54.151266 22431 raft_consensus.cc:515] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.152832 22431 leader_election.cc:290] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 5 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:54.154162 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:54.154162 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02b85578a6140ab8d3bb048afec6c95" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:54.155074 22356 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:54.155045 22282 raft_consensus.cc:3055] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:54.155608 22282 raft_consensus.cc:740] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:54.156546 22282 consensus_queue.cc:260] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.157858 22282 raft_consensus.cc:3060] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:54.163959 22356 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 5.
I20260709 13:10:54.165315 22395 leader_election.cc:304] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:54.165890 22282 raft_consensus.cc:2468] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 5.
I20260709 13:10:54.166157 22574 raft_consensus.cc:2804] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:54.167724 22574 raft_consensus.cc:697] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 5 LEADER]: Becoming Leader. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:54.168576 22574 consensus_queue.cc:237] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.176537 22181 catalog_manager.cc:5697] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 reported cstate change: term changed from 4 to 5, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to f26944a219514330933cbf63a91527d1 (127.21.33.3). New cstate: current_term: 5 leader_uuid: "f26944a219514330933cbf63a91527d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.490276 22568 raft_consensus.cc:993] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:54.490710 22568 raft_consensus.cc:1081] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Signalling peer f26944a219514330933cbf63a91527d1 to start an election
I20260709 13:10:54.492666 22431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae"
dest_uuid: "f26944a219514330933cbf63a91527d1"
 from {username='slave'} at 127.0.0.1:39548
I20260709 13:10:54.493222 22431 raft_consensus.cc:493] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:54.493605 22431 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:54.498517 22431 raft_consensus.cc:515] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.500272 22431 leader_election.cc:290] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 4 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), 4c791639cce64ab489670508e36f682a (127.21.33.2:36473)
I20260709 13:10:54.501271 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:54.501420 22356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3f31fd90b28408bb19233cd0d171cae" candidate_uuid: "f26944a219514330933cbf63a91527d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4c791639cce64ab489670508e36f682a"
I20260709 13:10:54.501870 22282 raft_consensus.cc:3055] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:54.502059 22356 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:54.502188 22282 raft_consensus.cc:740] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:54.502911 22282 consensus_queue.cc:260] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.504112 22282 raft_consensus.cc:3060] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:54.509093 22356 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 4.
I20260709 13:10:54.510231 22282 raft_consensus.cc:2468] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f26944a219514330933cbf63a91527d1 in term 4.
I20260709 13:10:54.510509 22395 leader_election.cc:304] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:54.511370 22574 raft_consensus.cc:2804] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:54.511827 22574 raft_consensus.cc:697] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 4 LEADER]: Becoming Leader. State: Replica: f26944a219514330933cbf63a91527d1, State: Running, Role: LEADER
I20260709 13:10:54.512593 22574 consensus_queue.cc:237] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.522789 22181 catalog_manager.cc:5697] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 reported cstate change: term changed from 3 to 4, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to f26944a219514330933cbf63a91527d1 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "f26944a219514330933cbf63a91527d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.542546 22610 raft_consensus.cc:993] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower 4c791639cce64ab489670508e36f682a to start an election
I20260709 13:10:54.543092 22568 raft_consensus.cc:1081] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Signalling peer 4c791639cce64ab489670508e36f682a to start an election
I20260709 13:10:54.544910 22356 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03"
dest_uuid: "4c791639cce64ab489670508e36f682a"
 from {username='slave'} at 127.0.0.1:56008
I20260709 13:10:54.545533 22356 raft_consensus.cc:493] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:54.545905 22356 raft_consensus.cc:3060] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.551617 22356 raft_consensus.cc:515] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.553797 22356 leader_election.cc:290] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 2 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:54.554690 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:54.554916 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce2b70e6efcb4ce383dcb4288dbb3c03" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:54.555299 22282 raft_consensus.cc:3055] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:54.555547 22431 raft_consensus.cc:3060] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.555653 22282 raft_consensus.cc:740] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:54.556460 22282 consensus_queue.cc:260] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.557416 22282 raft_consensus.cc:3060] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:54.561836 22431 raft_consensus.cc:2468] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 2.
I20260709 13:10:54.562175 22282 raft_consensus.cc:2468] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 2.
I20260709 13:10:54.564306 22322 leader_election.cc:304] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, 56bd3251a0c944dcb474c7f5ec3a11f9; no voters: 
I20260709 13:10:54.565337 22480 raft_consensus.cc:2804] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:54.566888 22480 raft_consensus.cc:697] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:54.568610 22480 consensus_queue.cc:237] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.577135 22181 catalog_manager.cc:5697] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 1 to 2, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 2 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.626108 22282 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 5 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:54.628038 22574 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:54.628875 22356 raft_consensus.cc:1275] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 5 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:54.630649 22574 consensus_queue.cc:1048] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:54.641758 22610 raft_consensus.cc:993] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: : Instructing follower 4c791639cce64ab489670508e36f682a to start an election
I20260709 13:10:54.648686 22568 raft_consensus.cc:1081] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Signalling peer 4c791639cce64ab489670508e36f682a to start an election
I20260709 13:10:54.663125 22356 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3"
dest_uuid: "4c791639cce64ab489670508e36f682a"
 from {username='slave'} at 127.0.0.1:56008
I20260709 13:10:54.663838 22356 raft_consensus.cc:493] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:54.664242 22356 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:54.672210 22356 raft_consensus.cc:515] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.674880 22356 leader_election.cc:290] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 3 election: Requested vote from peers 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863), f26944a219514330933cbf63a91527d1 (127.21.33.3:38091)
I20260709 13:10:54.676137 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f26944a219514330933cbf63a91527d1"
I20260709 13:10:54.676138 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46386a22c8cf46acae0712ef631824c3" candidate_uuid: "4c791639cce64ab489670508e36f682a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9"
I20260709 13:10:54.676788 22431 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:54.676812 22282 raft_consensus.cc:3055] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:54.677286 22282 raft_consensus.cc:740] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 56bd3251a0c944dcb474c7f5ec3a11f9, State: Running, Role: LEADER
I20260709 13:10:54.677950 22282 consensus_queue.cc:260] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.678864 22282 raft_consensus.cc:3060] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:54.682494 22431 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 3.
I20260709 13:10:54.684093 22321 leader_election.cc:304] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c791639cce64ab489670508e36f682a, f26944a219514330933cbf63a91527d1; no voters: 
I20260709 13:10:54.685109 22480 raft_consensus.cc:2804] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:54.685765 22480 raft_consensus.cc:697] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 3 LEADER]: Becoming Leader. State: Replica: 4c791639cce64ab489670508e36f682a, State: Running, Role: LEADER
I20260709 13:10:54.686699 22480 consensus_queue.cc:237] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } }
I20260709 13:10:54.687125 22282 raft_consensus.cc:2468] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c791639cce64ab489670508e36f682a in term 3.
I20260709 13:10:54.694883 22178 catalog_manager.cc:5697] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a reported cstate change: term changed from 2 to 3, leader changed from 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1) to 4c791639cce64ab489670508e36f682a (127.21.33.2). New cstate: current_term: 3 leader_uuid: "4c791639cce64ab489670508e36f682a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.976923 22431 raft_consensus.cc:1275] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Refusing update from remote peer 4c791639cce64ab489670508e36f682a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.978881 22480 consensus_queue.cc:1048] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:54.991212 22282 raft_consensus.cc:1275] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Refusing update from remote peer 4c791639cce64ab489670508e36f682a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:54.993530 22629 consensus_queue.cc:1048] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:55.043066 22356 raft_consensus.cc:1275] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:55.045037 22574 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c791639cce64ab489670508e36f682a" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:55.056258 22282 raft_consensus.cc:1275] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Refusing update from remote peer f26944a219514330933cbf63a91527d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:55.060252 22611 consensus_queue.cc:1048] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd3251a0c944dcb474c7f5ec3a11f9" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:55.108270 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:55.115257 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
W20260709 13:10:55.129205 22396 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 28 similar messages]
W20260709 13:10:55.133319 22396 consensus_peers.cc:597] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 -> Peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Couldn't send request to peer 56bd3251a0c944dcb474c7f5ec3a11f9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.151893 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:55.152865 21636 tablet_replica.cc:333] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.153900 21636 raft_consensus.cc:2243] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.154536 21636 raft_consensus.cc:2272] T 1221f647ca7c40a3b50c4d3414a33e89 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.157891 21636 tablet_replica.cc:333] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.158510 21636 raft_consensus.cc:2243] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.158883 21636 raft_consensus.cc:2272] T 46386a22c8cf46acae0712ef631824c3 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.160837 21636 tablet_replica.cc:333] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.161412 21636 raft_consensus.cc:2243] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.161944 21636 raft_consensus.cc:2272] T f02b85578a6140ab8d3bb048afec6c95 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.164626 21636 tablet_replica.cc:333] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.165560 21636 raft_consensus.cc:2243] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.166543 21636 raft_consensus.cc:2272] T 17b472ae65ee42e2bfed4a439cf4b88c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.170244 21636 tablet_replica.cc:333] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.170902 21636 raft_consensus.cc:2243] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.171381 21636 raft_consensus.cc:2272] T 1dfba1fdc47d42b796048719a3ec7488 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.173696 21636 tablet_replica.cc:333] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.174523 21636 raft_consensus.cc:2243] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.175864 21636 raft_consensus.cc:2272] T 4385d13d29c04839b94a9c984bef179c P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.179659 21636 tablet_replica.cc:333] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.180424 21636 raft_consensus.cc:2243] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.181123 21636 raft_consensus.cc:2272] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.185627 21636 tablet_replica.cc:333] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.186321 21636 raft_consensus.cc:2243] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.186820 21636 raft_consensus.cc:2272] T b3f31fd90b28408bb19233cd0d171cae P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.189852 21636 tablet_replica.cc:333] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.190470 21636 raft_consensus.cc:2243] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.191327 21636 raft_consensus.cc:2272] T acaac9b2b1b94d16bd89a6696dd3de46 P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.193290 21636 tablet_replica.cc:333] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9: stopping tablet replica
I20260709 13:10:55.193900 21636 raft_consensus.cc:2243] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.194746 21636 raft_consensus.cc:2272] T 34c135a95e524c95954daed5cce26c4b P 56bd3251a0c944dcb474c7f5ec3a11f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.232046 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
W20260709 13:10:55.253062 22396 consensus_peers.cc:597] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 -> Peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Couldn't send request to peer 56bd3251a0c944dcb474c7f5ec3a11f9. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.260803 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
I20260709 13:10:55.278680 22431 raft_consensus.cc:1275] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Refusing update from remote peer 4c791639cce64ab489670508e36f682a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:55.280709 22629 consensus_queue.cc:1048] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26944a219514330933cbf63a91527d1" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:55.315831 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:55.317788 21636 tablet_replica.cc:333] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a: stopping tablet replica
W20260709 13:10:55.318303 22322 consensus_peers.cc:597] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a -> Peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Couldn't send request to peer 56bd3251a0c944dcb474c7f5ec3a11f9. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.319741 21636 raft_consensus.cc:2243] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.320444 21636 raft_consensus.cc:2272] T acaac9b2b1b94d16bd89a6696dd3de46 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.322561 21636 tablet_replica.cc:333] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.323318 21636 raft_consensus.cc:2243] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.323839 21636 raft_consensus.cc:2272] T b3f31fd90b28408bb19233cd0d171cae P 4c791639cce64ab489670508e36f682a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.327252 21636 tablet_replica.cc:333] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.327992 21636 raft_consensus.cc:2243] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.328559 21636 raft_consensus.cc:2272] T f02b85578a6140ab8d3bb048afec6c95 P 4c791639cce64ab489670508e36f682a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.331526 21636 tablet_replica.cc:333] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.332172 21636 raft_consensus.cc:2243] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.332968 21636 raft_consensus.cc:2272] T 46386a22c8cf46acae0712ef631824c3 P 4c791639cce64ab489670508e36f682a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.336356 21636 tablet_replica.cc:333] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.337242 21636 raft_consensus.cc:2243] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.337845 21636 raft_consensus.cc:2272] T 1dfba1fdc47d42b796048719a3ec7488 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:55.339742 22322 consensus_peers.cc:597] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a -> Peer 56bd3251a0c944dcb474c7f5ec3a11f9 (127.21.33.1:38863): Couldn't send request to peer 56bd3251a0c944dcb474c7f5ec3a11f9. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.340548 21636 tablet_replica.cc:333] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.341159 21636 raft_consensus.cc:2243] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.341681 21636 raft_consensus.cc:2272] T 17b472ae65ee42e2bfed4a439cf4b88c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.343581 21636 tablet_replica.cc:333] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.344149 21636 raft_consensus.cc:2243] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.344974 21636 raft_consensus.cc:2272] T 1221f647ca7c40a3b50c4d3414a33e89 P 4c791639cce64ab489670508e36f682a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.346926 21636 tablet_replica.cc:333] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.347916 21636 raft_consensus.cc:2243] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.348467 21636 raft_consensus.cc:2272] T 34c135a95e524c95954daed5cce26c4b P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.350416 21636 tablet_replica.cc:333] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.350960 21636 raft_consensus.cc:2243] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.351435 21636 raft_consensus.cc:2272] T 4385d13d29c04839b94a9c984bef179c P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.353606 21636 tablet_replica.cc:333] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a: stopping tablet replica
I20260709 13:10:55.354187 21636 raft_consensus.cc:2243] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.355083 21636 raft_consensus.cc:2272] T ce2b70e6efcb4ce383dcb4288dbb3c03 P 4c791639cce64ab489670508e36f682a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.392045 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:10:55.417429 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
I20260709 13:10:55.451727 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:55.452780 21636 tablet_replica.cc:333] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.453716 21636 raft_consensus.cc:2243] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.454576 21636 raft_consensus.cc:2272] T 46386a22c8cf46acae0712ef631824c3 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:55.454751 22395 consensus_peers.cc:597] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 -> Peer 4c791639cce64ab489670508e36f682a (127.21.33.2:36473): Couldn't send request to peer 4c791639cce64ab489670508e36f682a. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:36473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.458546 21636 tablet_replica.cc:333] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.459437 21636 raft_consensus.cc:2243] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.460119 21636 raft_consensus.cc:2272] T 1221f647ca7c40a3b50c4d3414a33e89 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.462828 21636 tablet_replica.cc:333] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.463531 21636 raft_consensus.cc:2243] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.464000 21636 raft_consensus.cc:2272] T ce2b70e6efcb4ce383dcb4288dbb3c03 P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.469954 21636 tablet_replica.cc:333] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.470736 21636 raft_consensus.cc:2243] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.471251 21636 raft_consensus.cc:2272] T 4385d13d29c04839b94a9c984bef179c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.476331 21636 tablet_replica.cc:333] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.478060 21636 raft_consensus.cc:2243] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.479182 21636 raft_consensus.cc:2272] T 17b472ae65ee42e2bfed4a439cf4b88c P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.481462 21636 tablet_replica.cc:333] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.482786 21636 raft_consensus.cc:2243] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.483774 21636 raft_consensus.cc:2272] T 1dfba1fdc47d42b796048719a3ec7488 P f26944a219514330933cbf63a91527d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.486116 21636 tablet_replica.cc:333] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.486760 21636 raft_consensus.cc:2243] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.487269 21636 raft_consensus.cc:2272] T acaac9b2b1b94d16bd89a6696dd3de46 P f26944a219514330933cbf63a91527d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.489161 21636 tablet_replica.cc:333] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.489786 21636 raft_consensus.cc:2243] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.490670 21636 raft_consensus.cc:2272] T b3f31fd90b28408bb19233cd0d171cae P f26944a219514330933cbf63a91527d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.493188 21636 tablet_replica.cc:333] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.493913 21636 raft_consensus.cc:2243] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.494788 21636 raft_consensus.cc:2272] T f02b85578a6140ab8d3bb048afec6c95 P f26944a219514330933cbf63a91527d1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.497885 21636 tablet_replica.cc:333] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1: stopping tablet replica
I20260709 13:10:55.498476 21636 raft_consensus.cc:2243] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:55.498984 21636 raft_consensus.cc:2272] T 34c135a95e524c95954daed5cce26c4b P f26944a219514330933cbf63a91527d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.542520 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:10:55.575533 21636 master.cc:562] Master@127.21.33.62:37807 shutting down...
I20260709 13:10:55.615161 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:55.615901 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:55.616479 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4fa5ade16aa14d1098b97ad7a8bc938c: stopping tablet replica
I20260709 13:10:55.641588 21636 master.cc:584] Master@127.21.33.62:37807 shutdown complete.
4c791639cce64ab489670508e36f682a  4
56bd3251a0c944dcb474c7f5ec3a11f9  5
f26944a219514330933cbf63a91527d1  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (11082 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:10:55.694024 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:44143
I20260709 13:10:55.695452 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:55.702376 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:55.702455 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:55.702466 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:55.704322 21636 server_base.cc:1061] running on GCE node
I20260709 13:10:55.705624 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:55.705873 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:55.706157 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602655706065 us; error 0 us; skew 500 ppm
I20260709 13:10:55.706861 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:55.710606 21636 webserver.cc:533] Webserver started at http://127.21.33.62:38425/ using document root <none> and password file <none>
I20260709 13:10:55.711194 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:55.711412 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:55.711761 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:55.713043 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "6e189717484b49e8bb20c98667e6d294"
format_stamp: "Formatted at 2026-07-09 13:10:55 on dist-test-slave-p8rg"
I20260709 13:10:55.718389 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:10:55.723544 22647 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:55.724972 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:10:55.725474 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "6e189717484b49e8bb20c98667e6d294"
format_stamp: "Formatted at 2026-07-09 13:10:55 on dist-test-slave-p8rg"
I20260709 13:10:55.726207 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:55.737833 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:55.739246 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:55.795500 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:44143
I20260709 13:10:55.795639 22698 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:44143 every 8 connection(s)
I20260709 13:10:55.800380 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:55.811889 22699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Bootstrap starting.
I20260709 13:10:55.818382 22699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.824029 22699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: No bootstrap required, opened a new log
I20260709 13:10:55.826694 22699 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e189717484b49e8bb20c98667e6d294" member_type: VOTER }
I20260709 13:10:55.827147 22699 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.827319 22699 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e189717484b49e8bb20c98667e6d294, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.828029 22699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e189717484b49e8bb20c98667e6d294" member_type: VOTER }
I20260709 13:10:55.828617 22699 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:55.829066 22699 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:55.829393 22699 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.835279 22699 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e189717484b49e8bb20c98667e6d294" member_type: VOTER }
I20260709 13:10:55.835963 22699 leader_election.cc:304] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e189717484b49e8bb20c98667e6d294; no voters: 
I20260709 13:10:55.837415 22699 leader_election.cc:290] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:55.838336 22702 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.840240 22702 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 1 LEADER]: Becoming Leader. State: Replica: 6e189717484b49e8bb20c98667e6d294, State: Running, Role: LEADER
I20260709 13:10:55.841464 22702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e189717484b49e8bb20c98667e6d294" member_type: VOTER }
I20260709 13:10:55.843200 22699 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:55.847116 22703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e189717484b49e8bb20c98667e6d294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e189717484b49e8bb20c98667e6d294" member_type: VOTER } }
I20260709 13:10:55.847781 22703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:55.852391 22704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e189717484b49e8bb20c98667e6d294. Latest consensus state: current_term: 1 leader_uuid: "6e189717484b49e8bb20c98667e6d294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e189717484b49e8bb20c98667e6d294" member_type: VOTER } }
I20260709 13:10:55.853791 22704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:55.856606 22708 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:55.864464 22708 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:55.869282 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:55.877346 22708 catalog_manager.cc:1383] Generated new cluster ID: fd43f9f5d7a34b14b9c28b3816ff34c4
I20260709 13:10:55.877825 22708 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:55.910554 22708 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:55.912014 22708 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:55.932126 22708 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Generated new TSK 0
I20260709 13:10:55.932870 22708 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:56.001852 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:56.009727 22720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:56.010310 22721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.013896 21636 server_base.cc:1061] running on GCE node
W20260709 13:10:56.013984 22723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.015404 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:56.015618 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:56.015738 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602656015727 us; error 0 us; skew 500 ppm
I20260709 13:10:56.016265 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:56.019774 21636 webserver.cc:533] Webserver started at http://127.21.33.1:42095/ using document root <none> and password file <none>
I20260709 13:10:56.020306 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:56.020483 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:56.020723 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:56.022055 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "acfda9b350734def9b6c621f66a5f247"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-p8rg"
I20260709 13:10:56.027833 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:10:56.032155 22728 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.033313 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:10:56.033856 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "acfda9b350734def9b6c621f66a5f247"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-p8rg"
I20260709 13:10:56.034323 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:56.052645 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:56.054713 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:56.056758 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:56.059638 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:56.059892 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.060170 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:56.060374 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.117121 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:45095
I20260709 13:10:56.117224 22790 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:45095 every 8 connection(s)
I20260709 13:10:56.123062 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:56.133052 22795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:56.135615 22796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.137732 21636 server_base.cc:1061] running on GCE node
W20260709 13:10:56.139003 22798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.140146 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:56.140352 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:56.140576 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602656140555 us; error 0 us; skew 500 ppm
I20260709 13:10:56.141172 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:56.144606 21636 webserver.cc:533] Webserver started at http://127.21.33.2:35817/ using document root <none> and password file <none>
I20260709 13:10:56.145264 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:56.145355 22791 heartbeater.cc:344] Connected to a master server at 127.21.33.62:44143
I20260709 13:10:56.145694 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:56.145865 22791 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:56.146062 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:56.146891 22791 heartbeater.cc:507] Master 127.21.33.62:44143 requested a full tablet report, sending...
I20260709 13:10:56.147845 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "776baf137f8745dd866379ca15ab8cee"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-p8rg"
I20260709 13:10:56.149835 22664 ts_manager.cc:194] Registered new tserver with Master: acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:10:56.151805 22664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51522
I20260709 13:10:56.154167 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.005s
I20260709 13:10:56.158821 22803 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.159761 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:10:56.160063 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "776baf137f8745dd866379ca15ab8cee"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-p8rg"
I20260709 13:10:56.160341 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:56.185137 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:56.186908 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:56.188576 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:56.191927 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:56.192147 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.192481 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:56.192710 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.250167 22865 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:41075 every 8 connection(s)
I20260709 13:10:56.250059 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:41075
I20260709 13:10:56.257258 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:56.267418 22870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:56.268832 22869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.272894 21636 server_base.cc:1061] running on GCE node
W20260709 13:10:56.273391 22872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.274940 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:56.275285 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:56.275342 22866 heartbeater.cc:344] Connected to a master server at 127.21.33.62:44143
I20260709 13:10:56.275527 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602656275504 us; error 0 us; skew 500 ppm
I20260709 13:10:56.276005 22866 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:56.276512 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:56.276930 22866 heartbeater.cc:507] Master 127.21.33.62:44143 requested a full tablet report, sending...
I20260709 13:10:56.279577 22664 ts_manager.cc:194] Registered new tserver with Master: 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:56.279984 21636 webserver.cc:533] Webserver started at http://127.21.33.3:38101/ using document root <none> and password file <none>
I20260709 13:10:56.280701 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:56.280974 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:56.281328 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:56.282136 22664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51536
I20260709 13:10:56.283702 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "6545453e74344bb2957dd3729c5b3870"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-p8rg"
I20260709 13:10:56.289402 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:10:56.294816 22877 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.296182 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:10:56.296813 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "6545453e74344bb2957dd3729c5b3870"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-p8rg"
I20260709 13:10:56.297250 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:56.318161 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:56.319752 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:56.321975 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:56.324642 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:56.324965 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.325284 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:56.325471 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.381163 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:35233
I20260709 13:10:56.381356 22939 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:35233 every 8 connection(s)
I20260709 13:10:56.399646 22940 heartbeater.cc:344] Connected to a master server at 127.21.33.62:44143
I20260709 13:10:56.400102 22940 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:56.400939 22940 heartbeater.cc:507] Master 127.21.33.62:44143 requested a full tablet report, sending...
I20260709 13:10:56.403602 22664 ts_manager.cc:194] Registered new tserver with Master: 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:56.404690 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019510641s
I20260709 13:10:56.406066 22664 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51552
I20260709 13:10:56.439960 22664 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51568:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:56.447326 22664 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:56.698025 22905 tablet_service.cc:1511] Processing CreateTablet for tablet a71fa884e652460c841521a01aa8b5a0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:56.698405 22904 tablet_service.cc:1511] Processing CreateTablet for tablet bf5430bef8774025a919427a6e425b65 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:56.699915 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71fa884e652460c841521a01aa8b5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.700369 22903 tablet_service.cc:1511] Processing CreateTablet for tablet da64727f9cd84f6bb87d91e54eeac2d5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:56.703238 22902 tablet_service.cc:1511] Processing CreateTablet for tablet 7ede022de6a346e284376d4212e1f6b7 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:56.703572 22830 tablet_service.cc:1511] Processing CreateTablet for tablet bf5430bef8774025a919427a6e425b65 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:56.704856 22903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da64727f9cd84f6bb87d91e54eeac2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.705101 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5430bef8774025a919427a6e425b65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.706830 22829 tablet_service.cc:1511] Processing CreateTablet for tablet da64727f9cd84f6bb87d91e54eeac2d5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:56.708575 22829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da64727f9cd84f6bb87d91e54eeac2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.710860 22828 tablet_service.cc:1511] Processing CreateTablet for tablet 7ede022de6a346e284376d4212e1f6b7 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:56.712380 22828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ede022de6a346e284376d4212e1f6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.713764 22756 tablet_service.cc:1511] Processing CreateTablet for tablet a71fa884e652460c841521a01aa8b5a0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:56.714360 22901 tablet_service.cc:1511] Processing CreateTablet for tablet e36250f751be4e7480619d5807252ac0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:56.711346 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ede022de6a346e284376d4212e1f6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.715696 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71fa884e652460c841521a01aa8b5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.718274 22831 tablet_service.cc:1511] Processing CreateTablet for tablet a71fa884e652460c841521a01aa8b5a0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:56.718972 22755 tablet_service.cc:1511] Processing CreateTablet for tablet bf5430bef8774025a919427a6e425b65 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:56.720507 22755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5430bef8774025a919427a6e425b65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.721904 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5430bef8774025a919427a6e425b65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.722970 22754 tablet_service.cc:1511] Processing CreateTablet for tablet da64727f9cd84f6bb87d91e54eeac2d5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:56.723908 22827 tablet_service.cc:1511] Processing CreateTablet for tablet e36250f751be4e7480619d5807252ac0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:56.724722 22754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da64727f9cd84f6bb87d91e54eeac2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.722970 22826 tablet_service.cc:1511] Processing CreateTablet for tablet b86ad75e17d742e2926f8af53f082682 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:56.725989 22901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36250f751be4e7480619d5807252ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.727212 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 7ede022de6a346e284376d4212e1f6b7 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:56.729950 22831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71fa884e652460c841521a01aa8b5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.730162 22752 tablet_service.cc:1511] Processing CreateTablet for tablet e36250f751be4e7480619d5807252ac0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:56.731735 22900 tablet_service.cc:1511] Processing CreateTablet for tablet b86ad75e17d742e2926f8af53f082682 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:56.732579 22751 tablet_service.cc:1511] Processing CreateTablet for tablet b86ad75e17d742e2926f8af53f082682 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:56.733110 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36250f751be4e7480619d5807252ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.733983 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86ad75e17d742e2926f8af53f082682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.734248 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ede022de6a346e284376d4212e1f6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.737069 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86ad75e17d742e2926f8af53f082682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.737767 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86ad75e17d742e2926f8af53f082682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.745237 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36250f751be4e7480619d5807252ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.777001 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 2e04261a736f42d2a1fd4e24bc787276 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:56.779105 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e04261a736f42d2a1fd4e24bc787276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.795236 22825 tablet_service.cc:1511] Processing CreateTablet for tablet 2e04261a736f42d2a1fd4e24bc787276 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:56.796996 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e04261a736f42d2a1fd4e24bc787276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.801812 22749 tablet_service.cc:1511] Processing CreateTablet for tablet 82f76532c7d04012a58fce30b62b344d (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:56.803838 22749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f76532c7d04012a58fce30b62b344d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.805735 22748 tablet_service.cc:1511] Processing CreateTablet for tablet c52c0b7017594a0f8651f1fe9e9f51a2 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:56.807706 22824 tablet_service.cc:1511] Processing CreateTablet for tablet 82f76532c7d04012a58fce30b62b344d (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:56.807821 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 82f76532c7d04012a58fce30b62b344d (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:56.809207 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f76532c7d04012a58fce30b62b344d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.809902 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f76532c7d04012a58fce30b62b344d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.811491 22823 tablet_service.cc:1511] Processing CreateTablet for tablet c52c0b7017594a0f8651f1fe9e9f51a2 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:56.813117 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52c0b7017594a0f8651f1fe9e9f51a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.818034 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52c0b7017594a0f8651f1fe9e9f51a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.819504 22905 tablet_service.cc:1511] Processing CreateTablet for tablet 2e04261a736f42d2a1fd4e24bc787276 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:56.828841 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e04261a736f42d2a1fd4e24bc787276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.832705 22898 tablet_service.cc:1511] Processing CreateTablet for tablet c52c0b7017594a0f8651f1fe9e9f51a2 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:56.837172 22958 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:56.846339 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c52c0b7017594a0f8651f1fe9e9f51a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:56.863981 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:56.873903 22958 tablet_bootstrap.cc:654] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.885417 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 4572187533dd409bad9adf0de57c6940 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:56.887320 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4572187533dd409bad9adf0de57c6940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.918919 22960 tablet_bootstrap.cc:492] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:56.919960 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 4572187533dd409bad9adf0de57c6940 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:56.921741 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4572187533dd409bad9adf0de57c6940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.931154 22748 tablet_service.cc:1511] Processing CreateTablet for tablet cae1e4795b8046c9b632ce5907911c39 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:56.931658 22830 tablet_service.cc:1511] Processing CreateTablet for tablet cae1e4795b8046c9b632ce5907911c39 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:56.932650 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae1e4795b8046c9b632ce5907911c39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.928556 22824 tablet_service.cc:1511] Processing CreateTablet for tablet 4572187533dd409bad9adf0de57c6940 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:56.936656 22901 tablet_service.cc:1511] Processing CreateTablet for tablet cae1e4795b8046c9b632ce5907911c39 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:56.938120 22901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae1e4795b8046c9b632ce5907911c39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.938712 22959 tablet_bootstrap.cc:492] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:56.940246 22960 tablet_bootstrap.cc:654] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.944139 22755 tablet_service.cc:1511] Processing CreateTablet for tablet 26c26954cc1042d884cd8813a9346a27 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:56.946156 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae1e4795b8046c9b632ce5907911c39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.947175 22902 tablet_service.cc:1511] Processing CreateTablet for tablet 26c26954cc1042d884cd8813a9346a27 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:56.948526 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c26954cc1042d884cd8813a9346a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.951844 22827 tablet_service.cc:1511] Processing CreateTablet for tablet 485fd0c9273e482692ddc7589cb09f02 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:56.953320 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fd0c9273e482692ddc7589cb09f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.955206 22958 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:56.955732 22958 ts_tablet_manager.cc:1403] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.119s	user 0.009s	sys 0.011s
I20260709 13:10:56.956123 22831 tablet_service.cc:1511] Processing CreateTablet for tablet 26c26954cc1042d884cd8813a9346a27 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:56.962978 22960 tablet_bootstrap.cc:492] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:56.963486 22960 ts_tablet_manager.cc:1403] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.002s
I20260709 13:10:56.963315 22958 raft_consensus.cc:359] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:56.964371 22958 raft_consensus.cc:385] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:56.964669 22958 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:56.966394 22753 tablet_service.cc:1511] Processing CreateTablet for tablet a6da5381f5554dc69fb15991d351fb20 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:56.957211 22959 tablet_bootstrap.cc:654] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:56.957774 22755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c26954cc1042d884cd8813a9346a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.959646 22904 tablet_service.cc:1511] Processing CreateTablet for tablet 485fd0c9273e482692ddc7589cb09f02 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:56.968638 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fd0c9273e482692ddc7589cb09f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.968727 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6da5381f5554dc69fb15991d351fb20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.951844 22749 tablet_service.cc:1511] Processing CreateTablet for tablet 485fd0c9273e482692ddc7589cb09f02 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:56.970530 22749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fd0c9273e482692ddc7589cb09f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.976642 22751 tablet_service.cc:1511] Processing CreateTablet for tablet 38001bb9b85f4814aff1beccb3ae9acb (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:56.978305 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38001bb9b85f4814aff1beccb3ae9acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.978588 22825 tablet_service.cc:1511] Processing CreateTablet for tablet a6da5381f5554dc69fb15991d351fb20 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:56.979539 22905 tablet_service.cc:1511] Processing CreateTablet for tablet a6da5381f5554dc69fb15991d351fb20 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:56.980964 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6da5381f5554dc69fb15991d351fb20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.961735 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4572187533dd409bad9adf0de57c6940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.977780 22823 tablet_service.cc:1511] Processing CreateTablet for tablet 38001bb9b85f4814aff1beccb3ae9acb (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:56.986726 22831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c26954cc1042d884cd8813a9346a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.994298 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38001bb9b85f4814aff1beccb3ae9acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.998720 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6da5381f5554dc69fb15991d351fb20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.998975 22756 tablet_service.cc:1511] Processing CreateTablet for tablet 9318e6ae5d8a4923a2bc67c1c2f6a754 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:57.000470 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9318e6ae5d8a4923a2bc67c1c2f6a754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.005852 22958 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:57.007894 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 38001bb9b85f4814aff1beccb3ae9acb (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:57.009591 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38001bb9b85f4814aff1beccb3ae9acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.013597 22826 tablet_service.cc:1511] Processing CreateTablet for tablet 9318e6ae5d8a4923a2bc67c1c2f6a754 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:57.018316 22754 tablet_service.cc:1511] Processing CreateTablet for tablet 4c3f341bf2b9433090bb7beb432566e1 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:57.027736 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9318e6ae5d8a4923a2bc67c1c2f6a754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.028012 22754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c3f341bf2b9433090bb7beb432566e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.028062 22958 ts_tablet_manager.cc:1434] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.006s
I20260709 13:10:57.029572 22940 heartbeater.cc:499] Master 127.21.33.62:44143 was elected leader, sending a full tablet report...
I20260709 13:10:57.016292 22898 tablet_service.cc:1511] Processing CreateTablet for tablet 9318e6ae5d8a4923a2bc67c1c2f6a754 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:57.031050 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9318e6ae5d8a4923a2bc67c1c2f6a754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.034687 22958 tablet_bootstrap.cc:492] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:57.048580 22958 tablet_bootstrap.cc:654] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.008029 22960 raft_consensus.cc:359] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.050374 22960 raft_consensus.cc:385] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.050953 22960 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.052024 22960 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.062861 22829 tablet_service.cc:1511] Processing CreateTablet for tablet 4c3f341bf2b9433090bb7beb432566e1 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:57.065019 22829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c3f341bf2b9433090bb7beb432566e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.077018 22904 tablet_service.cc:1511] Processing CreateTablet for tablet 4c3f341bf2b9433090bb7beb432566e1 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:57.078670 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c3f341bf2b9433090bb7beb432566e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.080142 22752 tablet_service.cc:1511] Processing CreateTablet for tablet 6d283692c3bb4f979788fa989a74bd80 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:57.081647 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d283692c3bb4f979788fa989a74bd80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.093286 22830 tablet_service.cc:1511] Processing CreateTablet for tablet 6d283692c3bb4f979788fa989a74bd80 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:57.094921 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d283692c3bb4f979788fa989a74bd80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.096962 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 0b88bd2b1b1d4ced918486ea751fbe25 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:57.103852 22828 tablet_service.cc:1511] Processing CreateTablet for tablet 0b88bd2b1b1d4ced918486ea751fbe25 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:57.105079 22959 tablet_bootstrap.cc:492] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:57.105298 22828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b88bd2b1b1d4ced918486ea751fbe25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.105789 22959 ts_tablet_manager.cc:1403] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.168s	user 0.025s	sys 0.014s
I20260709 13:10:57.100611 22900 tablet_service.cc:1511] Processing CreateTablet for tablet 6d283692c3bb4f979788fa989a74bd80 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:57.109983 22903 tablet_service.cc:1511] Processing CreateTablet for tablet 0b88bd2b1b1d4ced918486ea751fbe25 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:57.099215 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b88bd2b1b1d4ced918486ea751fbe25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.115489 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d283692c3bb4f979788fa989a74bd80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.118103 22903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b88bd2b1b1d4ced918486ea751fbe25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.151849 22824 tablet_service.cc:1511] Processing CreateTablet for tablet bd464cd43c544f7abc90c38571db4d1c (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:57.154106 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd464cd43c544f7abc90c38571db4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.109107 22959 raft_consensus.cc:359] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.143487 22748 tablet_service.cc:1511] Processing CreateTablet for tablet bd464cd43c544f7abc90c38571db4d1c (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:57.158766 22791 heartbeater.cc:499] Master 127.21.33.62:44143 was elected leader, sending a full tablet report...
I20260709 13:10:57.162063 22898 tablet_service.cc:1511] Processing CreateTablet for tablet bd464cd43c544f7abc90c38571db4d1c (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:57.164521 22756 tablet_service.cc:1511] Processing CreateTablet for tablet eac8195b1d874d3aa822a4f96dee4738 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:57.157178 22959 raft_consensus.cc:385] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.180150 22866 heartbeater.cc:499] Master 127.21.33.62:44143 was elected leader, sending a full tablet report...
I20260709 13:10:57.180590 22960 ts_tablet_manager.cc:1434] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.217s	user 0.007s	sys 0.024s
I20260709 13:10:57.181679 22960 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:57.182045 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac8195b1d874d3aa822a4f96dee4738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.182133 22901 tablet_service.cc:1511] Processing CreateTablet for tablet eac8195b1d874d3aa822a4f96dee4738 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:57.185559 22823 tablet_service.cc:1511] Processing CreateTablet for tablet eac8195b1d874d3aa822a4f96dee4738 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:57.186869 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd464cd43c544f7abc90c38571db4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.187111 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac8195b1d874d3aa822a4f96dee4738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.191856 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd464cd43c544f7abc90c38571db4d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.192960 22965 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.175233 22959 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.208294 22901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eac8195b1d874d3aa822a4f96dee4738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.204118 22965 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.210268 22959 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.213152 22965 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:57.245947 22830 tablet_service.cc:1511] Processing CreateTablet for tablet a17d5efe4abb490b8dc12b9e45d754ca (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:57.246906 22825 tablet_service.cc:1511] Processing CreateTablet for tablet 1c406ce5efa449d4b877f6747d1f14ab (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:57.248256 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a17d5efe4abb490b8dc12b9e45d754ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.240931 22755 tablet_service.cc:1511] Processing CreateTablet for tablet a17d5efe4abb490b8dc12b9e45d754ca (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:57.262537 22958 tablet_bootstrap.cc:492] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:57.263264 22958 ts_tablet_manager.cc:1403] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.229s	user 0.006s	sys 0.022s
I20260709 13:10:57.266916 22958 raft_consensus.cc:359] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.267716 22958 raft_consensus.cc:385] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.268030 22958 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.278445 22755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a17d5efe4abb490b8dc12b9e45d754ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.293718 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c406ce5efa449d4b877f6747d1f14ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.305459 22751 tablet_service.cc:1511] Processing CreateTablet for tablet 1c406ce5efa449d4b877f6747d1f14ab (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:57.305814 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 3e701ce8413c4f11afc5ca006f97eb50 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:57.307382 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c406ce5efa449d4b877f6747d1f14ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.310431 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e701ce8413c4f11afc5ca006f97eb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.313894 22960 tablet_bootstrap.cc:654] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.323766 22967 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.324364 22967 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.340610 22967 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:57.342929 22826 tablet_service.cc:1511] Processing CreateTablet for tablet 3e701ce8413c4f11afc5ca006f97eb50 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:57.296334 22958 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.361135 22958 ts_tablet_manager.cc:1434] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.098s	user 0.002s	sys 0.006s
I20260709 13:10:57.362773 22958 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:57.350703 22959 ts_tablet_manager.cc:1434] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.245s	user 0.014s	sys 0.031s
I20260709 13:10:57.379123 22900 tablet_service.cc:1511] Processing CreateTablet for tablet 3e701ce8413c4f11afc5ca006f97eb50 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:57.380192 22905 tablet_service.cc:1511] Processing CreateTablet for tablet a17d5efe4abb490b8dc12b9e45d754ca (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:57.380765 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e701ce8413c4f11afc5ca006f97eb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.376168 22964 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.385957 22964 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:57.395201 22959 tablet_bootstrap.cc:492] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:57.391150 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a17d5efe4abb490b8dc12b9e45d754ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.393113 22964 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:10:57.376278 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 1c406ce5efa449d4b877f6747d1f14ab (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:57.427839 22958 tablet_bootstrap.cc:654] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.443965 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c406ce5efa449d4b877f6747d1f14ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.445037 22959 tablet_bootstrap.cc:654] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.431527 22960 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:57.463234 22827 tablet_service.cc:1511] Processing CreateTablet for tablet 11d0cdd09217481cbfc11c1d2d14a5a0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:57.465828 22960 ts_tablet_manager.cc:1403] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.284s	user 0.026s	sys 0.021s
I20260709 13:10:57.474128 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e701ce8413c4f11afc5ca006f97eb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.472740 22904 tablet_service.cc:1511] Processing CreateTablet for tablet 11d0cdd09217481cbfc11c1d2d14a5a0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:57.497227 22902 tablet_service.cc:1511] Processing CreateTablet for tablet af39123de9434cc0b451f4b03d305310 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:57.498103 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d0cdd09217481cbfc11c1d2d14a5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.498741 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af39123de9434cc0b451f4b03d305310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.501546 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d0cdd09217481cbfc11c1d2d14a5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.513227 22825 tablet_service.cc:1511] Processing CreateTablet for tablet af39123de9434cc0b451f4b03d305310 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:57.514892 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af39123de9434cc0b451f4b03d305310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.520795 22960 raft_consensus.cc:359] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.521780 22960 raft_consensus.cc:385] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.522117 22960 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.523136 22960 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.527894 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:10:57.529999 22730 leader_election.cc:343] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.539618 22958 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:57.540236 22958 ts_tablet_manager.cc:1403] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.178s	user 0.004s	sys 0.019s
I20260709 13:10:57.542572 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:57.544569 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:10:57.546782 22807 leader_election.cc:343] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:57.548646 22806 leader_election.cc:343] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.546538 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 11d0cdd09217481cbfc11c1d2d14a5a0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:57.549177 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:57.556767 22903 tablet_service.cc:1511] Processing CreateTablet for tablet e19799c88cba4c9194a9308860a76da7 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:57.558605 22903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19799c88cba4c9194a9308860a76da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.558657 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d0cdd09217481cbfc11c1d2d14a5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.560036 22748 tablet_service.cc:1511] Processing CreateTablet for tablet e19799c88cba4c9194a9308860a76da7 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:57.572486 22830 tablet_service.cc:1511] Processing CreateTablet for tablet 76de9d7042974967a385da8f613f4cb5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:57.574538 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76de9d7042974967a385da8f613f4cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.574769 22965 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.580996 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 76de9d7042974967a385da8f613f4cb5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:57.561432 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19799c88cba4c9194a9308860a76da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.591135 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76de9d7042974967a385da8f613f4cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.558799 22751 tablet_service.cc:1511] Processing CreateTablet for tablet af39123de9434cc0b451f4b03d305310 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:57.591928 22898 tablet_service.cc:1511] Processing CreateTablet for tablet 76de9d7042974967a385da8f613f4cb5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:57.592818 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af39123de9434cc0b451f4b03d305310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.593434 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76de9d7042974967a385da8f613f4cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.593600 22756 tablet_service.cc:1511] Processing CreateTablet for tablet 2a537de05aee48178205ec0e7e0d436e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:57.595281 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a537de05aee48178205ec0e7e0d436e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.568522 22823 tablet_service.cc:1511] Processing CreateTablet for tablet e19799c88cba4c9194a9308860a76da7 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:57.604326 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19799c88cba4c9194a9308860a76da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.615201 22960 ts_tablet_manager.cc:1434] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.149s	user 0.010s	sys 0.000s
I20260709 13:10:57.616380 22960 tablet_bootstrap.cc:492] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:57.621306 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:57.622859 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:10:57.623817 22880 leader_election.cc:343] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:57.624707 22732 leader_election.cc:343] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.625187 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:57.626042 22967 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.629846 22829 tablet_service.cc:1511] Processing CreateTablet for tablet 2a537de05aee48178205ec0e7e0d436e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:57.631744 22829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a537de05aee48178205ec0e7e0d436e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.636543 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:10:57.638433 22879 leader_election.cc:343] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.639055 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:57.639959 22964 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.565421 22958 raft_consensus.cc:359] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.642078 22958 raft_consensus.cc:385] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.642417 22958 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.643424 22958 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.646981 22958 ts_tablet_manager.cc:1434] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.106s	user 0.008s	sys 0.002s
I20260709 13:10:57.648219 22959 tablet_bootstrap.cc:492] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:57.648855 22959 ts_tablet_manager.cc:1403] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.254s	user 0.003s	sys 0.019s
I20260709 13:10:57.685273 22900 tablet_service.cc:1511] Processing CreateTablet for tablet 2a537de05aee48178205ec0e7e0d436e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:57.686851 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a537de05aee48178205ec0e7e0d436e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.691836 22827 tablet_service.cc:1511] Processing CreateTablet for tablet 0818ffaecde64fc0af038b3c024b6008 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:57.693359 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0818ffaecde64fc0af038b3c024b6008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.694279 22960 tablet_bootstrap.cc:654] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.676919 22959 raft_consensus.cc:359] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.697000 22959 raft_consensus.cc:385] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.697551 22959 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.698418 22959 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.700911 22959 ts_tablet_manager.cc:1434] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.052s	user 0.002s	sys 0.006s
I20260709 13:10:57.702188 22959 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:57.703799 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 0818ffaecde64fc0af038b3c024b6008 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:57.714937 22959 tablet_bootstrap.cc:654] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.717329 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0818ffaecde64fc0af038b3c024b6008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.722095 22964 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.723049 22958 tablet_bootstrap.cc:492] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:57.722923 22964 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.727612 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:10:57.733143 22880 leader_election.cc:343] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.733916 22902 tablet_service.cc:1511] Processing CreateTablet for tablet 0818ffaecde64fc0af038b3c024b6008 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:57.735410 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0818ffaecde64fc0af038b3c024b6008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.736999 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:57.738332 22825 tablet_service.cc:1511] Processing CreateTablet for tablet a9fc1639a8fa47ba91cd2fc158066fb2 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 13:10:57.738996 22879 leader_election.cc:343] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.739501 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:57.739805 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9fc1639a8fa47ba91cd2fc158066fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.741766 22756 tablet_service.cc:1511] Processing CreateTablet for tablet a9fc1639a8fa47ba91cd2fc158066fb2 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:57.743263 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9fc1639a8fa47ba91cd2fc158066fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.749200 22823 tablet_service.cc:1511] Processing CreateTablet for tablet 659f33da809544df8f5fc087bbe7cc17 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:57.751106 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659f33da809544df8f5fc087bbe7cc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.745297 22964 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:57.749738 22958 tablet_bootstrap.cc:654] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.751962 22964 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:57.743069 22898 tablet_service.cc:1511] Processing CreateTablet for tablet a9fc1639a8fa47ba91cd2fc158066fb2 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:57.753381 22904 tablet_service.cc:1511] Processing CreateTablet for tablet 659f33da809544df8f5fc087bbe7cc17 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:57.751111 22748 tablet_service.cc:1511] Processing CreateTablet for tablet 659f33da809544df8f5fc087bbe7cc17 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:57.757867 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9fc1639a8fa47ba91cd2fc158066fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.764492 22960 tablet_bootstrap.cc:492] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:57.764251 22753 tablet_service.cc:1511] Processing CreateTablet for tablet b00e5613b4864bd6a0873aee1004ed22 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:57.765368 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659f33da809544df8f5fc087bbe7cc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.765949 22960 ts_tablet_manager.cc:1403] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.150s	user 0.016s	sys 0.004s
I20260709 13:10:57.769317 22829 tablet_service.cc:1511] Processing CreateTablet for tablet b00e5613b4864bd6a0873aee1004ed22 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:57.768074 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00e5613b4864bd6a0873aee1004ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.772492 22829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00e5613b4864bd6a0873aee1004ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.773761 22751 tablet_service.cc:1511] Processing CreateTablet for tablet 072f69118119423dac7eba3f66879eee (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:57.774850 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659f33da809544df8f5fc087bbe7cc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.783476 22830 tablet_service.cc:1511] Processing CreateTablet for tablet 072f69118119423dac7eba3f66879eee (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:57.785228 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 072f69118119423dac7eba3f66879eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.797963 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 072f69118119423dac7eba3f66879eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.811115 22900 tablet_service.cc:1511] Processing CreateTablet for tablet b00e5613b4864bd6a0873aee1004ed22 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:57.818687 22960 raft_consensus.cc:359] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.819573 22960 raft_consensus.cc:385] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.820075 22960 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.820915 22960 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.822039 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00e5613b4864bd6a0873aee1004ed22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.829145 22958 tablet_bootstrap.cc:492] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:57.832608 22903 tablet_service.cc:1511] Processing CreateTablet for tablet 072f69118119423dac7eba3f66879eee (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:57.837990 22958 ts_tablet_manager.cc:1403] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.115s	user 0.009s	sys 0.009s
I20260709 13:10:57.838655 22903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 072f69118119423dac7eba3f66879eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.839926 22751 tablet_service.cc:1511] Processing CreateTablet for tablet 11f984a583c84fbe9ef7b3c817b06c44 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:57.841580 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f984a583c84fbe9ef7b3c817b06c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.844945 22958 raft_consensus.cc:359] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.845772 22958 raft_consensus.cc:385] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.846094 22958 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.852090 22960 ts_tablet_manager.cc:1434] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.000s
I20260709 13:10:57.852368 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 11f984a583c84fbe9ef7b3c817b06c44 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:57.853446 22960 tablet_bootstrap.cc:492] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:57.854233 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f984a583c84fbe9ef7b3c817b06c44. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:57.866705 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:57.868448 22965 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.869059 22965 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.873450 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:57.874408 22915 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:57.875253 22965 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:57.877019 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:57.878242 22959 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:57.877982 22830 tablet_service.cc:1511] Processing CreateTablet for tablet 11f984a583c84fbe9ef7b3c817b06c44 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:57.878998 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:10:57.879717 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f984a583c84fbe9ef7b3c817b06c44. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:57.881165 22806 leader_election.cc:343] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.884146 22827 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7e34bc1d8d404aae87baaaa171e0b1 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:57.885941 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7e34bc1d8d404aae87baaaa171e0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.885818 22965 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.879143 22959 ts_tablet_manager.cc:1403] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.177s	user 0.041s	sys 0.054s
I20260709 13:10:57.886307 22965 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.886713 22965 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.891812 22901 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7e34bc1d8d404aae87baaaa171e0b1 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:57.893967 22958 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.892320 22967 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.895727 22967 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.896457 22959 raft_consensus.cc:359] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:57.897280 22959 raft_consensus.cc:385] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:57.897665 22959 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:57.898200 22967 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:57.899979 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:57.900695 22841 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:10:57.902724 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:57.905575 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:57.906881 22958 ts_tablet_manager.cc:1434] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 13:10:57.907526 22967 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:57.908391 22967 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.909296 22967 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.893407 22901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7e34bc1d8d404aae87baaaa171e0b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:57.906988 22732 leader_election.cc:343] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:57.898852 22959 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:57.929160 22960 tablet_bootstrap.cc:654] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:57.940744 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 8670cf017371429ea31529b6f247493e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:57.926887 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7e34bc1d8d404aae87baaaa171e0b1 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:57.944269 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8670cf017371429ea31529b6f247493e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.944936 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7e34bc1d8d404aae87baaaa171e0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.944691 22823 tablet_service.cc:1511] Processing CreateTablet for tablet 8670cf017371429ea31529b6f247493e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:57.908430 22958 tablet_bootstrap.cc:492] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:57.950884 22756 tablet_service.cc:1511] Processing CreateTablet for tablet 023c6d83f2864eedb2775b615045955e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:57.954617 22900 tablet_service.cc:1511] Processing CreateTablet for tablet 8670cf017371429ea31529b6f247493e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:57.947333 22965 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.956674 22829 tablet_service.cc:1511] Processing CreateTablet for tablet 023c6d83f2864eedb2775b615045955e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:57.960180 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c6d83f2864eedb2775b615045955e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.966542 22989 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.967589 22989 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:57.968464 22959 ts_tablet_manager.cc:1434] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.009s
I20260709 13:10:57.968794 22903 tablet_service.cc:1511] Processing CreateTablet for tablet 023c6d83f2864eedb2775b615045955e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:57.970640 22748 tablet_service.cc:1511] Processing CreateTablet for tablet 17f68c054a034150b6ed9a31489ec653 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:57.973491 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8670cf017371429ea31529b6f247493e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.974713 22967 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:57.964764 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8670cf017371429ea31529b6f247493e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.983243 22755 tablet_service.cc:1511] Processing CreateTablet for tablet ba1db22a7176432c86e7264f87984ba3 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:57.984602 22903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c6d83f2864eedb2775b615045955e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.983952 22967 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:57.975196 22959 tablet_bootstrap.cc:492] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:57.976783 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f68c054a034150b6ed9a31489ec653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.988979 22898 tablet_service.cc:1511] Processing CreateTablet for tablet 17f68c054a034150b6ed9a31489ec653 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:57.989557 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:10:57.990231 22841 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:57.990427 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f68c054a034150b6ed9a31489ec653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:57.978271 22825 tablet_service.cc:1511] Processing CreateTablet for tablet 17f68c054a034150b6ed9a31489ec653 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:57.993974 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:57.994729 22829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c6d83f2864eedb2775b615045955e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:57.995750 22732 leader_election.cc:343] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.002131 22755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1db22a7176432c86e7264f87984ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.006246 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f68c054a034150b6ed9a31489ec653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.042708 22958 tablet_bootstrap.cc:654] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.044803 22959 tablet_bootstrap.cc:654] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.051524 22827 tablet_service.cc:1511] Processing CreateTablet for tablet ba1db22a7176432c86e7264f87984ba3 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:58.057984 22960 tablet_bootstrap.cc:492] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:58.058583 22960 ts_tablet_manager.cc:1403] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.205s	user 0.017s	sys 0.017s
I20260709 13:10:58.060107 22751 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad66b5c9842437d9320507b92b78fd5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:58.060715 22965 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.061074 22841 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:10:58.061280 22965 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.086432 22986 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.086256 22965 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.062868 22904 tablet_service.cc:1511] Processing CreateTablet for tablet ba1db22a7176432c86e7264f87984ba3 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:58.087008 22986 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.088021 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1db22a7176432c86e7264f87984ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.063699 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:10:58.089833 22965 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.064759 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1db22a7176432c86e7264f87984ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.064886 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad66b5c9842437d9320507b92b78fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.065591 22960 raft_consensus.cc:359] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.068186 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:58.093585 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:58.094429 22915 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:10:58.097725 22826 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad66b5c9842437d9320507b92b78fd5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:58.097963 22828 tablet_service.cc:1511] Processing CreateTablet for tablet 09deb1f2792d4748b024f33e42736814 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:58.098392 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad66b5c9842437d9320507b92b78fd5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:58.099828 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad66b5c9842437d9320507b92b78fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.100829 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:58.102061 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:58.100929 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:58.102572 22752 tablet_service.cc:1511] Processing CreateTablet for tablet 09deb1f2792d4748b024f33e42736814 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 13:10:58.103142 22880 leader_election.cc:343] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.103864 22988 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.099622 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:58.104435 22988 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.079296 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:58.079386 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:58.104099 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09deb1f2792d4748b024f33e42736814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.070124 22989 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:58.109359 22914 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:58.109967 22807 leader_election.cc:343] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.111450 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:10:58.113052 22730 leader_election.cc:343] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.093726 22960 raft_consensus.cc:385] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.113732 22967 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.113838 22960 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
W20260709 13:10:58.115365 22879 leader_election.cc:343] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.116031 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.116946 22964 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.092094 22986 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:58.104836 22988 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.132786 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad66b5c9842437d9320507b92b78fd5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:58.134014 22806 leader_election.cc:343] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.151005 22967 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:10:58.166754 22905 tablet_service.cc:1511] Processing CreateTablet for tablet 09deb1f2792d4748b024f33e42736814 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:58.167660 22967 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.168702 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09deb1f2792d4748b024f33e42736814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.170811 22902 tablet_service.cc:1511] Processing CreateTablet for tablet d6a04bfd7f184dd7a86718230ea0680e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:58.172431 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a04bfd7f184dd7a86718230ea0680e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.175664 22831 tablet_service.cc:1511] Processing CreateTablet for tablet 961d7db1909a4e40a4b410a32dad4d64 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:58.175951 22830 tablet_service.cc:1511] Processing CreateTablet for tablet d6a04bfd7f184dd7a86718230ea0680e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 13:10:58.162590 22806 leader_election.cc:343] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.192291 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.194376 22965 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.197962 22750 tablet_service.cc:1511] Processing CreateTablet for tablet d6a04bfd7f184dd7a86718230ea0680e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:58.198860 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 961d7db1909a4e40a4b410a32dad4d64 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:58.164932 22828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09deb1f2792d4748b024f33e42736814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.206256 22831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 961d7db1909a4e40a4b410a32dad4d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.205812 22897 tablet_service.cc:1511] Processing CreateTablet for tablet 961d7db1909a4e40a4b410a32dad4d64 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:58.208627 22897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 961d7db1909a4e40a4b410a32dad4d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.213321 22749 tablet_service.cc:1511] Processing CreateTablet for tablet f2aaf3c1715e4d29954bd6ee38914a7a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:58.215770 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a04bfd7f184dd7a86718230ea0680e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.214654 22959 tablet_bootstrap.cc:492] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:58.216650 22959 ts_tablet_manager.cc:1403] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.242s	user 0.017s	sys 0.014s
I20260709 13:10:58.217861 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 961d7db1909a4e40a4b410a32dad4d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.221882 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a04bfd7f184dd7a86718230ea0680e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.222126 22749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2aaf3c1715e4d29954bd6ee38914a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.211572 22960 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.227833 22960 ts_tablet_manager.cc:1434] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.169s	user 0.012s	sys 0.000s
I20260709 13:10:58.229179 22960 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:58.250283 22959 raft_consensus.cc:359] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.266351 22824 tablet_service.cc:1511] Processing CreateTablet for tablet f2aaf3c1715e4d29954bd6ee38914a7a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:58.249756 22958 tablet_bootstrap.cc:492] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:58.297134 22823 tablet_service.cc:1511] Processing CreateTablet for tablet a18b5b24775b48b589edb90581a5cad5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:58.286343 22901 tablet_service.cc:1511] Processing CreateTablet for tablet f2aaf3c1715e4d29954bd6ee38914a7a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:58.292672 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2aaf3c1715e4d29954bd6ee38914a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.298166 22958 ts_tablet_manager.cc:1403] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.390s	user 0.025s	sys 0.020s
I20260709 13:10:58.282824 22959 raft_consensus.cc:385] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.301857 22959 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.302827 22959 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.205240 22914 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:10:58.307004 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.308367 22965 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.308928 22965 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:10:58.310534 22965 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.322854 22988 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:58.325192 22988 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:58.293678 22755 tablet_service.cc:1511] Processing CreateTablet for tablet a18b5b24775b48b589edb90581a5cad5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:58.346607 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:10:58.349562 22730 leader_election.cc:343] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.360720 22958 raft_consensus.cc:359] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.361618 22958 raft_consensus.cc:385] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.361931 22958 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.363149 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a18b5b24775b48b589edb90581a5cad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.362843 22958 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.346535 22901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2aaf3c1715e4d29954bd6ee38914a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.372346 22959 ts_tablet_manager.cc:1434] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.155s	user 0.018s	sys 0.000s
I20260709 13:10:58.326925 22664 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.376958 22755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a18b5b24775b48b589edb90581a5cad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.391188 22958 ts_tablet_manager.cc:1434] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.000s
I20260709 13:10:58.393045 22958 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:58.398204 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:58.399466 22914 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.408689 22897 tablet_service.cc:1511] Processing CreateTablet for tablet a18b5b24775b48b589edb90581a5cad5 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:58.411528 22960 tablet_bootstrap.cc:654] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.419848 22959 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:58.420867 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 25ae06dc4f8940f1a419a39f791d6e18 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:58.422341 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25ae06dc4f8940f1a419a39f791d6e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.422765 22897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a18b5b24775b48b589edb90581a5cad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.426354 22831 tablet_service.cc:1511] Processing CreateTablet for tablet 25ae06dc4f8940f1a419a39f791d6e18 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:58.428136 22831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25ae06dc4f8940f1a419a39f791d6e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.432389 22959 tablet_bootstrap.cc:654] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.445626 22662 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.450657 22752 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf57270bf9649da86a032e1c9ce4bd6 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:58.453158 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf57270bf9649da86a032e1c9ce4bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.469300 22965 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.470192 22958 tablet_bootstrap.cc:654] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.470139 22965 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.428996 22900 tablet_service.cc:1511] Processing CreateTablet for tablet 25ae06dc4f8940f1a419a39f791d6e18 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:58.474109 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:58.476261 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25ae06dc4f8940f1a419a39f791d6e18. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:58.477715 22806 leader_election.cc:343] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.478405 22825 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf57270bf9649da86a032e1c9ce4bd6 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:58.481900 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf57270bf9649da86a032e1c9ce4bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.481779 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:58.482579 22915 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:58.483994 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.484266 22959 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:58.484793 22959 ts_tablet_manager.cc:1403] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.065s	user 0.013s	sys 0.035s
I20260709 13:10:58.485105 22991 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.479722 22965 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.486083 22991 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.486444 22991 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.488368 22959 raft_consensus.cc:359] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.489116 22959 raft_consensus.cc:385] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.489382 22959 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.490358 22959 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.494228 22959 ts_tablet_manager.cc:1434] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.005s
I20260709 13:10:58.498276 22914 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:10:58.499680 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:58.499826 22905 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf57270bf9649da86a032e1c9ce4bd6 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:58.500960 22988 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.501457 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf57270bf9649da86a032e1c9ce4bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.501483 22988 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:10:58.505594 22988 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:58.507001 22991 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.518584 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:10:58.521309 22806 leader_election.cc:343] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.522106 22827 tablet_service.cc:1511] Processing CreateTablet for tablet a663ecdecbd14089a4074b2581e94517 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:58.523044 22750 tablet_service.cc:1511] Processing CreateTablet for tablet a663ecdecbd14089a4074b2581e94517 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:58.523545 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a663ecdecbd14089a4074b2581e94517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.523172 22904 tablet_service.cc:1511] Processing CreateTablet for tablet a663ecdecbd14089a4074b2581e94517 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:58.524396 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a663ecdecbd14089a4074b2581e94517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.524573 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a663ecdecbd14089a4074b2581e94517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.511404 22991 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.516155 22960 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:58.531503 22960 ts_tablet_manager.cc:1403] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.303s	user 0.007s	sys 0.016s
I20260709 13:10:58.533102 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 2f666ea4542d4951919f36de0b43b972 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:58.533785 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:58.541038 22914 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.537815 22959 tablet_bootstrap.cc:492] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:58.543416 22960 raft_consensus.cc:359] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.544246 22960 raft_consensus.cc:385] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.544548 22960 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.549180 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f666ea4542d4951919f36de0b43b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.562623 22986 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.563210 22986 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.563421 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 2f666ea4542d4951919f36de0b43b972 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:58.563674 22824 tablet_service.cc:1511] Processing CreateTablet for tablet 2f666ea4542d4951919f36de0b43b972 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:58.564869 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f666ea4542d4951919f36de0b43b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.565013 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f666ea4542d4951919f36de0b43b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.565477 22986 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:58.567381 22749 tablet_service.cc:1511] Processing CreateTablet for tablet f12b50ff391746348739bd5f7a08a44a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:58.570472 22826 tablet_service.cc:1511] Processing CreateTablet for tablet f12b50ff391746348739bd5f7a08a44a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:58.570818 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:58.568735 22902 tablet_service.cc:1511] Processing CreateTablet for tablet f12b50ff391746348739bd5f7a08a44a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:58.572647 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b50ff391746348739bd5f7a08a44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.573040 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b50ff391746348739bd5f7a08a44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.574148 22756 tablet_service.cc:1511] Processing CreateTablet for tablet adf76c1fde374ccbbb4a0edda126b851 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:58.566283 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:58.571630 22839 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
W20260709 13:10:58.576705 22880 leader_election.cc:343] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.568698 22958 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:58.577282 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.577940 22958 ts_tablet_manager.cc:1403] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.186s	user 0.017s	sys 0.000s
I20260709 13:10:58.578815 22986 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.579247 22986 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.579631 22986 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.583105 22830 tablet_service.cc:1511] Processing CreateTablet for tablet adf76c1fde374ccbbb4a0edda126b851 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:58.584532 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf76c1fde374ccbbb4a0edda126b851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.570919 22749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b50ff391746348739bd5f7a08a44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.586133 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf76c1fde374ccbbb4a0edda126b851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.589488 22958 raft_consensus.cc:359] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.590299 22958 raft_consensus.cc:385] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.590602 22958 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.591524 22958 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.557173 22960 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.624650 22960 ts_tablet_manager.cc:1434] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.093s	user 0.002s	sys 0.006s
I20260709 13:10:58.625700 22959 tablet_bootstrap.cc:654] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.631781 22960 tablet_bootstrap.cc:492] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:58.637094 22914 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:10:58.639058 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.640897 22991 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.641426 22991 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:10:58.642920 22958 ts_tablet_manager.cc:1434] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:10:58.643128 22991 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.645720 22904 tablet_service.cc:1511] Processing CreateTablet for tablet adf76c1fde374ccbbb4a0edda126b851 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:58.647176 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf76c1fde374ccbbb4a0edda126b851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.660913 22958 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
W20260709 13:10:58.668614 22732 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:58.680733 22959 tablet_bootstrap.cc:492] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:58.681340 22959 ts_tablet_manager.cc:1403] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.154s	user 0.017s	sys 0.019s
I20260709 13:10:58.682852 22749 tablet_service.cc:1511] Processing CreateTablet for tablet bac9a57b26684fe6a49a43a8a61151e6 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:58.685009 22749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac9a57b26684fe6a49a43a8a61151e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.684803 22959 raft_consensus.cc:359] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.686335 22959 raft_consensus.cc:385] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.686661 22959 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.687783 22959 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.693789 22958 tablet_bootstrap.cc:654] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.694563 22988 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:58.697939 22959 ts_tablet_manager.cc:1434] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:10:58.698032 22986 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.699491 22960 tablet_bootstrap.cc:654] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.699875 22965 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.701565 22965 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.702188 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:10:58.703641 22880 leader_election.cc:343] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.703708 22965 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.707144 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:58.707484 22827 tablet_service.cc:1511] Processing CreateTablet for tablet bac9a57b26684fe6a49a43a8a61151e6 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 13:10:58.711817 22806 leader_election.cc:343] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.712167 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac9a57b26684fe6a49a43a8a61151e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.715195 22830 tablet_service.cc:1511] Processing CreateTablet for tablet 0670a9005da848cd93427b2d2b4aa41d (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:58.717105 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0670a9005da848cd93427b2d2b4aa41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.719301 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:10:58.721899 22807 leader_election.cc:343] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.722347 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.722540 22752 tablet_service.cc:1511] Processing CreateTablet for tablet 0670a9005da848cd93427b2d2b4aa41d (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:10:58.724355 22806 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:58.724506 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0670a9005da848cd93427b2d2b4aa41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.722540 22905 tablet_service.cc:1511] Processing CreateTablet for tablet bac9a57b26684fe6a49a43a8a61151e6 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:58.725764 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bac9a57b26684fe6a49a43a8a61151e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.725224 22965 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.728004 22986 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:58.729203 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:10:58.729839 22840 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.719135 22959 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:58.744768 22664 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.761193 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 9951bee4cd6b4966bef18c1e0d25eaca (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:58.762634 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9951bee4cd6b4966bef18c1e0d25eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.763289 22662 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.761193 22902 tablet_service.cc:1511] Processing CreateTablet for tablet 0670a9005da848cd93427b2d2b4aa41d (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:58.771128 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0670a9005da848cd93427b2d2b4aa41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.757045 22840 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:10:58.776401 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:10:58.777666 22986 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.779170 22986 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:10:58.780078 22986 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.784422 22958 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:58.784972 22958 ts_tablet_manager.cc:1403] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.124s	user 0.027s	sys 0.029s
I20260709 13:10:58.787747 22994 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.788286 22994 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.790648 22994 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.793872 22826 tablet_service.cc:1511] Processing CreateTablet for tablet 9951bee4cd6b4966bef18c1e0d25eaca (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:58.794440 22756 tablet_service.cc:1511] Processing CreateTablet for tablet bb34c8c8aec0452ab45aaa676b75558a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:58.795487 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9951bee4cd6b4966bef18c1e0d25eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.795800 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb34c8c8aec0452ab45aaa676b75558a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.795751 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:10:58.798557 22732 leader_election.cc:343] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.798760 22904 tablet_service.cc:1511] Processing CreateTablet for tablet 9951bee4cd6b4966bef18c1e0d25eaca (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:58.800077 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9951bee4cd6b4966bef18c1e0d25eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.744797 22959 tablet_bootstrap.cc:654] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.809540 22823 tablet_service.cc:1511] Processing CreateTablet for tablet bb34c8c8aec0452ab45aaa676b75558a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:58.788329 22958 raft_consensus.cc:359] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.810989 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb34c8c8aec0452ab45aaa676b75558a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.811569 22960 tablet_bootstrap.cc:492] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:58.812203 22960 ts_tablet_manager.cc:1403] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.187s	user 0.011s	sys 0.005s
I20260709 13:10:58.814284 22755 tablet_service.cc:1511] Processing CreateTablet for tablet 4dd4e5f26303415aa699a9fd74ac17e8 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:58.814592 22897 tablet_service.cc:1511] Processing CreateTablet for tablet bb34c8c8aec0452ab45aaa676b75558a (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:58.815725 22755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4e5f26303415aa699a9fd74ac17e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.815821 22960 raft_consensus.cc:359] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.815750 22897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb34c8c8aec0452ab45aaa676b75558a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.825879 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:58.810977 22958 raft_consensus.cc:385] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:10:58.845702 22730 leader_election.cc:343] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.846277 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:58.846381 22904 tablet_service.cc:1511] Processing CreateTablet for tablet 4dd4e5f26303415aa699a9fd74ac17e8 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:58.847597 22967 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.853953 22904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4e5f26303415aa699a9fd74ac17e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.836391 22960 raft_consensus.cc:385] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.856477 22830 tablet_service.cc:1511] Processing CreateTablet for tablet 4dd4e5f26303415aa699a9fd74ac17e8 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:58.857759 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4e5f26303415aa699a9fd74ac17e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.843674 22958 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.870823 22662 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.856333 22960 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.874347 22960 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.881727 22958 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.884730 22958 ts_tablet_manager.cc:1434] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.000s
W20260709 13:10:58.887003 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:58.890281 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 4f79e81f486b4e7dbf2d845891b6b9c0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:58.895627 22965 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:58.896410 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f79e81f486b4e7dbf2d845891b6b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.892750 22960 ts_tablet_manager.cc:1434] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.001s
I20260709 13:10:58.897810 22960 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:58.892668 22752 tablet_service.cc:1511] Processing CreateTablet for tablet f55844ac3c9a4ceaa35b3d5bdadad40e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:58.893121 22756 tablet_service.cc:1511] Processing CreateTablet for tablet 4f79e81f486b4e7dbf2d845891b6b9c0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:58.901386 22756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f79e81f486b4e7dbf2d845891b6b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.901614 22967 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.902087 22967 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.904421 22967 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.900915 22830 tablet_service.cc:1511] Processing CreateTablet for tablet f55844ac3c9a4ceaa35b3d5bdadad40e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:58.906741 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:58.907917 22830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55844ac3c9a4ceaa35b3d5bdadad40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.908448 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55844ac3c9a4ceaa35b3d5bdadad40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.907922 22915 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:10:58.909888 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:58.910298 22958 tablet_bootstrap.cc:492] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:58.910611 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:58.911794 22967 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:58.894887 22826 tablet_service.cc:1511] Processing CreateTablet for tablet 4f79e81f486b4e7dbf2d845891b6b9c0 (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:58.912863 22967 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.913457 22967 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:58.913951 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f79e81f486b4e7dbf2d845891b6b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.888733 22905 tablet_service.cc:1511] Processing CreateTablet for tablet f55844ac3c9a4ceaa35b3d5bdadad40e (DEFAULT_TABLE table=test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:58.918412 22905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55844ac3c9a4ceaa35b3d5bdadad40e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:58.912317 22730 leader_election.cc:343] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.927134 22967 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.930550 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:10:58.930876 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:58.931438 22915 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:58.932220 22730 leader_election.cc:343] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:58.936336 22959 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:58.948046 22958 tablet_bootstrap.cc:654] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.956880 22915 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:10:58.960005 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:58.961088 22988 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:58.961679 22988 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:10:58.962446 22988 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.964342 22958 tablet_bootstrap.cc:492] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:58.964941 22958 ts_tablet_manager.cc:1403] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.005s
I20260709 13:10:58.952929 22967 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.953280 22959 ts_tablet_manager.cc:1403] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.234s	user 0.021s	sys 0.002s
I20260709 13:10:58.968022 22994 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.968431 22960 tablet_bootstrap.cc:654] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:58.968559 22994 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.970914 22994 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:58.973917 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:58.977008 22959 raft_consensus.cc:359] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.977872 22959 raft_consensus.cc:385] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.978166 22959 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.979473 22959 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.982195 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:58.983163 22915 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:10:58.984524 22958 raft_consensus.cc:359] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.985065 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:58.985350 22958 raft_consensus.cc:385] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:58.985671 22958 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:58.986374 22958 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:58.991087 22959 ts_tablet_manager.cc:1434] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:10:58.990774 22662 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.992297 22959 tablet_bootstrap.cc:492] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:58.995332 22994 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.005115 22959 tablet_bootstrap.cc:654] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.008212 22994 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:58.989285 22958 ts_tablet_manager.cc:1434] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.003s
I20260709 13:10:59.016523 22960 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.017133 22960 ts_tablet_manager.cc:1403] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.004s
I20260709 13:10:59.019915 22960 raft_consensus.cc:359] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.020473 22960 raft_consensus.cc:385] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.020658 22960 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.021358 22960 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.022590 22958 tablet_bootstrap.cc:492] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.024761 22960 ts_tablet_manager.cc:1434] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:10:59.025014 22959 tablet_bootstrap.cc:492] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:59.025507 22959 ts_tablet_manager.cc:1403] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.016s
I20260709 13:10:59.026165 22960 tablet_bootstrap.cc:492] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.028716 22959 raft_consensus.cc:359] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.029558 22959 raft_consensus.cc:385] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.029879 22959 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.030342 22958 tablet_bootstrap.cc:654] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.030931 22959 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.034236 22959 ts_tablet_manager.cc:1434] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:10:59.035068 22960 tablet_bootstrap.cc:654] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.038681 22959 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.041059 22914 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.047700 22960 tablet_bootstrap.cc:492] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.048254 22960 ts_tablet_manager.cc:1403] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.003s
I20260709 13:10:59.050766 22959 tablet_bootstrap.cc:654] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.051544 22960 raft_consensus.cc:359] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.052251 22960 raft_consensus.cc:385] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.052829 22960 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.053978 22960 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.056677 22960 ts_tablet_manager.cc:1434] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:10:59.057760 22960 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.063953 23015 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.086676 22960 tablet_bootstrap.cc:654] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.089082 22994 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.096282 22959 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:59.097011 22959 ts_tablet_manager.cc:1403] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.033s
I20260709 13:10:59.101452 22959 raft_consensus.cc:359] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.102722 22959 raft_consensus.cc:385] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.103020 22959 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.103770 22959 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.106678 22959 ts_tablet_manager.cc:1434] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.005s
I20260709 13:10:59.107690 22959 tablet_bootstrap.cc:492] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.108620 22958 tablet_bootstrap.cc:492] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.109114 22958 ts_tablet_manager.cc:1403] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.087s	user 0.025s	sys 0.033s
I20260709 13:10:59.112128 22958 raft_consensus.cc:359] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.112846 22958 raft_consensus.cc:385] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.113104 22958 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.113886 22958 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.115048 22959 tablet_bootstrap.cc:654] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.116140 22958 ts_tablet_manager.cc:1434] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
W20260709 13:10:59.116256 22730 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:59.117327 22958 tablet_bootstrap.cc:492] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.122337 22994 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.122869 22994 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.123799 22958 tablet_bootstrap.cc:654] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.125144 22994 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.129606 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.130776 22839 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:10:59.131078 22960 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.131850 22960 ts_tablet_manager.cc:1403] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.074s	user 0.015s	sys 0.006s
I20260709 13:10:59.134013 22766 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.134227 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.135974 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:59.136826 22994 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.148172 23015 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.149248 23015 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.151484 22986 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.152185 22986 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.153548 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.154433 22764 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:59.156342 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.156754 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.157680 22965 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.158200 22965 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.158649 22965 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.159505 22986 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.160395 22839 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:10:59.162416 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:10:59.164026 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.164531 23015 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.165325 22764 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:10:59.168470 22986 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.169094 22986 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.169588 22986 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.169761 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.171096 22915 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:59.183092 22994 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.186465 22965 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.187649 22958 tablet_bootstrap.cc:492] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.188233 22958 ts_tablet_manager.cc:1403] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.071s	user 0.034s	sys 0.030s
W20260709 13:10:59.189189 23024 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:10:59.189883 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:10:59.190492 22764 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.192085 22958 raft_consensus.cc:359] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.192860 22958 raft_consensus.cc:385] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.193181 22958 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.194087 22958 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.197741 22958 ts_tablet_manager.cc:1434] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:59.198817 22958 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.199795 23015 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.206296 22958 tablet_bootstrap.cc:654] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.221993 22964 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.223357 22958 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.223973 22958 ts_tablet_manager.cc:1403] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.000s
I20260709 13:10:59.226109 22986 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.227250 22958 raft_consensus.cc:359] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.230988 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:10:59.234673 22839 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.241580 22959 tablet_bootstrap.cc:492] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:59.242219 22959 ts_tablet_manager.cc:1403] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.135s	user 0.030s	sys 0.055s
I20260709 13:10:59.228180 22958 raft_consensus.cc:385] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.230142 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:10:59.231097 22986 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.244194 22766 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.245648 22959 raft_consensus.cc:359] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.246639 22959 raft_consensus.cc:385] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.246979 22959 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.248165 22959 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.250869 22959 ts_tablet_manager.cc:1434] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:10:59.251978 22959 tablet_bootstrap.cc:492] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.257716 22964 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:59.259159 22959 tablet_bootstrap.cc:654] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.199451 22960 raft_consensus.cc:359] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:59.262892 22958 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.263726 22960 raft_consensus.cc:385] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.266183 22960 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.267288 22960 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:59.268779 22839 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:10:59.268479 22764 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:10:59.263790 22958 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.271226 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.272308 23030 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:59.272842 23030 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:10:59.253476 22965 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.273756 23030 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.275168 22958 ts_tablet_manager.cc:1434] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:10:59.276165 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:10:59.275527 22960 ts_tablet_manager.cc:1434] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.143s	user 0.009s	sys 0.001s
I20260709 13:10:59.276615 22958 tablet_bootstrap.cc:492] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.281014 22766 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:10:59.289013 22958 tablet_bootstrap.cc:654] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.291189 22964 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:59.307106 22986 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.307858 22986 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.311846 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:59.312734 22915 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.292703 22960 tablet_bootstrap.cc:492] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.325943 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.327039 22765 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:10:59.338042 22986 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.340855 22964 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:10:59.348842 22964 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.354167 22959 tablet_bootstrap.cc:492] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:59.354905 22959 ts_tablet_manager.cc:1403] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.103s	user 0.025s	sys 0.016s
I20260709 13:10:59.355087 22664 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:59.356850 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.359030 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.362301 22986 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.362855 22986 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.363350 22986 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:59.367188 22879 leader_election.cc:343] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.370105 23030 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.370656 23030 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.373230 23030 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.374912 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.383623 22914 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:10:59.374994 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.386807 22960 tablet_bootstrap.cc:654] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.379851 22986 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.388403 22806 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.391026 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:10:59.391640 22765 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.393170 22965 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.396347 22967 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.400513 22986 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.388326 22959 raft_consensus.cc:359] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.401901 22959 raft_consensus.cc:385] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.402561 22959 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.402864 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:10:59.405903 22879 leader_election.cc:343] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.406136 22965 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.406466 22959 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.407194 22765 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:10:59.404481 22915 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:10:59.409204 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:59.411159 22959 ts_tablet_manager.cc:1434] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.056s	user 0.003s	sys 0.004s
I20260709 13:10:59.412954 22958 tablet_bootstrap.cc:492] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.413589 22958 ts_tablet_manager.cc:1403] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.137s	user 0.026s	sys 0.054s
I20260709 13:10:59.414987 22986 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:59.415753 22986 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:10:59.416695 22986 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.426349 22958 raft_consensus.cc:359] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.427173 22965 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.427587 22958 raft_consensus.cc:385] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.427785 22965 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } }
I20260709 13:10:59.428002 22958 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.430055 22965 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:10:59.433347 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.436488 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.432687 22661 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:59.438616 22807 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.441015 22965 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.442526 22959 tablet_bootstrap.cc:492] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.434585 22958 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.452916 22958 ts_tablet_manager.cc:1434] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 13:10:59.454807 22958 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.465117 22958 tablet_bootstrap.cc:654] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.466953 22959 tablet_bootstrap.cc:654] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.473155 22988 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.485031 22959 tablet_bootstrap.cc:492] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:59.485186 22960 tablet_bootstrap.cc:492] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.485790 22959 ts_tablet_manager.cc:1403] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.004s
I20260709 13:10:59.490621 22662 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:59.492559 22959 raft_consensus.cc:359] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.485782 22960 ts_tablet_manager.cc:1403] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.193s	user 0.035s	sys 0.037s
I20260709 13:10:59.497229 22988 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.497812 22988 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.497678 22960 raft_consensus.cc:359] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.498723 22960 raft_consensus.cc:385] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.499246 22960 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.500119 22960 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.501314 22959 raft_consensus.cc:385] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.501483 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.501758 22959 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.502878 22959 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:10:59.503706 22732 leader_election.cc:343] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.506428 22959 ts_tablet_manager.cc:1434] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:10:59.508798 22959 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.510447 22958 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.510701 22988 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.510967 22958 ts_tablet_manager.cc:1403] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.056s	user 0.030s	sys 0.017s
I20260709 13:10:59.512614 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.514529 22959 tablet_bootstrap.cc:654] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.514130 22958 raft_consensus.cc:359] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.515110 22958 raft_consensus.cc:385] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.515430 22958 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.516134 22958 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.518843 22960 ts_tablet_manager.cc:1434] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 13:10:59.519793 22960 tablet_bootstrap.cc:492] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.523836 22958 ts_tablet_manager.cc:1434] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:10:59.525270 22958 tablet_bootstrap.cc:492] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.526741 22986 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.527053 22960 tablet_bootstrap.cc:654] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:59.520062 22730 leader_election.cc:343] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.527269 22986 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.527809 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:59.528858 22994 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.531219 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.531857 22958 tablet_bootstrap.cc:654] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.531688 22986 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.532567 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:10:59.534250 22879 leader_election.cc:343] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.535451 22959 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
W20260709 13:10:59.535563 22880 leader_election.cc:343] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.535977 22959 ts_tablet_manager.cc:1403] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:10:59.536239 22880 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.537833 22986 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.538895 22959 raft_consensus.cc:359] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.539719 22959 raft_consensus.cc:385] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.539995 22959 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.540666 22959 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.542867 22959 ts_tablet_manager.cc:1434] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:59.544131 22959 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.547091 22965 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.547683 22965 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.554107 22959 tablet_bootstrap.cc:654] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.566638 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.567765 22765 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:59.569680 22986 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.570376 22986 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.571113 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.572172 22914 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:59.574481 22986 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.578161 22958 tablet_bootstrap.cc:492] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.578701 22958 ts_tablet_manager.cc:1403] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.054s	user 0.036s	sys 0.015s
I20260709 13:10:59.578965 22960 tablet_bootstrap.cc:492] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.580026 22960 ts_tablet_manager.cc:1403] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.060s	user 0.027s	sys 0.011s
I20260709 13:10:59.581768 22958 raft_consensus.cc:359] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.582480 22958 raft_consensus.cc:385] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.582757 22958 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.582659 22960 raft_consensus.cc:359] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.583352 22960 raft_consensus.cc:385] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.583652 22960 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.583647 22958 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.584278 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.584435 22960 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.585566 22991 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.585932 22991 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.586258 22991 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.587606 22960 ts_tablet_manager.cc:1434] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:10:59.588598 22960 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.593102 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.579268 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.594364 22991 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.594830 22839 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:10:59.595378 22960 tablet_bootstrap.cc:654] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.596709 22991 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.586221 22958 ts_tablet_manager.cc:1434] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:59.598376 22958 tablet_bootstrap.cc:492] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.597677 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:10:59.599251 22986 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.599761 22986 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.600162 22986 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.604066 22994 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.604610 22994 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.607815 22958 tablet_bootstrap.cc:654] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.609409 22994 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.612677 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:10:59.613585 22766 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.615764 22964 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.616238 22960 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.616214 22964 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.616808 22960 ts_tablet_manager.cc:1403] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 13:10:59.618484 22964 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.620672 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.620855 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.620131 22960 raft_consensus.cc:359] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:10:59.622443 22880 leader_election.cc:343] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:59.622807 22730 leader_election.cc:343] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.623296 22960 raft_consensus.cc:385] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.623617 22960 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.624540 22960 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.626344 22958 tablet_bootstrap.cc:492] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.626906 22960 ts_tablet_manager.cc:1434] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.005s
I20260709 13:10:59.627050 22958 ts_tablet_manager.cc:1403] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.005s
I20260709 13:10:59.627929 22960 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.630194 22958 raft_consensus.cc:359] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.630898 22958 raft_consensus.cc:385] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.631199 22958 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.632121 22958 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.633675 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.634639 22958 ts_tablet_manager.cc:1434] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:59.635380 22960 tablet_bootstrap.cc:654] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.635629 22958 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
W20260709 13:10:59.639016 22732 leader_election.cc:343] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.639146 22840 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.639560 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:59.640480 22988 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.641753 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.644162 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:59.644984 22914 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:59.650375 22880 leader_election.cc:343] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.651563 22958 tablet_bootstrap.cc:654] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.566875 22965 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:10:59.662966 22879 leader_election.cc:343] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.663697 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.665005 22964 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.668677 22958 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.669312 22958 ts_tablet_manager.cc:1403] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.007s
I20260709 13:10:59.673286 22958 raft_consensus.cc:359] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.674436 22958 raft_consensus.cc:385] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.674860 22958 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.676391 22958 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.679770 22958 ts_tablet_manager.cc:1434] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.006s
I20260709 13:10:59.680944 22958 tablet_bootstrap.cc:492] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.683748 22960 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.684600 22960 ts_tablet_manager.cc:1403] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.017s
I20260709 13:10:59.688195 22958 tablet_bootstrap.cc:654] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.688745 22960 raft_consensus.cc:359] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.690007 22960 raft_consensus.cc:385] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.690300 22960 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.691210 22960 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.694808 22960 ts_tablet_manager.cc:1434] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:10:59.700639 22958 tablet_bootstrap.cc:492] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.702800 22958 ts_tablet_manager.cc:1403] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:10:59.705906 22958 raft_consensus.cc:359] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.706656 22958 raft_consensus.cc:385] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.706928 22958 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.707696 22958 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.709586 22958 ts_tablet_manager.cc:1434] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:10:59.710534 22958 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.714344 22960 tablet_bootstrap.cc:492] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.717533 22958 tablet_bootstrap.cc:654] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.723158 22965 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.725740 22766 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:10:59.727192 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.728986 22965 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:59.729756 22965 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:10:59.732430 22958 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.732930 22958 ts_tablet_manager.cc:1403] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.004s
I20260709 13:10:59.731271 22965 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.735688 22958 raft_consensus.cc:359] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.736233 22958 raft_consensus.cc:385] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.736505 22958 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.729813 22914 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:10:59.737350 22958 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.702051 22986 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.739527 22958 ts_tablet_manager.cc:1434] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:10:59.740671 22958 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:10:59.740640 22959 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:10:59.741730 22959 ts_tablet_manager.cc:1403] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.198s	user 0.059s	sys 0.078s
I20260709 13:10:59.744797 22959 raft_consensus.cc:359] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.745568 22959 raft_consensus.cc:385] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.745860 22959 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.746933 22959 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.749061 22958 tablet_bootstrap.cc:654] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.740592 22986 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.749576 22959 ts_tablet_manager.cc:1434] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:59.750753 22959 tablet_bootstrap.cc:492] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:10:59.753111 22960 tablet_bootstrap.cc:654] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.758965 22959 tablet_bootstrap.cc:654] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.768834 22960 tablet_bootstrap.cc:492] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.770174 22960 ts_tablet_manager.cc:1403] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.001s
I20260709 13:10:59.761392 23015 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:10:59.776118 22988 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.776921 22988 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.781021 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.780933 22960 raft_consensus.cc:359] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.781819 22840 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:10:59.783303 22988 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.783370 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:10:59.784129 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.785071 22915 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:10:59.785634 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.784345 22839 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.786711 22988 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.787197 22988 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.787636 22988 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.781805 22960 raft_consensus.cc:385] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.788157 22960 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.790241 22960 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.792487 22986 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.793035 22986 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.795284 22986 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.796710 22988 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.797417 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.800115 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.799890 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:10:59.799295 22960 ts_tablet_manager.cc:1434] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:10:59.801136 22840 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.802712 22960 tablet_bootstrap.cc:492] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.801154 22841 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
W20260709 13:10:59.804139 22880 leader_election.cc:343] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.805471 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.806820 22986 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.807353 22986 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.807821 22986 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.809332 22840 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:10:59.811512 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:10:59.812758 22994 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:59.813366 22994 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:10:59.814574 22994 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:10:59.816267 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45613us
I20260709 13:10:59.817585 22960 tablet_bootstrap.cc:654] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:59.823138 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:10:59.824371 22967 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.841850 22967 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.841761 22664 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:59.835647 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:59.845886 22915 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.847128 22967 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.836378 22988 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.849030 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:10:59.849267 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.849910 22913 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
W20260709 13:10:59.850687 22730 leader_election.cc:343] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:59.851452 22880 leader_election.cc:343] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.856165 22732 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:10:59.858093 22964 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.858596 22988 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.859300 22988 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.860262 22988 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.864470 22964 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.875973 22960 tablet_bootstrap.cc:492] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:10:59.876511 22960 ts_tablet_manager.cc:1403] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.001s
I20260709 13:10:59.881584 22988 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:10:59.901225 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:59.902315 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:10:59.907040 22730 leader_election.cc:343] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.908171 22988 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.908401 23038 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.892021 22960 raft_consensus.cc:359] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.909286 23038 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.909693 22960 raft_consensus.cc:385] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:59.910746 22960 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:10:59.912182 22960 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.913412 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:10:59.915563 22880 leader_election.cc:343] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.916092 23015 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.916673 23015 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.917996 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:10:59.919934 22879 leader_election.cc:343] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:59.920719 23015 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.920580 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.920562 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.924111 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.925211 22764 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:10:59.926131 22964 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:59.927081 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:10:59.930140 23015 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.935331 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:10:59.936337 22914 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.927297 23038 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:10:59.947078 22960 ts_tablet_manager.cc:1434] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.005s
I20260709 13:10:59.951557 22991 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.952692 22960 tablet_bootstrap.cc:492] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:10:59.952332 22991 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:10:59.956671 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:10:59.957691 22991 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:10:59.957456 22913 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:59.960114 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:10:59.963599 22991 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:59.964161 22991 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.964635 22991 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:59.965917 23039 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:59.974010 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:10:59.974794 22766 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:10:59.840579 22839 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:00.009655 22960 tablet_bootstrap.cc:654] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.012930 22986 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.019812 22915 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:00.047017 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.090775 22958 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:10:59.949172 23015 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:00.091513 22958 ts_tablet_manager.cc:1403] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.351s	user 0.109s	sys 0.180s
I20260709 13:11:00.091389 23039 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:11:00.091482 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44710us
I20260709 13:11:00.092372 23039 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:00.093757 23039 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.095073 22958 raft_consensus.cc:359] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.095842 22958 raft_consensus.cc:385] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.096179 22958 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.057593 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:00.096958 22958 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.097864 23038 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:00.098906 22880 leader_election.cc:343] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.099655 22958 ts_tablet_manager.cc:1434] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:11:00.100761 22958 tablet_bootstrap.cc:492] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:00.108204 22959 tablet_bootstrap.cc:492] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:00.107931 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:00.108598 22986 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.108887 22840 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.110745 22958 tablet_bootstrap.cc:654] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.119566 22840 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:00.121482 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.123404 22959 ts_tablet_manager.cc:1403] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.373s	user 0.082s	sys 0.051s
I20260709 13:11:00.126533 22986 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.127583 22986 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:00.130501 22964 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.131213 22964 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.134034 22959 raft_consensus.cc:359] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.135484 22959 raft_consensus.cc:385] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.135878 22959 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.137006 22959 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.137923 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.138892 22764 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:00.140686 22964 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.142630 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:00.144116 22964 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:00.144536 22964 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.144841 22964 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.146374 22959 ts_tablet_manager.cc:1434] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 13:11:00.148458 22959 tablet_bootstrap.cc:492] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:00.152323 22914 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:00.154273 22732 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:00.155097 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:00.156538 22994 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.157214 22994 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:00.157429 22991 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.161876 22994 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.163306 22991 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.129014 22986 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.165328 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:00.166003 22913 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:00.168411 22879 leader_election.cc:343] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.169824 22964 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.171466 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:00.172134 22764 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.173966 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:00.174821 22766 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.176352 22964 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.179082 22958 tablet_bootstrap.cc:492] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:00.179769 22958 ts_tablet_manager.cc:1403] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.079s	user 0.034s	sys 0.010s
I20260709 13:11:00.181181 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:00.184602 22879 leader_election.cc:343] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.183961 22958 raft_consensus.cc:359] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.185842 22958 raft_consensus.cc:385] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.186139 22958 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.187932 22958 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.189096 22964 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.189628 22964 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.190646 22958 ts_tablet_manager.cc:1434] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 13:11:00.200865 22958 tablet_bootstrap.cc:492] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:00.210067 22960 tablet_bootstrap.cc:492] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:00.210758 22960 ts_tablet_manager.cc:1403] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.259s	user 0.067s	sys 0.114s
I20260709 13:11:00.211293 22662 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.216439 22964 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.218153 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:00.219681 22879 leader_election.cc:343] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.221069 22913 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:00.216240 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.236016 22959 tablet_bootstrap.cc:654] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.226958 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:00.231530 22958 tablet_bootstrap.cc:654] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.231659 22664 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.239842 22764 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:00.242056 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:00.243322 22880 leader_election.cc:343] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.243755 22880 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.243908 22965 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.244366 22986 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:00.244906 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:00.245645 22986 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.246093 22986 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:00.246930 22986 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.247445 22994 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.247931 22994 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.250033 22994 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.244416 22965 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:00.253856 22960 raft_consensus.cc:359] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.254485 22960 raft_consensus.cc:385] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.254719 22960 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.252061 22965 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.256042 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:00.256459 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:00.256680 22913 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:00.261861 22960 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.263113 22840 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:00.268143 22960 ts_tablet_manager.cc:1434] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:11:00.269084 22960 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:00.270638 22959 tablet_bootstrap.cc:492] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:00.271270 22959 ts_tablet_manager.cc:1403] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.123s	user 0.015s	sys 0.001s
I20260709 13:11:00.272920 22958 tablet_bootstrap.cc:492] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:00.273411 22958 ts_tablet_manager.cc:1403] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.073s	user 0.009s	sys 0.003s
I20260709 13:11:00.275005 22959 raft_consensus.cc:359] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.275771 22959 raft_consensus.cc:385] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.276059 22959 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.276275 22958 raft_consensus.cc:359] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.276630 22960 tablet_bootstrap.cc:654] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.277017 22958 raft_consensus.cc:385] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.276798 22959 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.277328 22958 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.314697 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:00.278213 22958 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.319876 22991 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.335080 22991 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.339358 23015 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.342939 22958 ts_tablet_manager.cc:1434] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.002s
I20260709 13:11:00.344059 22991 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.345743 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:00.346864 22913 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:00.349292 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:00.351879 22965 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:00.352391 22965 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.352830 22965 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.357805 22994 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:00.358599 22958 tablet_bootstrap.cc:492] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:00.358367 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.358372 22994 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.360980 22988 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.360922 22994 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.361683 22988 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.367309 22988 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:00.369639 22806 leader_election.cc:343] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.373355 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:00.374971 22841 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:00.377818 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:00.375059 22991 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:11:00.391999 22959 ts_tablet_manager.cc:1434] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.120s	user 0.009s	sys 0.000s
I20260709 13:11:00.399358 22661 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:00.404896 23070 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.406956 23070 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:00.406958 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:00.412856 22960 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:00.409451 23070 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.417836 22960 ts_tablet_manager.cc:1403] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.149s	user 0.020s	sys 0.035s
I20260709 13:11:00.417778 22661 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:00.430847 22960 raft_consensus.cc:359] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.432118 22960 raft_consensus.cc:385] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.432901 22960 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.434489 22960 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.437764 22959 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:00.440418 22960 ts_tablet_manager.cc:1434] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:11:00.434868 22965 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.445449 22965 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.446520 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:00.436496 22994 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.452947 22958 tablet_bootstrap.cc:654] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.453089 23070 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:00.453907 22806 leader_election.cc:343] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.454785 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:00.455518 22914 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.456246 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:00.456936 22839 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.460424 22960 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:00.458011 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:00.465308 22912 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.458496 22994 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.489976 22664 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.495829 22839 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:00.504817 22960 tablet_bootstrap.cc:654] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.506062 22959 tablet_bootstrap.cc:654] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.506764 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:00.507817 23070 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.508387 23070 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:00.509478 23070 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.512192 22914 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:00.515228 22912 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:00.517880 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.519997 22991 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.520601 22991 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:00.526494 22991 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.529484 22964 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.530220 22964 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.546975 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:00.546975 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:00.550354 22879 leader_election.cc:343] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:00.551501 22880 leader_election.cc:343] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.552155 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.553453 22986 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:00.554348 22964 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.564401 22960 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:00.565162 22960 ts_tablet_manager.cc:1403] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.010s
I20260709 13:11:00.571130 22960 raft_consensus.cc:359] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.572247 22960 raft_consensus.cc:385] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.572603 22960 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.574287 22960 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.579711 22662 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.560456 22664 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.588311 22960 ts_tablet_manager.cc:1434] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:11:00.589098 22964 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.590514 22960 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:00.596765 22959 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:00.599189 22958 tablet_bootstrap.cc:492] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:00.600194 22958 ts_tablet_manager.cc:1403] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.242s	user 0.023s	sys 0.026s
I20260709 13:11:00.601114 22959 ts_tablet_manager.cc:1403] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.176s	user 0.054s	sys 0.007s
I20260709 13:11:00.602481 22991 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.603070 22991 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.604231 22958 raft_consensus.cc:359] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.604331 22959 raft_consensus.cc:359] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.605047 22959 raft_consensus.cc:385] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.605444 22959 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.605013 22958 raft_consensus.cc:385] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.606973 22958 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.606508 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.606246 22959 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.609544 22960 tablet_bootstrap.cc:654] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.609726 22959 ts_tablet_manager.cc:1434] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:11:00.611092 22959 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:00.610955 22958 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.611384 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:00.612411 22912 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:00.614069 22807 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.615474 22965 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.619263 22959 tablet_bootstrap.cc:654] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:00.629149 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.630728 22986 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.632400 22663 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:00.642791 22986 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:00.644479 22879 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.649436 22986 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.649967 22986 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.654448 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:00.656798 22879 leader_election.cc:343] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.619618 22958 ts_tablet_manager.cc:1434] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.019s	user 0.001s	sys 0.005s
I20260709 13:11:00.659653 22958 tablet_bootstrap.cc:492] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:00.661717 22986 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.668733 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.675552 23015 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.676064 22965 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:00.679982 22967 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:00.692797 22880 leader_election.cc:343] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.693614 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.694976 22964 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:00.696172 22964 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.707083 22991 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.714041 22958 tablet_bootstrap.cc:654] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.720670 22959 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:00.721334 22959 ts_tablet_manager.cc:1403] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.111s	user 0.053s	sys 0.026s
I20260709 13:11:00.727309 22959 raft_consensus.cc:359] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.738862 22959 raft_consensus.cc:385] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.739202 22959 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.739905 22959 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.742194 22959 ts_tablet_manager.cc:1434] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:11:00.743399 22959 tablet_bootstrap.cc:492] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:00.743577 22967 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.747450 22958 tablet_bootstrap.cc:492] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:00.748011 22958 ts_tablet_manager.cc:1403] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.089s	user 0.007s	sys 0.014s
I20260709 13:11:00.751368 22958 raft_consensus.cc:359] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.752065 22958 raft_consensus.cc:385] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.752331 22958 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.753031 22958 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.755795 22958 ts_tablet_manager.cc:1434] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:00.756867 22958 tablet_bootstrap.cc:492] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:00.761731 22991 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.762338 22991 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.764621 22991 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.767247 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.768229 22764 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:00.769896 22806 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:00.770895 22991 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:00.771255 22991 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.771710 22991 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.773818 22958 tablet_bootstrap.cc:654] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.778709 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:00.779909 22915 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:00.781827 23015 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.789283 22959 tablet_bootstrap.cc:654] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:00.804651 23100 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:11:00.844069 22964 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.844998 22960 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:00.846231 22960 ts_tablet_manager.cc:1403] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.256s	user 0.070s	sys 0.093s
I20260709 13:11:00.844794 22964 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.856578 22964 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.858863 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:00.862149 22879 leader_election.cc:343] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.863528 22960 raft_consensus.cc:359] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.864899 22960 raft_consensus.cc:385] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.865332 22960 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.867825 22960 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.869103 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:00.878265 22991 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.881428 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:00.881834 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:00.882095 22765 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:00.882457 22912 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:00.886564 22880 leader_election.cc:343] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.887075 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.887848 22960 ts_tablet_manager.cc:1434] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 13:11:00.888293 23039 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:00.888994 22960 tablet_bootstrap.cc:492] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:00.883723 22991 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:00.901923 22958 tablet_bootstrap.cc:492] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:00.902457 22958 ts_tablet_manager.cc:1403] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.146s	user 0.028s	sys 0.009s
I20260709 13:11:00.903932 22912 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:00.905270 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:00.907500 23030 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:00.908074 23030 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:00.908973 23030 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.917126 22958 raft_consensus.cc:359] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.920847 22958 raft_consensus.cc:385] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.921132 22958 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.921701 22967 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:00.926923 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.934849 22958 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.941475 22958 ts_tablet_manager.cc:1434] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:11:00.942700 22958 tablet_bootstrap.cc:492] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:00.946504 22959 tablet_bootstrap.cc:492] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:00.947059 22959 ts_tablet_manager.cc:1403] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.204s	user 0.014s	sys 0.005s
I20260709 13:11:00.929277 22765 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:00.954962 22960 tablet_bootstrap.cc:654] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.960661 23015 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:00.964905 22958 tablet_bootstrap.cc:654] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:00.970584 22959 raft_consensus.cc:359] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.971443 22959 raft_consensus.cc:385] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:00.971788 22959 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:00.974563 22959 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.975756 23039 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.976336 23039 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:00.976030 22663 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.978770 23039 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:00.981443 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:00.983389 22879 leader_election.cc:343] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.984598 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:00.986789 22880 leader_election.cc:343] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:00.987769 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:00.990082 23039 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.004015 22967 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:01.009768 22959 ts_tablet_manager.cc:1434] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.062s	user 0.011s	sys 0.000s
I20260709 13:11:01.011492 22959 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.039381 22958 tablet_bootstrap.cc:492] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.039963 22958 ts_tablet_manager.cc:1403] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.097s	user 0.037s	sys 0.022s
I20260709 13:11:01.053241 22960 tablet_bootstrap.cc:492] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.053872 22960 ts_tablet_manager.cc:1403] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.165s	user 0.018s	sys 0.014s
I20260709 13:11:01.054395 22959 tablet_bootstrap.cc:654] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.056924 22958 raft_consensus.cc:359] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.057709 22958 raft_consensus.cc:385] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.057957 22958 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.058602 22958 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.061112 22958 ts_tablet_manager.cc:1434] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 13:11:01.063719 22958 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.070712 22958 tablet_bootstrap.cc:654] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.081092 22960 raft_consensus.cc:359] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.082559 22960 raft_consensus.cc:385] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.083292 22960 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.084805 22960 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.087909 22960 ts_tablet_manager.cc:1434] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:11:01.089564 22960 tablet_bootstrap.cc:492] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.090826 22967 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.091511 22967 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.096402 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:01.101552 22730 leader_election.cc:343] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.101676 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.103539 22915 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:01.105469 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:01.106739 22988 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.107245 22988 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.107689 22988 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.102761 22967 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.120334 22988 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.121331 22959 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
W20260709 13:11:01.122398 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:01.127038 22959 ts_tablet_manager.cc:1403] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.116s	user 0.021s	sys 0.053s
I20260709 13:11:01.128432 22988 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.125957 22958 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.129195 22958 ts_tablet_manager.cc:1403] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.067s	user 0.034s	sys 0.018s
I20260709 13:11:01.129547 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:01.129881 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:01.130074 22959 raft_consensus.cc:359] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.126480 22960 tablet_bootstrap.cc:654] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.130568 22915 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:01.131434 22730 leader_election.cc:343] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.132537 22958 raft_consensus.cc:359] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.133236 22958 raft_consensus.cc:385] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.130865 22959 raft_consensus.cc:385] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.133538 22958 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.133857 22959 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.134311 22958 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.134673 22959 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.136464 22958 ts_tablet_manager.cc:1434] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:01.137018 22959 ts_tablet_manager.cc:1434] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:11:01.138016 22959 tablet_bootstrap.cc:492] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.138016 22958 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.142861 22960 tablet_bootstrap.cc:492] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.143630 22960 ts_tablet_manager.cc:1403] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.000s
I20260709 13:11:01.143956 22988 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.144577 22958 tablet_bootstrap.cc:654] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.144425 22988 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.145773 22915 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:01.146878 22988 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.147112 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:01.147296 22960 raft_consensus.cc:359] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.148155 22960 raft_consensus.cc:385] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.149736 22967 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:01.150445 22967 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:01.151609 22967 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.148929 22960 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.153666 22960 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.149684 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.155879 22912 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:01.149708 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.157640 22839 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:01.159400 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.160605 23070 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.161136 23070 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.161602 23070 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.166024 23015 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.166591 23015 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.167932 22960 ts_tablet_manager.cc:1434] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:11:01.169449 23015 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.171684 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.174160 22839 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.176077 22960 tablet_bootstrap.cc:492] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.180365 22959 tablet_bootstrap.cc:654] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.182622 22764 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.184051 22960 tablet_bootstrap.cc:654] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.186496 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.189109 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.191037 23039 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.191707 23039 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.195847 23039 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.197988 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:01.200297 22880 leader_election.cc:343] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.201887 23070 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.206148 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:01.206981 22841 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.211400 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:01.214273 22879 leader_election.cc:343] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.215107 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.217109 23039 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.222937 22841 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:01.224838 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.232148 22994 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:01.232738 22994 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:01.233898 22994 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.259291 22959 tablet_bootstrap.cc:492] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:01.263427 22959 ts_tablet_manager.cc:1403] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.126s	user 0.016s	sys 0.000s
I20260709 13:11:01.261627 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:01.266178 22915 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.278380 22960 tablet_bootstrap.cc:492] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.279017 22960 ts_tablet_manager.cc:1403] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.103s	user 0.031s	sys 0.018s
I20260709 13:11:01.278721 22959 raft_consensus.cc:359] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.279461 22959 raft_consensus.cc:385] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.260062 23070 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.279750 22959 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.283133 22958 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.283646 22958 ts_tablet_manager.cc:1403] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.146s	user 0.043s	sys 0.081s
I20260709 13:11:01.285177 22960 raft_consensus.cc:359] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.287133 22960 raft_consensus.cc:385] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.286710 22958 raft_consensus.cc:359] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.287658 22958 raft_consensus.cc:385] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.282371 22959 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.287947 22958 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.287483 22960 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.288805 22958 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.289237 22960 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.290835 22958 ts_tablet_manager.cc:1434] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:01.291961 22958 tablet_bootstrap.cc:492] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.292546 22959 ts_tablet_manager.cc:1434] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.003s
I20260709 13:11:01.293771 22959 tablet_bootstrap.cc:492] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.294250 23015 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.295766 23107 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:01.299959 23121 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:01.300717 23121 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.314494 22960 ts_tablet_manager.cc:1434] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:11:01.315774 22960 tablet_bootstrap.cc:492] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.301766 22915 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:01.320459 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.324203 23121 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.343504 22662 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.329164 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.346632 22662 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.350063 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:01.350113 22958 tablet_bootstrap.cc:654] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.350986 23070 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:01.358829 22959 tablet_bootstrap.cc:654] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.369234 22960 tablet_bootstrap.cc:654] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.371675 23107 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.383737 23107 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.395591 23107 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.397426 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.398188 22764 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.399848 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.400964 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.401494 23107 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.401671 22915 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.401986 23107 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.402448 23107 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.414251 23015 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:01.415753 22958 tablet_bootstrap.cc:492] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.416231 22958 ts_tablet_manager.cc:1403] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.125s	user 0.021s	sys 0.026s
I20260709 13:11:01.417145 22959 tablet_bootstrap.cc:492] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:01.417776 22959 ts_tablet_manager.cc:1403] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.124s	user 0.023s	sys 0.010s
I20260709 13:11:01.419325 22958 raft_consensus.cc:359] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.420006 22958 raft_consensus.cc:385] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.420260 22958 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.420466 22959 raft_consensus.cc:359] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.421648 22958 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.424841 22960 tablet_bootstrap.cc:492] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.425422 22960 ts_tablet_manager.cc:1403] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.000s
I20260709 13:11:01.424849 22958 ts_tablet_manager.cc:1434] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:01.429180 22958 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.432101 22960 raft_consensus.cc:359] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.432893 22960 raft_consensus.cc:385] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.433140 22960 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.433872 22960 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.443567 22958 tablet_bootstrap.cc:654] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.421134 22959 raft_consensus.cc:385] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.445672 22959 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.446575 22959 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.436067 22960 ts_tablet_manager.cc:1434] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:11:01.451881 22960 tablet_bootstrap.cc:492] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.461961 22959 ts_tablet_manager.cc:1434] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.001s
I20260709 13:11:01.454490 23107 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.465802 23107 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.467733 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:01.468418 22912 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.471817 22959 tablet_bootstrap.cc:492] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.473107 23015 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.473726 23015 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.474757 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:01.475391 22764 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.477257 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.481897 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.482741 22915 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:01.484021 22960 tablet_bootstrap.cc:654] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.484743 23015 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.484093 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.487624 22991 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.497128 22912 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:01.498425 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:01.499365 22959 tablet_bootstrap.cc:654] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.497179 22991 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.501349 22764 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:01.499373 23015 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:01.502769 23015 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:01.504345 22958 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.504887 22958 ts_tablet_manager.cc:1403] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.029s
I20260709 13:11:01.503788 23015 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.507829 22960 tablet_bootstrap.cc:492] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.508354 22960 ts_tablet_manager.cc:1403] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.057s	user 0.004s	sys 0.008s
I20260709 13:11:01.508119 22958 raft_consensus.cc:359] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.508865 22958 raft_consensus.cc:385] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.509207 22958 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.510008 22958 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.511525 22960 raft_consensus.cc:359] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.512279 22960 raft_consensus.cc:385] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.512435 22958 ts_tablet_manager.cc:1434] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:01.512585 22960 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.513756 23039 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.514197 22958 tablet_bootstrap.cc:492] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.514258 23039 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.514915 22959 tablet_bootstrap.cc:492] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:01.515407 22959 ts_tablet_manager.cc:1403] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.001s
I20260709 13:11:01.517186 23039 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.517779 22960 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.518723 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.519239 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.520174 22960 ts_tablet_manager.cc:1434] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
W20260709 13:11:01.520752 22880 leader_election.cc:343] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.521371 22960 tablet_bootstrap.cc:492] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
W20260709 13:11:01.522838 22879 leader_election.cc:343] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.523334 22879 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.523850 22958 tablet_bootstrap.cc:654] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.527727 22959 raft_consensus.cc:359] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.528937 22959 raft_consensus.cc:385] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.529242 22959 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.532766 23039 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.534019 22663 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.529947 22959 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.552175 22959 ts_tablet_manager.cc:1434] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:11:01.553419 22959 tablet_bootstrap.cc:492] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.555537 22960 tablet_bootstrap.cc:654] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.563798 22960 tablet_bootstrap.cc:492] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.564420 22960 ts_tablet_manager.cc:1403] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.001s
I20260709 13:11:01.564965 22959 tablet_bootstrap.cc:654] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.567735 22960 raft_consensus.cc:359] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.568341 22960 raft_consensus.cc:385] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.568579 22960 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.570669 22960 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.572489 22959 tablet_bootstrap.cc:492] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:01.573179 22959 ts_tablet_manager.cc:1403] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 13:11:01.573558 22960 ts_tablet_manager.cc:1434] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:11:01.574939 22960 tablet_bootstrap.cc:492] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.576822 22959 raft_consensus.cc:359] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.577627 22959 raft_consensus.cc:385] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.577931 22959 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.578614 22959 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.581337 22959 ts_tablet_manager.cc:1434] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.003s
I20260709 13:11:01.582840 22959 tablet_bootstrap.cc:492] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.585904 23039 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.586576 23039 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.589404 23039 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.590375 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:01.592398 22880 leader_election.cc:343] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.593488 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:01.595160 22879 leader_election.cc:343] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.595626 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.596077 22959 tablet_bootstrap.cc:654] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.596941 23039 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.598059 22958 tablet_bootstrap.cc:492] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.598436 22958 ts_tablet_manager.cc:1403] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.085s	user 0.019s	sys 0.003s
I20260709 13:11:01.601199 22958 raft_consensus.cc:359] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.601886 22960 tablet_bootstrap.cc:654] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.601857 22958 raft_consensus.cc:385] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.602357 22958 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.603111 22958 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.606592 23039 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.606817 22959 tablet_bootstrap.cc:492] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:01.607311 22959 ts_tablet_manager.cc:1403] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 13:11:01.607069 23039 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.609859 22958 ts_tablet_manager.cc:1434] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:11:01.610206 22959 raft_consensus.cc:359] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.610781 22958 tablet_bootstrap.cc:492] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.610895 22959 raft_consensus.cc:385] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.611186 22959 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.611959 22959 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.616279 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.617028 22959 ts_tablet_manager.cc:1434] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 13:11:01.617844 22879 leader_election.cc:343] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.618775 23039 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.619346 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.621116 22964 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.621623 22964 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:01.621944 22880 leader_election.cc:343] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.622445 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.624119 23135 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.625203 22958 tablet_bootstrap.cc:654] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.626550 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.630613 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:01.633278 22880 leader_election.cc:343] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:01.634074 22879 leader_election.cc:343] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.634541 22879 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.635320 23039 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.623993 22964 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.638823 22958 tablet_bootstrap.cc:492] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.639336 22958 ts_tablet_manager.cc:1403] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.002s
I20260709 13:11:01.639470 22960 tablet_bootstrap.cc:492] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.639953 22960 ts_tablet_manager.cc:1403] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.018s
I20260709 13:11:01.640669 22959 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.642472 22958 raft_consensus.cc:359] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.643249 22958 raft_consensus.cc:385] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.643509 22958 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.643815 22960 raft_consensus.cc:359] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.644780 22960 raft_consensus.cc:385] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.644186 22958 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.645043 22960 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.645959 22960 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.648089 22959 tablet_bootstrap.cc:654] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.650436 22958 ts_tablet_manager.cc:1434] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:11:01.651437 22958 tablet_bootstrap.cc:492] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.659256 23070 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:11:01.658566 22958 tablet_bootstrap.cc:654] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.678182 22960 ts_tablet_manager.cc:1434] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.003s
I20260709 13:11:01.679823 22960 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.684718 23015 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.685374 23015 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.687608 23015 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.687747 22958 tablet_bootstrap.cc:492] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.688267 22958 ts_tablet_manager.cc:1403] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.009s
I20260709 13:11:01.689252 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.690013 22764 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.691670 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.691864 22958 raft_consensus.cc:359] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.692325 22915 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.692523 22958 raft_consensus.cc:385] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.693975 22964 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.694432 22964 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.695668 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.696467 22964 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.697355 23015 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.697762 23015 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.698109 23015 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.699988 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.700701 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:01.700860 23070 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:01.702064 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.703019 22964 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.703357 22964 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.703693 22964 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.704221 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.705183 22839 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:01.710589 22958 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.711475 22958 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.712755 22964 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.715781 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:01.716346 22764 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.717105 23070 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:01.719045 22991 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.719554 22991 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.721724 22991 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.723627 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.723541 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.724104 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:01.724489 22915 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.724714 22765 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.725405 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.725828 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:01.726348 23135 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:01.726666 22991 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.726846 23135 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:01.727039 22991 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.727398 22991 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.727820 23135 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.755075 22958 ts_tablet_manager.cc:1434] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.066s	user 0.002s	sys 0.005s
I20260709 13:11:01.756561 22964 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.760233 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:01.760898 22839 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.763540 22960 tablet_bootstrap.cc:654] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.752130 22991 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.744720 23015 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.764907 22958 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.779583 22839 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:01.785992 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:01.786672 22764 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.789083 22959 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:01.789978 22959 ts_tablet_manager.cc:1403] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.150s	user 0.059s	sys 0.045s
I20260709 13:11:01.791512 22991 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.795727 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:01.796541 22914 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.796731 23015 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.790944 23070 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:11:01.803552 22958 tablet_bootstrap.cc:654] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.805266 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:01.805962 22913 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.808748 22764 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:01.809315 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:01.809937 22765 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.812132 23015 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.813635 23015 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.812786 22959 raft_consensus.cc:359] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.815696 22959 raft_consensus.cc:385] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.816061 22959 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.816856 22959 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.817986 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.821897 23107 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:01.822424 23107 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:01.823410 23107 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.832520 22662 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:01.822391 23015 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.837332 22959 ts_tablet_manager.cc:1434] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 13:11:01.836416 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.837417 22913 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:01.843815 23121 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.844360 22959 tablet_bootstrap.cc:492] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:01.844386 23121 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.852560 23121 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.845845 22915 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.860873 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:01.864981 22806 leader_election.cc:343] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:01.866494 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:01.868162 23015 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:01.868868 23015 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:01.869848 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:01.869993 23015 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.870795 22991 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.871194 22991 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.871309 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.871552 22991 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.874553 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.878441 22914 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:01.885434 22960 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:01.886471 22960 ts_tablet_manager.cc:1403] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.207s	user 0.015s	sys 0.022s
I20260709 13:11:01.877373 22765 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:01.888577 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:01.890044 23121 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:01.896344 22958 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:01.899362 22958 ts_tablet_manager.cc:1403] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.135s	user 0.012s	sys 0.003s
I20260709 13:11:01.902256 22958 raft_consensus.cc:359] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.905802 22958 raft_consensus.cc:385] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.906157 22958 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.898850 22959 tablet_bootstrap.cc:654] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:01.905074 22960 raft_consensus.cc:359] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.907356 22960 raft_consensus.cc:385] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:01.907687 22960 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:01.897034 22991 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.908577 22960 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.912950 22958 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.945024 23153 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.945904 22960 ts_tablet_manager.cc:1434] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.059s	user 0.022s	sys 0.014s
I20260709 13:11:01.947669 22958 ts_tablet_manager.cc:1434] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 13:11:01.949942 22964 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.948863 23153 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.953006 23153 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.953925 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.954288 22958 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:01.954586 22764 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:01.960384 22662 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.962507 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:01.956540 22964 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.966383 22964 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:01.967248 22965 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:01.967654 22965 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.967968 22965 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.969462 22960 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:01.976672 23135 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.984015 23135 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.973658 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:01.969129 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:01.970199 22991 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:01.968730 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:01.989770 22965 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:01.987108 22913 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:01.986531 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:01.992684 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:01.999181 22763 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.007113 22959 tablet_bootstrap.cc:492] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:02.007656 22959 ts_tablet_manager.cc:1403] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.010s
I20260709 13:11:02.003816 22915 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.017797 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.018016 22965 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.019799 23135 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:02.009039 22806 leader_election.cc:343] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.022943 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:02.023576 22766 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.025944 22958 tablet_bootstrap.cc:654] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:02.029103 22880 leader_election.cc:343] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:02.029702 22879 leader_election.cc:343] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.030341 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.034933 22663 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.039057 22959 raft_consensus.cc:359] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.039570 22959 raft_consensus.cc:385] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.039736 22959 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.040537 22959 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.038844 23135 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:02.054767 22880 leader_election.cc:343] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.062276 22959 ts_tablet_manager.cc:1434] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.002s
I20260709 13:11:02.062160 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.067876 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:02.075091 22914 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.076292 22959 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:02.068799 22913 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
W20260709 13:11:02.081104 22879 leader_election.cc:343] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.081732 22879 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.082549 22964 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.086987 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.087944 23070 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.088410 23070 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.089185 22958 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.090480 23070 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.092852 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:02.093922 22958 ts_tablet_manager.cc:1403] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.019s
I20260709 13:11:02.095801 22964 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.096580 23154 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.096279 22964 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.096621 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.097764 23154 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:02.098664 23154 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.100453 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:02.103300 22880 leader_election.cc:343] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.100626 22964 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.105398 22960 tablet_bootstrap.cc:654] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.106295 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:02.107932 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.108783 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.109094 22913 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:02.109730 22991 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.110297 22991 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:02.111310 22991 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.114334 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:02.112918 22879 leader_election.cc:343] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.116801 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.118549 23156 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:02.126754 22964 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.133131 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:02.135207 23070 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.136739 22959 tablet_bootstrap.cc:654] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.137727 23156 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.141716 22958 raft_consensus.cc:359] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.142632 22958 raft_consensus.cc:385] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.142959 22958 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.143677 22958 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.151433 23070 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.154583 22958 ts_tablet_manager.cc:1434] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.003s
I20260709 13:11:02.155781 22958 tablet_bootstrap.cc:492] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.174198 22958 tablet_bootstrap.cc:654] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.180846 22960 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.181607 22960 ts_tablet_manager.cc:1403] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.225s	user 0.031s	sys 0.015s
I20260709 13:11:02.184132 23039 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.198721 22663 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.205130 22960 raft_consensus.cc:359] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.205900 22960 raft_consensus.cc:385] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.206163 22960 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.209164 22960 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.211787 22960 ts_tablet_manager.cc:1434] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:11:02.212818 22960 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.220930 23156 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.221630 23156 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.226722 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:02.227571 22913 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.228614 23156 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.232966 22807 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:02.235872 22958 tablet_bootstrap.cc:492] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.236392 22958 ts_tablet_manager.cc:1403] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.003s
I20260709 13:11:02.237290 23156 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:02.237448 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.237838 23156 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.238327 23156 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:02.239012 22806 leader_election.cc:343] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.251061 23156 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.252635 22960 tablet_bootstrap.cc:654] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.249399 22663 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.254452 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:02.254792 23156 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:02.255961 22806 leader_election.cc:343] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.256289 22959 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:02.256771 22959 ts_tablet_manager.cc:1403] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.186s	user 0.008s	sys 0.030s
I20260709 13:11:02.258229 22964 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.255797 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:02.264737 22914 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.259539 22958 raft_consensus.cc:359] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.268205 22958 raft_consensus.cc:385] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.268581 22958 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.269425 22958 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.273608 22959 raft_consensus.cc:359] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.274678 22959 raft_consensus.cc:385] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.274979 22959 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.275375 22958 ts_tablet_manager.cc:1434] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:11:02.276530 22958 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.275797 22959 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.280493 22959 ts_tablet_manager.cc:1434] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 13:11:02.281423 22914 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.290237 22807 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.291644 23156 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.292178 23156 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:02.292773 22959 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:02.293079 23156 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:02.296243 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:02.301994 22958 tablet_bootstrap.cc:654] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.309993 22959 tablet_bootstrap.cc:654] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.320600 22960 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.321467 22960 ts_tablet_manager.cc:1403] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.109s	user 0.028s	sys 0.016s
I20260709 13:11:02.322183 22664 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.325551 22958 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.326056 22958 ts_tablet_manager.cc:1403] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.002s
I20260709 13:11:02.326081 22960 raft_consensus.cc:359] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.327010 22960 raft_consensus.cc:385] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.327620 22960 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.328586 22960 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.329373 22958 raft_consensus.cc:359] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.330106 22958 raft_consensus.cc:385] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.330406 22958 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.331862 22960 ts_tablet_manager.cc:1434] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:11:02.333582 22960 tablet_bootstrap.cc:492] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.331463 22958 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.339675 22958 ts_tablet_manager.cc:1434] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:11:02.340824 22958 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.340951 22960 tablet_bootstrap.cc:654] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.346760 22959 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:02.346975 22960 tablet_bootstrap.cc:492] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.347218 22959 ts_tablet_manager.cc:1403] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.005s
I20260709 13:11:02.347609 22960 ts_tablet_manager.cc:1403] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:11:02.350083 22959 raft_consensus.cc:359] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.351018 22959 raft_consensus.cc:385] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.351305 22959 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.352330 22959 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.353924 22959 ts_tablet_manager.cc:1434] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:02.354935 22959 tablet_bootstrap.cc:492] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:02.350368 22960 raft_consensus.cc:359] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.361657 22960 raft_consensus.cc:385] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.362015 22960 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.362771 22960 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.364571 22960 ts_tablet_manager.cc:1434] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:11:02.366118 22960 tablet_bootstrap.cc:492] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.366938 22958 tablet_bootstrap.cc:654] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.368428 22959 tablet_bootstrap.cc:654] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.374652 22960 tablet_bootstrap.cc:654] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.382302 22958 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.382880 22958 ts_tablet_manager.cc:1403] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.008s
I20260709 13:11:02.391234 22958 raft_consensus.cc:359] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.392310 22958 raft_consensus.cc:385] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.392640 22958 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.393330 22958 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.395725 22958 ts_tablet_manager.cc:1434] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:11:02.397081 22958 tablet_bootstrap.cc:492] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.399181 22991 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.404327 22958 tablet_bootstrap.cc:654] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.411739 22958 tablet_bootstrap.cc:492] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.412245 22958 ts_tablet_manager.cc:1403] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:11:02.414662 22958 raft_consensus.cc:359] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.415221 22958 raft_consensus.cc:385] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.415526 22958 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.416349 22958 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.419003 22958 ts_tablet_manager.cc:1434] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:11:02.420969 22959 tablet_bootstrap.cc:492] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:02.424749 23015 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.421422 22959 ts_tablet_manager.cc:1403] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.067s	user 0.007s	sys 0.005s
I20260709 13:11:02.419041 22960 tablet_bootstrap.cc:492] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.431510 22960 ts_tablet_manager.cc:1403] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.010s
I20260709 13:11:02.428269 22959 raft_consensus.cc:359] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.433846 23107 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.435853 22958 tablet_bootstrap.cc:492] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.434947 22959 raft_consensus.cc:385] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.440485 22959 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.441620 22959 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.442597 23107 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.444425 22960 raft_consensus.cc:359] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.444809 22959 ts_tablet_manager.cc:1434] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:11:02.445116 22960 raft_consensus.cc:385] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.445559 22960 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.447041 22959 tablet_bootstrap.cc:492] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:02.443104 23107 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.449455 23107 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.447086 22960 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.452945 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:02.453378 22960 ts_tablet_manager.cc:1434] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:11:02.453714 22912 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.455009 22958 tablet_bootstrap.cc:654] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.455399 22807 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:02.455832 22959 tablet_bootstrap.cc:654] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.456681 22991 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:02.457125 22991 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.457541 22991 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.458720 22960 tablet_bootstrap.cc:492] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.462000 22964 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.462517 22964 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.464637 22964 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.465658 23156 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.466157 23156 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.467460 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.468266 23156 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.466185 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:02.468847 22879 leader_election.cc:343] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:02.471460 22880 leader_election.cc:343] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.471881 22880 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.472991 22964 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.473241 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:02.475138 22806 leader_election.cc:343] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.475836 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:02.476353 23107 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.480367 22960 tablet_bootstrap.cc:654] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.481371 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.482292 22991 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.476759 22913 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.491832 22958 tablet_bootstrap.cc:492] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.492480 22958 ts_tablet_manager.cc:1403] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.009s
I20260709 13:11:02.495473 22958 raft_consensus.cc:359] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.496025 22959 tablet_bootstrap.cc:492] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:02.496223 22958 raft_consensus.cc:385] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.496559 22958 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.496733 22959 ts_tablet_manager.cc:1403] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.006s
I20260709 13:11:02.497255 22958 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.492266 22763 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.498482 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.499336 23107 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:02.499711 23107 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.500041 23107 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.503585 22991 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.508210 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:02.508808 22958 ts_tablet_manager.cc:1434] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:11:02.508917 22766 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.510320 22958 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.511628 23107 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.517000 23107 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.519785 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:02.520517 22913 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.521575 22960 tablet_bootstrap.cc:492] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.522090 22960 ts_tablet_manager.cc:1403] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.064s	user 0.007s	sys 0.009s
I20260709 13:11:02.542627 22991 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:02.549717 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:02.550028 22991 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.550424 22914 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.550556 22991 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.554605 22960 raft_consensus.cc:359] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.556579 22960 raft_consensus.cc:385] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.556942 22960 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.557663 22960 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.555178 22959 raft_consensus.cc:359] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.559477 22959 raft_consensus.cc:385] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.559739 22959 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.556128 22991 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.570161 22960 ts_tablet_manager.cc:1434] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.003s
I20260709 13:11:02.571468 22960 tablet_bootstrap.cc:492] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.572157 22766 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.572424 22913 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.521595 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:02.573320 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:02.573735 22807 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:02.574040 22915 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.574543 22914 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.575228 23154 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.575717 23154 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:02.576406 22959 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.576562 23154 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.578675 22959 ts_tablet_manager.cc:1434] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.082s	user 0.005s	sys 0.010s
I20260709 13:11:02.579739 22959 tablet_bootstrap.cc:492] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:02.580415 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:02.581466 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:02.582238 23015 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.582888 23015 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:02.583767 23015 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:02.586994 22806 leader_election.cc:343] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.582135 23156 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:02.595600 22959 tablet_bootstrap.cc:654] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.588698 23156 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.598162 23156 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.600350 23135 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.600972 23135 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.601711 23039 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.602196 23039 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.604199 23135 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.604691 22958 tablet_bootstrap.cc:654] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.603132 22988 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.606015 22988 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.607184 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.582890 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.609632 22988 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.607506 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:02.611792 22879 leader_election.cc:343] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.611883 22960 tablet_bootstrap.cc:654] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.614105 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.612689 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:02.613204 22880 leader_election.cc:343] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.616930 22914 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:02.616995 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:02.620386 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:02.620754 22964 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.610584 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.606034 23039 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.622721 22964 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:02.625487 22880 leader_election.cc:343] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.625977 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.626456 22964 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.627135 22964 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:02.631580 22879 leader_election.cc:343] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.633250 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.646425 23039 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.635480 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.653746 22839 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:02.654062 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.655038 22959 tablet_bootstrap.cc:492] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
W20260709 13:11:02.655615 22730 leader_election.cc:343] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.656126 23154 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.656862 22959 ts_tablet_manager.cc:1403] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.077s	user 0.016s	sys 0.032s
I20260709 13:11:02.649003 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.658378 23070 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.657774 22662 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.624383 22988 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:02.658403 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:02.660328 23070 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.661124 23135 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:02.661628 23135 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.662006 23135 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.659714 22662 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.667024 23070 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.660336 22988 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.669078 22988 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.685947 22958 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.691068 22958 ts_tablet_manager.cc:1403] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.181s	user 0.026s	sys 0.025s
I20260709 13:11:02.694455 22991 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:02.694432 23156 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.703343 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.711686 22991 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.709013 23156 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.701174 22959 raft_consensus.cc:359] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.715013 22959 raft_consensus.cc:385] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.715454 22959 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.728736 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:02.729738 22764 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.733963 22959 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.740105 22765 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:02.751559 22988 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.767467 22988 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:02.771498 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:02.720875 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56763.1us
I20260709 13:11:02.775384 22958 raft_consensus.cc:359] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.776283 22958 raft_consensus.cc:385] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.777004 22958 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.780215 22958 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.781584 22959 ts_tablet_manager.cc:1434] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.124s	user 0.004s	sys 0.005s
I20260709 13:11:02.783357 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:02.784272 22911 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:02.787842 22964 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.788539 22964 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.791939 22964 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.793053 22958 ts_tablet_manager.cc:1434] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.102s	user 0.004s	sys 0.004s
I20260709 13:11:02.776674 22914 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.794772 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:02.796876 22880 leader_election.cc:343] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.797101 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.793097 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:02.799307 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:02.800362 23107 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.800834 23107 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:02.801582 22959 tablet_bootstrap.cc:492] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:02.801824 22960 tablet_bootstrap.cc:492] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.794342 22958 tablet_bootstrap.cc:492] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:02.805826 22960 ts_tablet_manager.cc:1403] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.235s	user 0.064s	sys 0.053s
I20260709 13:11:02.804291 22914 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:02.809628 22732 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:02.811432 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:02.812888 22910 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.813268 22960 raft_consensus.cc:359] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.814039 22960 raft_consensus.cc:385] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.814286 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.801659 23107 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.814340 22960 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.819306 22960 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.815492 22967 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.828954 22960 ts_tablet_manager.cc:1434] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:11:02.830260 22960 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.831805 22967 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:02.837847 22879 leader_election.cc:343] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.840642 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.812316 23135 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.843207 22964 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.849479 22959 tablet_bootstrap.cc:654] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.850427 22910 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:02.850797 22730 leader_election.cc:343] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.861101 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:02.861779 22763 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.863345 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:02.865837 22988 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.866300 22988 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:02.866348 23107 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:02.867223 22988 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.870093 23135 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.869925 22662 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.871783 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:02.872391 22841 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:02.888613 22763 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:02.919684 22960 tablet_bootstrap.cc:654] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.927332 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:02.931766 22964 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:02.932370 22964 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:02.933215 22964 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.939682 22958 tablet_bootstrap.cc:654] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.927733 23154 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:11:02.947750 22662 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:02.963757 22959 tablet_bootstrap.cc:492] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:02.964471 22959 ts_tablet_manager.cc:1403] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.166s	user 0.018s	sys 0.009s
I20260709 13:11:02.963665 22661 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:02.966069 22960 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:02.966573 22960 ts_tablet_manager.cc:1403] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.137s	user 0.012s	sys 0.003s
I20260709 13:11:02.967391 22959 raft_consensus.cc:359] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.968101 22959 raft_consensus.cc:385] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.968372 22959 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.972131 22960 raft_consensus.cc:359] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.972911 22960 raft_consensus.cc:385] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.973219 22960 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.974087 22960 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.976861 22960 ts_tablet_manager.cc:1434] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:11:02.978051 22960 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:02.978331 23191 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.978331 22841 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:02.978940 23191 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.979815 22959 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.982061 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:02.982419 22959 ts_tablet_manager.cc:1434] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:11:02.983443 22959 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
W20260709 13:11:02.983937 22880 leader_election.cc:343] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.985903 22960 tablet_bootstrap.cc:654] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.988466 23191 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:02.990523 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:02.990911 22958 tablet_bootstrap.cc:492] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:02.991398 22958 ts_tablet_manager.cc:1403] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.197s	user 0.006s	sys 0.008s
W20260709 13:11:02.992251 22879 leader_election.cc:343] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:02.992690 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:02.993829 22964 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:02.994302 22958 raft_consensus.cc:359] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:02.995008 22958 raft_consensus.cc:385] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:02.995291 22958 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:02.995534 22959 tablet_bootstrap.cc:654] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:02.996014 22958 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.019667 22960 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.020398 22960 ts_tablet_manager.cc:1403] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.009s
I20260709 13:11:03.021601 23156 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:03.023875 22960 raft_consensus.cc:359] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.024796 22960 raft_consensus.cc:385] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.025215 22960 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.026122 22960 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.031188 23156 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:03.033957 22960 ts_tablet_manager.cc:1434] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:11:03.042807 22958 ts_tablet_manager.cc:1434] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:11:03.045545 22958 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.045742 22960 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.052151 23156 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.053977 23156 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.063001 23156 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.063797 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.068030 22960 tablet_bootstrap.cc:654] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.069284 22959 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.070245 22958 tablet_bootstrap.cc:654] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.073371 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.074712 22913 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:03.074554 22867 tablet.cc:2406] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:11:03.069882 22959 ts_tablet_manager.cc:1403] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.087s	user 0.024s	sys 0.029s
I20260709 13:11:03.086795 22964 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.087495 22964 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.089744 22964 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.092913 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.093865 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:03.094455 22880 leader_election.cc:343] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.092659 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:03.095296 23189 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:03.097550 22879 leader_election.cc:343] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.098541 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:03.101415 22964 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:03.103417 23189 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:03.108459 22959 raft_consensus.cc:359] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.109561 22959 raft_consensus.cc:385] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.109936 22959 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.123795 22959 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.126784 22958 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.127422 22958 ts_tablet_manager.cc:1403] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.010s
I20260709 13:11:03.129832 22765 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.130842 22958 raft_consensus.cc:359] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.131699 22958 raft_consensus.cc:385] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.132025 22958 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.134722 22960 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.139751 22960 ts_tablet_manager.cc:1403] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.094s	user 0.029s	sys 0.015s
I20260709 13:11:03.141050 23189 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:03.141793 22958 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.143679 22959 ts_tablet_manager.cc:1434] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 13:11:03.146476 22959 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.144094 22960 raft_consensus.cc:359] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.154631 22960 raft_consensus.cc:385] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.147559 22958 ts_tablet_manager.cc:1434] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:11:03.155884 22958 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.156140 22960 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.156917 22960 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.161101 22960 ts_tablet_manager.cc:1434] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:11:03.167004 23156 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:03.170957 22960 tablet_bootstrap.cc:492] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.164681 22959 tablet_bootstrap.cc:654] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.183291 22960 tablet_bootstrap.cc:654] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.187168 22958 tablet_bootstrap.cc:654] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.197795 22991 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.198309 22991 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.200089 22958 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.200627 22959 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.200636 22958 ts_tablet_manager.cc:1403] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.003s
I20260709 13:11:03.200742 22991 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.201391 22959 ts_tablet_manager.cc:1403] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.008s
I20260709 13:11:03.204126 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.204517 22958 raft_consensus.cc:359] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.204721 22959 raft_consensus.cc:359] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.205291 22958 raft_consensus.cc:385] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.205719 22959 raft_consensus.cc:385] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.205750 22958 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.206162 22959 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.206609 22958 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.204836 22764 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:03.206913 22959 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.208776 22958 ts_tablet_manager.cc:1434] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:03.209165 22959 ts_tablet_manager.cc:1434] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:11:03.208685 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.210775 22958 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.210940 22959 tablet_bootstrap.cc:492] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.215471 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.216967 22913 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:03.210865 23156 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.220990 23156 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.221361 23156 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.225565 22958 tablet_bootstrap.cc:654] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.228641 22964 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.229156 22964 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.233292 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:03.234827 22880 leader_election.cc:343] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.237269 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:03.240108 22879 leader_election.cc:343] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.240535 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:03.240886 23156 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.241315 23191 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:03.234901 22964 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.245786 22959 tablet_bootstrap.cc:654] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.250934 23156 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.251566 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:03.252255 22764 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.252846 22960 tablet_bootstrap.cc:492] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.253736 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:03.254030 22960 ts_tablet_manager.cc:1403] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.011s
I20260709 13:11:03.254410 22910 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.255697 22959 tablet_bootstrap.cc:492] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.256263 22959 ts_tablet_manager.cc:1403] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.006s
I20260709 13:11:03.257457 22960 raft_consensus.cc:359] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.258235 22960 raft_consensus.cc:385] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.258509 22960 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.259267 22960 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.260766 22764 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:03.259654 22959 raft_consensus.cc:359] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.262365 22910 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:03.263110 22958 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.263496 22959 raft_consensus.cc:385] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.263934 22959 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.265067 22959 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.263989 22958 ts_tablet_manager.cc:1403] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.002s
I20260709 13:11:03.266419 22988 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:03.266790 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.267300 22959 ts_tablet_manager.cc:1434] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:11:03.268219 22991 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:03.268677 22991 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:03.268800 23189 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:03.269428 22991 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.268522 22958 raft_consensus.cc:359] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.271647 22958 raft_consensus.cc:385] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.271922 22958 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.272607 22958 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.274010 22960 ts_tablet_manager.cc:1434] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.006s
I20260709 13:11:03.274668 22958 ts_tablet_manager.cc:1434] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:11:03.275455 22960 tablet_bootstrap.cc:492] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.275756 22958 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.285180 22958 tablet_bootstrap.cc:654] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.287664 22960 tablet_bootstrap.cc:654] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.295550 22959 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.305647 22959 tablet_bootstrap.cc:654] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.314652 22964 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.315337 22964 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.329245 22964 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.335161 22663 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:03.342160 22958 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.342752 22958 ts_tablet_manager.cc:1403] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.034s
I20260709 13:11:03.345489 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.346254 22764 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:03.347920 22958 raft_consensus.cc:359] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.348941 22958 raft_consensus.cc:385] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.349395 22958 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.350162 22958 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.351338 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:03.357427 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.360778 22964 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.361166 22964 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.361472 22964 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.361764 23191 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:03.358090 22958 ts_tablet_manager.cc:1434] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:11:03.358069 22839 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:03.373206 22964 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.373939 22958 tablet_bootstrap.cc:492] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.378754 22960 tablet_bootstrap.cc:492] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.379335 22960 ts_tablet_manager.cc:1403] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.004s
I20260709 13:11:03.380805 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:03.381429 22765 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.382215 22964 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.384266 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:03.384963 22841 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.387506 22991 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:03.392462 22841 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:03.393637 22958 tablet_bootstrap.cc:654] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.394305 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:03.395468 22964 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:03.397047 22964 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:03.397938 22964 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.396601 22959 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.399340 22959 ts_tablet_manager.cc:1403] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.013s
I20260709 13:11:03.408427 22959 raft_consensus.cc:359] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.409250 22959 raft_consensus.cc:385] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.409483 22959 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.410437 22959 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.413928 22960 raft_consensus.cc:359] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.415160 22960 raft_consensus.cc:385] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.415439 22960 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.416115 22960 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.417675 22960 ts_tablet_manager.cc:1434] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:11:03.418687 22960 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.430200 22959 ts_tablet_manager.cc:1434] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:11:03.432813 22959 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
W20260709 13:11:03.456363 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:03.430619 23195 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:11:03.470301 22958 tablet_bootstrap.cc:492] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.474184 22958 ts_tablet_manager.cc:1403] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.100s	user 0.024s	sys 0.024s
I20260709 13:11:03.464112 23135 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:11:03.479893 22959 tablet_bootstrap.cc:654] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.476755 22765 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:03.493088 22960 tablet_bootstrap.cc:654] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.495507 22958 raft_consensus.cc:359] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.496356 22958 raft_consensus.cc:385] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.496665 22958 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.497475 22958 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.501721 22958 ts_tablet_manager.cc:1434] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.027s	user 0.001s	sys 0.005s
I20260709 13:11:03.502782 22958 tablet_bootstrap.cc:492] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.506906 22662 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.509320 22958 tablet_bootstrap.cc:654] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.522572 23156 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.523331 23156 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.530844 23156 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.534655 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.535647 22766 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:03.538419 22964 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.539090 22964 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.542906 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:03.544813 22880 leader_election.cc:343] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.547235 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.548695 23156 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.549220 23156 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.549710 22964 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.549746 23156 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.551316 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:03.552975 22879 leader_election.cc:343] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.553388 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:03.560355 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.561134 22915 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:03.564685 23135 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:03.568120 23195 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.568981 22959 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.568683 23195 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.569568 22959 ts_tablet_manager.cc:1403] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.137s	user 0.031s	sys 0.016s
I20260709 13:11:03.571321 23195 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.572832 22959 raft_consensus.cc:359] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.573526 22959 raft_consensus.cc:385] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.573801 22959 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.574424 22959 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.576817 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.577664 22914 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:03.579010 22960 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.579702 22960 ts_tablet_manager.cc:1403] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.034s
I20260709 13:11:03.584460 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:03.585528 22839 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:03.595626 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.603142 23195 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.603847 23195 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.604427 23195 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.607074 22960 raft_consensus.cc:359] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.607864 22960 raft_consensus.cc:385] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.608156 22960 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.608258 23156 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.608973 22960 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.611668 23156 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.612257 22959 ts_tablet_manager.cc:1434] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.013s
I20260709 13:11:03.613682 22959 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.614326 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:03.615052 22764 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.620591 22959 tablet_bootstrap.cc:654] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.622208 22960 ts_tablet_manager.cc:1434] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.015s
I20260709 13:11:03.623322 22960 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.631253 22958 tablet_bootstrap.cc:492] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.631830 22958 ts_tablet_manager.cc:1403] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.129s	user 0.028s	sys 0.029s
I20260709 13:11:03.634450 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:03.635119 22914 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.635118 22958 raft_consensus.cc:359] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.635931 22958 raft_consensus.cc:385] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.636224 22958 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.636987 22958 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.640764 22960 tablet_bootstrap.cc:654] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.644905 22988 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.645450 22988 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.647842 22988 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.647994 22958 ts_tablet_manager.cc:1434] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.004s
I20260709 13:11:03.649087 22958 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.652086 22914 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:03.643965 23195 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.659101 22959 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.664199 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.664502 22959 ts_tablet_manager.cc:1403] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.005s
I20260709 13:11:03.662400 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:03.664913 22910 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:03.667281 22958 tablet_bootstrap.cc:654] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.667131 23195 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.672032 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:03.672780 22910 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.673041 22958 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
W20260709 13:11:03.673041 22730 leader_election.cc:343] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.673902 22958 ts_tablet_manager.cc:1403] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 13:11:03.677146 22958 raft_consensus.cc:359] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.677951 22958 raft_consensus.cc:385] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.678254 22958 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.667450 22959 raft_consensus.cc:359] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.679014 22959 raft_consensus.cc:385] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.679399 22959 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.678937 22958 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.677534 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:03.679988 22959 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.680820 23224 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.681174 23224 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.681484 23224 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.681660 22958 ts_tablet_manager.cc:1434] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:03.682261 22959 ts_tablet_manager.cc:1434] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:11:03.682832 22958 tablet_bootstrap.cc:492] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:03.683326 22959 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.669849 22960 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.685762 22991 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.685885 22960 ts_tablet_manager.cc:1403] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.063s	user 0.019s	sys 0.000s
I20260709 13:11:03.686378 23135 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.686882 23135 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.689018 22960 raft_consensus.cc:359] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.689960 22960 raft_consensus.cc:385] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.690258 22960 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.690599 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:03.690922 22964 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.685065 22764 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:03.691412 22964 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.692050 23154 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:03.692606 23154 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:03.690940 22960 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.693948 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.693625 23154 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.694301 22958 tablet_bootstrap.cc:654] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:03.695405 22880 leader_election.cc:343] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.695663 22960 ts_tablet_manager.cc:1434] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:03.696228 22964 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.696787 22960 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.689340 23135 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.698800 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:03.686759 22991 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:03.701184 22879 leader_election.cc:343] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.702569 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:03.703291 22959 tablet_bootstrap.cc:654] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.703630 22839 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:03.705963 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:03.707604 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.710528 22765 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:03.708729 23135 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.711202 23135 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.711663 23135 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.713575 22960 tablet_bootstrap.cc:654] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.713814 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:03.714778 22839 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.715730 23189 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.707928 22910 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:03.716204 23189 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.719231 23189 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.709879 22991 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.726562 22880 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:03.727281 22959 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.727833 22959 ts_tablet_manager.cc:1403] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.002s
I20260709 13:11:03.727963 22964 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.728760 22964 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.729179 22964 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.731045 22959 raft_consensus.cc:359] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.731841 22959 raft_consensus.cc:385] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.732147 22959 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.732898 22959 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.737864 22959 ts_tablet_manager.cc:1434] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 13:11:03.738824 22959 tablet_bootstrap.cc:492] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.739675 22964 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.740640 23224 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.750517 23224 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:03.738744 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61332us
I20260709 13:11:03.755139 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:03.755487 22959 tablet_bootstrap.cc:654] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.755816 22915 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.756320 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.757319 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.759955 22967 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:03.760594 22967 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:03.762650 22967 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.765864 22959 tablet_bootstrap.cc:492] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.766404 22959 ts_tablet_manager.cc:1403] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.004s
I20260709 13:11:03.767547 22915 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:03.769320 22806 leader_election.cc:343] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.770140 22959 raft_consensus.cc:359] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.771248 22959 raft_consensus.cc:385] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.771543 22959 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.772212 22959 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.775292 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:03.776216 22765 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.778986 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.779964 22914 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:03.782642 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:03.783268 22959 ts_tablet_manager.cc:1434] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:11:03.794121 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:03.832135 22959 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:03.800961 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:03.811420 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:03.819912 23191 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:03.839459 22959 tablet_bootstrap.cc:654] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:03.840788 22662 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:03.842417 22839 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:03.844280 22960 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:03.844798 22960 ts_tablet_manager.cc:1403] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.148s	user 0.039s	sys 0.045s
I20260709 13:11:03.797417 23135 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.829919 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:03.831903 22914 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:03.846331 22991 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.846652 22991 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.846963 22991 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.847783 22960 raft_consensus.cc:359] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.848531 22960 raft_consensus.cc:385] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.852831 22991 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.851639 22960 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.854940 23195 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:03.856432 22879 leader_election.cc:343] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.858112 22991 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:03.858939 22958 tablet_bootstrap.cc:492] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:03.859429 22958 ts_tablet_manager.cc:1403] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.177s	user 0.055s	sys 0.064s
I20260709 13:11:03.859555 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:03.828634 22964 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.861948 22988 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:03.863843 22988 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:03.863937 23015 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:03.864609 23015 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.864724 22988 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:03.866590 22730 leader_election.cc:343] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:03.867538 22959 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:03.868017 22959 ts_tablet_manager.cc:1403] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.005s
W20260709 13:11:03.862880 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80309.3us
I20260709 13:11:03.802636 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:03.863013 22960 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.865095 23015 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.855868 23195 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.880961 22991 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:03.886960 22960 ts_tablet_manager.cc:1434] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.042s	user 0.000s	sys 0.007s
I20260709 13:11:03.888160 22960 tablet_bootstrap.cc:492] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:03.897445 22958 raft_consensus.cc:359] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.898576 22958 raft_consensus.cc:385] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:03.898931 22958 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:03.891592 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:03.900579 22915 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.906493 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:03.907482 22761 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:03.926124 22958 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.994925 22964 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.931151 22959 raft_consensus.cc:359] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.995493 22964 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.992105 23191 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.949429 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:04.001982 23191 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.004320 22964 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:04.004343 23191 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:03.944876 22765 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:03.937988 23135 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:03.930029 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53159.5us
I20260709 13:11:04.007346 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:04.007166 22959 raft_consensus.cc:385] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.008325 22959 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.008396 22839 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.009159 22959 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:03.946339 23195 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.016928 22959 ts_tablet_manager.cc:1434] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.149s	user 0.008s	sys 0.003s
I20260709 13:11:03.968134 23039 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
I20260709 13:11:04.018446 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.020058 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.014756 23135 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.022552 23135 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.029834 23246 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.041291 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.041859 23189 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.042272 22840 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:04.043977 23189 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.048036 22959 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
W20260709 13:11:04.029789 22879 leader_election.cc:343] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:04.053253 22879 leader_election.cc:343] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.051268 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:04.055452 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:04.030611 23246 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.072890 22915 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:04.077189 23189 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:04.080972 22880 leader_election.cc:343] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.085390 22880 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:04.085934 22761 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:04.081082 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:04.081300 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:04.086757 22960 tablet_bootstrap.cc:654] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.086805 22915 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:04.083890 22958 ts_tablet_manager.cc:1434] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.224s	user 0.032s	sys 0.035s
I20260709 13:11:04.089177 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:04.086299 23039 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.090266 23039 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:04.091192 23039 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.091850 23189 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.092341 23189 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:04.098939 22839 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:04.076414 23135 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:04.111496 22958 tablet_bootstrap.cc:492] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:04.099815 23154 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:04.112856 23154 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.113291 23154 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.093176 23189 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:04.101645 22880 leader_election.cc:343] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.120857 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.126067 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:04.128010 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 230086us
I20260709 13:11:04.129092 23250 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.129614 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.132951 23246 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.134512 22991 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:04.138643 22663 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:04.141750 23250 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:04.147699 22879 leader_election.cc:343] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.141361 23015 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.152797 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:04.153618 22915 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:04.145675 23250 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.144865 23135 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:04.162465 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.175089 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:04.176092 22764 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
W20260709 13:11:04.149850 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49799us
I20260709 13:11:04.191306 22958 tablet_bootstrap.cc:654] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.162626 23154 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.196920 22839 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:04.202086 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:04.204198 23154 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.210021 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:04.210700 22911 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.178822 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:04.179452 22806 leader_election.cc:343] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.227810 23015 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:04.198092 22880 leader_election.cc:343] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.214036 23195 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:04.238337 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64241.1us
I20260709 13:11:04.239362 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.260109 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:04.260771 22914 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.260416 22730 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:04.270779 22991 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:04.267458 23135 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:04.275478 22960 tablet_bootstrap.cc:492] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:04.276144 22960 ts_tablet_manager.cc:1403] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.388s	user 0.031s	sys 0.042s
I20260709 13:11:04.268568 23195 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.243566 23246 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:11:04.290753 22988 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:04.279325 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83591.2us
I20260709 13:11:04.304159 22959 tablet_bootstrap.cc:654] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:04.318773 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53103.6us
I20260709 13:11:04.309240 22914 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:04.332558 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:04.338069 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:04.342491 22960 raft_consensus.cc:359] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.347110 22960 raft_consensus.cc:385] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.347383 22960 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.336762 22662 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:04.326208 22911 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:04.331804 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:04.343611 23107 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.365553 23107 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:04.366504 23107 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.372099 22663 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:04.377375 22958 tablet_bootstrap.cc:492] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:04.378000 22958 ts_tablet_manager.cc:1403] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.279s	user 0.013s	sys 0.020s
I20260709 13:11:04.379238 22960 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:04.414954 22806 leader_election.cc:343] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.415509 22958 raft_consensus.cc:359] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.416246 23156 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:04.416240 22958 raft_consensus.cc:385] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.418540 22958 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.420559 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:04.419417 22958 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.423293 22960 ts_tablet_manager.cc:1434] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.147s	user 0.007s	sys 0.003s
I20260709 13:11:04.426183 22960 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:04.429687 22988 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:04.430114 23154 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.433027 23195 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:04.437534 22958 ts_tablet_manager.cc:1434] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 13:11:04.444123 22988 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.444749 22988 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.451377 22988 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.453269 22958 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:04.454102 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.453117 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:04.456142 22910 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:04.457593 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:04.459780 22967 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:04.460176 22967 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.460533 22967 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:04.476342 22730 leader_election.cc:343] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.504524 22662 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:04.509018 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 79882us
I20260709 13:11:04.523029 22960 tablet_bootstrap.cc:654] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.535739 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:04.519253 23154 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:04.544391 22959 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:04.547494 22959 ts_tablet_manager.cc:1403] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.500s	user 0.021s	sys 0.032s
W20260709 13:11:04.556694 22730 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.567902 22880 leader_election.cc:343] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.568457 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.571617 23250 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.572106 23039 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:04.572209 23250 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.576087 22967 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.591693 23135 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.592032 23154 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.592427 23135 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.595243 23135 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:04.586426 22663 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:04.625432 22958 tablet_bootstrap.cc:654] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.631376 22967 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.628840 22959 raft_consensus.cc:359] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.640357 22959 raft_consensus.cc:385] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.640810 22959 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.641932 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:04.661906 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:04.669431 22959 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.661769 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.680410 22959 ts_tablet_manager.cc:1434] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.132s	user 0.009s	sys 0.000s
I20260709 13:11:04.681727 23271 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.682327 23271 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.684584 23271 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.671389 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:04.688212 22912 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:04.689072 22730 leader_election.cc:343] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.689867 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:04.690512 22914 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:04.692029 23250 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:04.692229 22766 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:04.681912 22959 tablet_bootstrap.cc:492] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:04.705024 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:04.707228 22879 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.708124 22991 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:04.708503 22879 leader_election.cc:343] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.708242 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:04.710717 22806 leader_election.cc:343] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.720148 23250 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:04.718052 22838 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
W20260709 13:11:04.730273 22880 leader_election.cc:343] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.733981 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.735734 23250 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:04.736140 23250 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.736462 23250 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.759344 22991 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:04.759711 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.772841 23107 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:11:04.803726 23279 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:11:04.805871 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:04.820042 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:04.808279 23107 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:04.836544 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55195.9us
I20260709 13:11:04.838115 22960 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:04.816123 22663 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:04.837430 22806 consensus_peers.cc:597] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.838663 22960 ts_tablet_manager.cc:1403] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.413s	user 0.025s	sys 0.014s
I20260709 13:11:04.840723 22663 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:04.828047 23250 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.846608 23250 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:04.847522 22912 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:04.847788 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:04.849121 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:04.849983 23195 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:04.850468 23195 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:04.850900 22880 leader_election.cc:343] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:04.851365 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:04.841578 23107 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.851322 23195 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.856957 23247 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.857563 23247 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.859992 23247 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:04.847260 22959 tablet_bootstrap.cc:654] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:04.870666 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:04.871801 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:04.875882 22761 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:04.877936 23135 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:04.892189 22730 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:04.903996 23107 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.905306 22837 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:04.907493 22958 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:04.908226 22958 ts_tablet_manager.cc:1403] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.458s	user 0.018s	sys 0.004s
I20260709 13:11:04.903643 22960 raft_consensus.cc:359] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.911057 22960 raft_consensus.cc:385] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.911554 22960 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.916369 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:04.922906 22958 raft_consensus.cc:359] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.923722 22958 raft_consensus.cc:385] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:04.924027 22958 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:04.925011 22958 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.926410 22960 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.939093 23271 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.939644 23271 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.940747 22960 ts_tablet_manager.cc:1434] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.003s
I20260709 13:11:04.934545 23189 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:11:04.953727 22958 ts_tablet_manager.cc:1434] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.045s	user 0.013s	sys 0.000s
I20260709 13:11:04.955003 22958 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:04.963635 23245 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:04.985749 23135 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.986375 23135 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:04.994128 22959 tablet_bootstrap.cc:492] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:04.994730 22959 ts_tablet_manager.cc:1403] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.313s	user 0.013s	sys 0.025s
I20260709 13:11:04.999730 23271 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.003154 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.005259 22910 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:05.009068 22880 leader_election.cc:343] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.009617 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.010948 23283 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:05.011476 23039 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.012181 22964 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.012940 22959 raft_consensus.cc:359] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.013793 22959 raft_consensus.cc:385] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.014093 22959 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.015499 23107 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.016522 23135 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:05.018877 22960 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.027019 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.027922 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:05.028837 22912 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:05.034685 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.040338 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:05.032788 22663 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.041955 23270 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:05.050948 22880 leader_election.cc:343] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.057462 22806 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.061192 23189 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.072253 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.085650 23189 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.085246 22959 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.098099 22960 tablet_bootstrap.cc:654] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.111099 23290 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.116322 23290 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.119462 23290 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:05.120275 22759 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.120451 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:05.122032 22880 leader_election.cc:343] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:05.129118 22879 leader_election.cc:343] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.127321 22759 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:05.129628 22879 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.131707 22964 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.134868 22959 ts_tablet_manager.cc:1434] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.140s	user 0.009s	sys 0.001s
I20260709 13:11:05.136475 22959 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:05.140241 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:05.141191 23189 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.141784 23189 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:05.143010 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.142678 23189 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.150028 22960 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.150630 22960 ts_tablet_manager.cc:1403] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.132s	user 0.008s	sys 0.007s
I20260709 13:11:05.160247 22958 tablet_bootstrap.cc:654] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:05.165222 22879 leader_election.cc:343] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.165767 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.168737 23290 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.183338 23107 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.193907 22960 raft_consensus.cc:359] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.194792 22960 raft_consensus.cc:385] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.195287 22960 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.196094 22960 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.200004 22959 tablet_bootstrap.cc:654] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.210397 22960 ts_tablet_manager.cc:1434] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.000s
I20260709 13:11:05.211547 22960 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.214533 22958 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:05.215174 22958 ts_tablet_manager.cc:1403] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.260s	user 0.009s	sys 0.005s
I20260709 13:11:05.217762 22960 tablet_bootstrap.cc:654] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.217566 22662 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.218187 22958 raft_consensus.cc:359] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.219007 22958 raft_consensus.cc:385] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.219599 22958 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.220736 22958 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.223681 22958 ts_tablet_manager.cc:1434] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:11:05.225447 22958 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:05.235041 22958 tablet_bootstrap.cc:654] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.248299 22959 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:05.248838 22959 ts_tablet_manager.cc:1403] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.113s	user 0.011s	sys 0.007s
I20260709 13:11:05.251664 22959 raft_consensus.cc:359] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.253760 22959 raft_consensus.cc:385] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.254060 22959 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.254926 22959 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.257323 22959 ts_tablet_manager.cc:1434] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:11:05.261780 22960 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.262295 22960 ts_tablet_manager.cc:1403] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.012s
I20260709 13:11:05.264333 23189 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.265281 22958 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:05.265868 22958 ts_tablet_manager.cc:1403] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.005s
I20260709 13:11:05.265615 22960 raft_consensus.cc:359] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.266284 22960 raft_consensus.cc:385] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.266592 22960 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.267298 22960 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.264854 23189 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.269780 22959 tablet_bootstrap.cc:492] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:05.270320 23189 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.274410 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.276808 22959 tablet_bootstrap.cc:654] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.280090 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.280377 22960 ts_tablet_manager.cc:1434] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:11:05.281663 22960 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.281821 22958 raft_consensus.cc:359] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.282526 22958 raft_consensus.cc:385] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.282816 22958 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.288365 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.289589 22960 tablet_bootstrap.cc:654] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.289803 23156 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.294451 23245 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.293139 22958 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.294937 23245 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.304687 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.312785 22958 ts_tablet_manager.cc:1434] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:11:05.314159 22958 tablet_bootstrap.cc:492] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:05.315964 23245 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.318178 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.321759 22958 tablet_bootstrap.cc:654] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.324837 22732 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:05.327833 23247 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.328078 22960 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.329041 22960 ts_tablet_manager.cc:1403] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.009s
I20260709 13:11:05.332661 22960 raft_consensus.cc:359] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.333755 22960 raft_consensus.cc:385] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.334252 22958 tablet_bootstrap.cc:492] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:05.334194 22960 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.335484 22958 ts_tablet_manager.cc:1403] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.005s
I20260709 13:11:05.336014 22960 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.338430 22958 raft_consensus.cc:359] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.339139 22958 raft_consensus.cc:385] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.339392 22958 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.339588 22960 ts_tablet_manager.cc:1434] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:11:05.340166 22958 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.344760 22958 ts_tablet_manager.cc:1434] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:11:05.347887 22958 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:05.356511 23245 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.359117 22960 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.365935 22960 tablet_bootstrap.cc:654] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.371045 22958 tablet_bootstrap.cc:654] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.399555 22959 tablet_bootstrap.cc:492] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:05.400141 22959 ts_tablet_manager.cc:1403] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.131s	user 0.038s	sys 0.034s
I20260709 13:11:05.400584 22960 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.401081 22960 ts_tablet_manager.cc:1403] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.017s
I20260709 13:11:05.403147 22959 raft_consensus.cc:359] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.403898 22959 raft_consensus.cc:385] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.404167 22959 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.404675 22960 raft_consensus.cc:359] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.404982 22959 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.405541 22960 raft_consensus.cc:385] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.406453 22960 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.407238 22959 ts_tablet_manager.cc:1434] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:05.407814 22967 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.407663 22960 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.409026 22959 tablet_bootstrap.cc:492] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:05.410454 22960 ts_tablet_manager.cc:1434] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:05.411468 22960 tablet_bootstrap.cc:492] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.417836 22960 tablet_bootstrap.cc:654] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.423730 23266 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.424296 23266 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.426905 22959 tablet_bootstrap.cc:654] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.427028 23266 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.429895 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.436091 22960 tablet_bootstrap.cc:492] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.436677 22960 ts_tablet_manager.cc:1403] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.005s
I20260709 13:11:05.439517 22958 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:05.440073 22958 ts_tablet_manager.cc:1403] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.092s	user 0.008s	sys 0.007s
I20260709 13:11:05.439752 22960 raft_consensus.cc:359] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.440546 22960 raft_consensus.cc:385] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.440877 22960 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.441571 22960 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.443868 22960 ts_tablet_manager.cc:1434] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:05.446643 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.449188 22958 raft_consensus.cc:359] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.449795 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.449970 22958 raft_consensus.cc:385] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.451647 22958 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.452359 22958 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.456694 22958 ts_tablet_manager.cc:1434] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:11:05.453765 22960 tablet_bootstrap.cc:492] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.454263 23266 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.465458 22960 tablet_bootstrap.cc:654] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.454540 22959 tablet_bootstrap.cc:492] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:05.466125 22959 ts_tablet_manager.cc:1403] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.000s
I20260709 13:11:05.469414 22959 raft_consensus.cc:359] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.470374 22959 raft_consensus.cc:385] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.470876 22959 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.472069 22958 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:05.471846 22959 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.486510 23195 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.486995 23195 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.487927 22959 ts_tablet_manager.cc:1434] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:11:05.490849 22959 tablet_bootstrap.cc:492] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:05.492092 23195 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.494539 22958 tablet_bootstrap.cc:654] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.496325 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.497021 22910 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:05.497412 22959 tablet_bootstrap.cc:654] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.498366 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:05.500239 22730 leader_election.cc:343] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.500909 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:05.501718 23195 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.502205 23195 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.502640 23195 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.503620 22960 tablet_bootstrap.cc:492] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.504117 22960 ts_tablet_manager.cc:1403] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.016s
I20260709 13:11:05.508229 22960 raft_consensus.cc:359] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.508893 22960 raft_consensus.cc:385] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.512559 22960 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.513373 22960 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.515264 23195 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.515921 22960 ts_tablet_manager.cc:1434] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:11:05.520397 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:05.521034 22960 tablet_bootstrap.cc:492] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
W20260709 13:11:05.523557 22730 leader_election.cc:343] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.532936 22960 tablet_bootstrap.cc:654] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.533396 23290 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.533918 23290 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.537029 23195 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.536937 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:05.538987 22880 leader_election.cc:343] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.542761 22958 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:05.542549 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:05.543871 22958 ts_tablet_manager.cc:1403] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.010s
I20260709 13:11:05.543941 22959 tablet_bootstrap.cc:492] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:05.544448 22959 ts_tablet_manager.cc:1403] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.018s
I20260709 13:11:05.542560 22960 tablet_bootstrap.cc:492] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.543967 22911 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.546833 22960 ts_tablet_manager.cc:1403] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.000s
I20260709 13:11:05.547082 22958 raft_consensus.cc:359] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.547911 22958 raft_consensus.cc:385] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.548179 22958 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.548869 22958 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.549350 22959 raft_consensus.cc:359] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.550045 22959 raft_consensus.cc:385] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.550314 22959 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.551128 22958 ts_tablet_manager.cc:1434] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:05.550938 22959 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.552208 22958 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:05.553205 22959 ts_tablet_manager.cc:1434] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:05.554257 22959 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:05.554281 23290 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:05.559954 22958 tablet_bootstrap.cc:654] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.560220 22959 tablet_bootstrap.cc:654] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.560972 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.562294 22759 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.562788 22960 raft_consensus.cc:359] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.563467 22960 raft_consensus.cc:385] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.563716 22960 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.564379 22960 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.564566 22837 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:05.565313 23270 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.565807 23270 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.568692 23270 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.571568 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.572050 22960 ts_tablet_manager.cc:1434] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:11:05.572263 22912 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:05.575523 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:05.578691 23189 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.579123 23189 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.579447 23189 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.580873 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.582784 22960 tablet_bootstrap.cc:492] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:05.586406 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:11:05.589092 22806 leader_election.cc:343] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.589960 23250 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.602069 23250 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.602592 23250 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.607527 23107 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.608212 23107 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.611025 23107 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.614547 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.615314 22910 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:05.616928 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:05.617022 22967 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.617990 22967 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.620493 22967 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.621299 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.621660 23270 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.622056 22838 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:05.622197 23270 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.622645 23270 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.623167 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:05.624006 22967 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:05.624431 22967 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.625075 22967 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.629688 22958 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:05.630234 22958 ts_tablet_manager.cc:1403] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.017s
I20260709 13:11:05.630771 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.631537 22910 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:05.632920 22958 raft_consensus.cc:359] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.633623 22958 raft_consensus.cc:385] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.634089 22958 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.633492 22967 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.634828 22958 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.636719 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:05.637317 22838 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.637558 23266 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:05.643108 23250 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.645682 22837 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.652712 22964 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.653226 22964 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.654585 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:05.655158 22839 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.655467 22958 ts_tablet_manager.cc:1434] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.005s
I20260709 13:11:05.592585 22960 tablet_bootstrap.cc:654] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.595961 22911 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:05.659368 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:05.646350 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.660583 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
W20260709 13:11:05.662289 22880 leader_election.cc:343] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.662853 23189 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.664700 22964 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:05.665761 22958 tablet_bootstrap.cc:492] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:05.666774 22964 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.667258 22964 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.658034 23250 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:05.679533 22967 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.679916 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.684191 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:05.683100 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:05.686335 22964 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:05.687045 22880 leader_election.cc:343] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.695415 23271 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.687105 22967 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.699817 23189 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.702050 23290 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.703308 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.688445 22910 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.715564 22839 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:05.718254 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.704558 23271 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.712495 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:05.716516 22967 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:05.733341 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.735177 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:05.738075 22964 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.721801 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:05.738699 22879 leader_election.cc:343] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.738777 22964 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:05.739849 22964 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.717067 23290 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:05.746837 22879 leader_election.cc:343] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.747432 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:05.744494 22911 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.743888 22838 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:05.749617 23283 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:05.779568 22959 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:05.780375 22959 ts_tablet_manager.cc:1403] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.226s	user 0.030s	sys 0.037s
I20260709 13:11:05.780260 22967 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.785977 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:05.790807 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67666.8us
W20260709 13:11:05.793517 22806 leader_election.cc:343] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.794401 22960 tablet_bootstrap.cc:492] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:05.794988 22960 ts_tablet_manager.cc:1403] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.212s	user 0.012s	sys 0.003s
I20260709 13:11:05.798738 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:05.813185 23270 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.816079 22959 raft_consensus.cc:359] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.816839 22959 raft_consensus.cc:385] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:05.817101 22959 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.817411 23283 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.817909 23283 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.818931 22958 tablet_bootstrap.cc:654] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:05.819834 22959 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.822872 23290 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:05.824656 22959 ts_tablet_manager.cc:1434] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:11:05.826601 22959 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:05.808270 23319 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:05.833534 22910 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:05.859464 22960 raft_consensus.cc:359] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:05.860929 22806 leader_election.cc:343] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:05.862393 22960 raft_consensus.cc:385] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:11:05.855062 23317 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:11:05.864445 22960 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:05.870309 22960 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.855727 23319 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:05.896366 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:05.935001 22759 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.947134 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:05.947888 22914 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:05.958702 22914 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:05.947212 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:05.966353 23270 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.966547 22910 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:05.974350 23271 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:05.964517 23319 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:05.963295 23327 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:11:06.011451 22959 tablet_bootstrap.cc:654] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:06.018667 22880 leader_election.cc:343] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.019147 22880 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.018147 22663 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:05.990291 23283 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:06.020803 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66387.5us
I20260709 13:11:06.012283 22841 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.015213 23326 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.028405 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:06.017433 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:06.022341 22911 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:06.015534 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:06.027290 23326 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.031226 22958 tablet_bootstrap.cc:492] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:06.032399 22958 ts_tablet_manager.cc:1403] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.367s	user 0.032s	sys 0.010s
I20260709 13:11:06.033288 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:11:06.034456 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 82592.3us
I20260709 13:11:06.028090 23250 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:06.049525 22880 leader_election.cc:343] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.052215 23326 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:06.053628 23330 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.037432 23107 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:06.047300 23266 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:06.054311 23330 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.057018 23266 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.057483 23266 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.067054 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:06.064821 22958 raft_consensus.cc:359] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.072009 22958 raft_consensus.cc:385] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.072371 22958 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.075793 23330 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:06.073730 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:06.080751 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:06.063889 23107 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:06.093151 22759 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:06.095868 22958 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.101248 23107 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.116532 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.122507 22958 ts_tablet_manager.cc:1434] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.003s
W20260709 13:11:06.130560 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55270.1us
I20260709 13:11:06.157680 22958 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Bootstrap starting.
I20260709 13:11:06.160128 22960 ts_tablet_manager.cc:1434] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.365s	user 0.018s	sys 0.014s
I20260709 13:11:06.161099 22959 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:06.161810 22959 ts_tablet_manager.cc:1403] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.336s	user 0.016s	sys 0.009s
I20260709 13:11:06.167667 23318 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:06.168097 22960 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:06.170307 23271 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:06.178200 22880 leader_election.cc:343] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.171095 23271 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:06.183352 22880 leader_election.cc:343] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.176877 23318 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:06.204070 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:06.209002 23290 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.211874 23290 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.188789 23318 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.242395 22959 raft_consensus.cc:359] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.247838 22959 raft_consensus.cc:385] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.248131 22959 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.245573 23271 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:06.253489 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:11:06.247408 23341 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:11:06.248790 22959 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.256762 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:06.260702 22959 ts_tablet_manager.cc:1434] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.000s
I20260709 13:11:06.263507 22959 tablet_bootstrap.cc:492] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
W20260709 13:11:06.264518 22879 leader_election.cc:343] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.265347 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:06.266944 22879 leader_election.cc:343] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.267541 22879 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.210455 23266 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.282825 22960 tablet_bootstrap.cc:654] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.276242 23326 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:06.275198 23322 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:06.317230 22730 leader_election.cc:343] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.321373 23266 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:06.321857 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:06.325562 23290 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:06.329890 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:06.334463 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:06.335565 22912 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 1: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:06.345352 22958 tablet_bootstrap.cc:654] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:06.346962 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64858.8us
W20260709 13:11:06.348271 22879 leader_election.cc:343] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.348752 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.343655 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:06.353453 22880 leader_election.cc:343] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.356457 23324 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:06.356526 22806 leader_election.cc:343] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.362519 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:06.365967 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:06.366763 22911 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:06.375450 22732 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:06.376580 22967 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:06.377074 22967 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.377447 22967 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.387102 23247 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.387667 23247 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.387878 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:06.391570 23247 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:06.393004 22664 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.394887 22664 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.384485 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:06.400547 22879 leader_election.cc:343] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.400678 23156 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.403522 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.408622 22967 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.411404 22960 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:06.412051 22960 ts_tablet_manager.cc:1403] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.244s	user 0.012s	sys 0.012s
I20260709 13:11:06.413686 22967 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:06.414237 23350 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:06.416500 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:06.417049 22915 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.422142 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:06.422947 22914 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:06.408733 23290 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:06.426333 22967 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:06.430011 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:06.443084 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:06.445329 22909 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.480260 23156 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:06.497726 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:06.450286 22960 raft_consensus.cc:359] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.482250 22958 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: No bootstrap required, opened a new log
I20260709 13:11:06.526093 22960 raft_consensus.cc:385] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.526640 22960 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.526789 22958 ts_tablet_manager.cc:1403] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent bootstrapping tablet: real 0.387s	user 0.021s	sys 0.022s
I20260709 13:11:06.527436 22960 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.451280 22915 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:06.530279 22837 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.491241 22959 tablet_bootstrap.cc:654] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.533308 22958 raft_consensus.cc:359] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.534579 22958 raft_consensus.cc:385] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.534883 22958 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.535665 22958 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.537609 22958 ts_tablet_manager.cc:1434] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:11:06.538228 22909 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:06.539084 22837 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:06.508160 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:06.547076 22967 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.550316 22662 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.557734 22960 ts_tablet_manager.cc:1434] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.145s	user 0.007s	sys 0.006s
I20260709 13:11:06.557343 22662 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.558897 22960 tablet_bootstrap.cc:492] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:06.569236 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.554857 22967 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:06.573626 23156 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:06.574229 23156 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:06.575124 23156 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.577927 22732 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:06.579429 23195 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:06.579929 23195 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:06.580799 23195 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.582615 22959 tablet_bootstrap.cc:492] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:06.583276 22959 ts_tablet_manager.cc:1403] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.320s	user 0.009s	sys 0.008s
I20260709 13:11:06.586437 22959 raft_consensus.cc:359] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.587843 22959 raft_consensus.cc:385] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.588177 22959 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.591313 23331 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:06.593199 22960 tablet_bootstrap.cc:654] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.588960 22959 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.606578 22959 ts_tablet_manager.cc:1434] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:11:06.607754 22959 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:06.609855 23331 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:06.619838 22959 tablet_bootstrap.cc:654] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.638798 22960 tablet_bootstrap.cc:492] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:06.639673 22960 ts_tablet_manager.cc:1403] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.008s
I20260709 13:11:06.643148 22664 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.643401 22960 raft_consensus.cc:359] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.644591 22960 raft_consensus.cc:385] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.645386 22967 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:06.649698 22959 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:06.650802 22959 ts_tablet_manager.cc:1403] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.002s
I20260709 13:11:06.653741 22959 raft_consensus.cc:359] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.654458 22959 raft_consensus.cc:385] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.654716 22959 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.655476 22959 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.657795 22959 ts_tablet_manager.cc:1434] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:06.658808 22959 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:06.644961 22960 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.684962 22762 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.661166 22960 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.707253 22960 ts_tablet_manager.cc:1434] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.067s	user 0.010s	sys 0.000s
I20260709 13:11:06.708349 22960 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:06.729943 23322 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.730546 23322 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.732837 23322 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:06.734788 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:06.736492 22880 leader_election.cc:343] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.737066 22959 tablet_bootstrap.cc:654] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.735809 22664 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.740955 22960 tablet_bootstrap.cc:654] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.744858 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:06.747442 22879 leader_election.cc:343] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:06.747948 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:06.750828 23334 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:06.779247 23326 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:06.796259 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48854.1us
I20260709 13:11:06.838572 23156 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:06.898075 22960 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:06.898895 22960 ts_tablet_manager.cc:1403] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.191s	user 0.033s	sys 0.041s
I20260709 13:11:06.898911 22959 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:06.899709 22959 ts_tablet_manager.cc:1403] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.241s	user 0.070s	sys 0.073s
I20260709 13:11:06.901659 22960 raft_consensus.cc:359] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.902388 22960 raft_consensus.cc:385] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.902725 22960 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.902529 22959 raft_consensus.cc:359] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.903221 22959 raft_consensus.cc:385] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.903515 22959 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.903545 22960 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.904244 22959 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.905789 22959 ts_tablet_manager.cc:1434] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:06.906347 22960 ts_tablet_manager.cc:1434] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:11:06.907881 22960 tablet_bootstrap.cc:492] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:06.915288 22959 tablet_bootstrap.cc:492] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:06.923269 22959 tablet_bootstrap.cc:654] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.932439 22960 tablet_bootstrap.cc:654] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.942487 22960 tablet_bootstrap.cc:492] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:06.943017 22960 ts_tablet_manager.cc:1403] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.003s
I20260709 13:11:06.942471 22959 tablet_bootstrap.cc:492] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:06.945175 22959 ts_tablet_manager.cc:1403] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.013s
I20260709 13:11:06.947876 22959 raft_consensus.cc:359] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.948585 22959 raft_consensus.cc:385] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.948848 22959 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.949075 22960 raft_consensus.cc:359] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.949808 22960 raft_consensus.cc:385] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:06.950062 22960 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:06.949654 22959 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.951938 22959 ts_tablet_manager.cc:1434] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:06.952845 22960 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.953469 22959 tablet_bootstrap.cc:492] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:06.960595 22959 tablet_bootstrap.cc:654] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.962930 23337 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.963436 23337 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:06.966214 22960 ts_tablet_manager.cc:1434] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:11:06.966997 23337 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:06.970738 22960 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:06.974846 23189 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:06.975565 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:06.976898 23334 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:06.981228 22762 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:06.985314 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:06.986306 23156 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:06.986668 23156 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.986981 23156 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:06.992704 22960 tablet_bootstrap.cc:654] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:06.996205 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.007241 22912 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:06.994728 23156 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.010670 22960 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:07.011408 22960 ts_tablet_manager.cc:1403] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.003s
I20260709 13:11:07.012750 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:07.013443 22762 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.015403 22960 raft_consensus.cc:359] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.016419 22960 raft_consensus.cc:385] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.016772 22960 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.021111 22960 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.023178 23156 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.024276 22960 ts_tablet_manager.cc:1434] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:11:07.029853 23247 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.038142 23247 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.039251 22960 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:07.040865 23247 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.046150 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.046985 22909 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:07.050837 22960 tablet_bootstrap.cc:654] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.053283 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:07.053375 22762 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:07.053980 22915 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.059458 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:07.061985 23338 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:07.062567 23338 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:07.067646 23338 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.068650 23189 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.070140 22959 tablet_bootstrap.cc:492] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:07.070966 22959 ts_tablet_manager.cc:1403] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.118s	user 0.024s	sys 0.078s
I20260709 13:11:07.070710 23189 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.073657 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:07.075003 22806 leader_election.cc:343] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.074735 22959 raft_consensus.cc:359] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.075765 22959 raft_consensus.cc:385] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.076177 23189 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.076243 22959 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.078455 22959 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.081423 22959 ts_tablet_manager.cc:1434] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:11:07.082799 22959 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:07.088490 22915 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:07.092597 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.093304 22915 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:07.095280 23189 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:07.097191 22959 tablet_bootstrap.cc:654] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.099440 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:07.100564 23334 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.101042 23334 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.102644 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:07.131335 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:07.134895 23331 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.105746 23334 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:07.193639 23382 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:11:07.185437 23334 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.197777 23334 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.206629 22960 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:07.207415 22960 ts_tablet_manager.cc:1403] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.169s	user 0.024s	sys 0.041s
I20260709 13:11:07.207489 22664 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:07.214933 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:07.214936 22960 raft_consensus.cc:359] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.215852 22915 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.215916 22960 raft_consensus.cc:385] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.216514 22960 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.218959 22960 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.231211 23331 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:07.237353 23195 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:07.233068 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:07.238966 23189 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.239593 23189 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:07.242918 22806 leader_election.cc:343] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.243643 23331 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:07.246271 23189 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.246515 22960 ts_tablet_manager.cc:1434] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:11:07.247572 22960 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:07.253666 22915 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:07.255224 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:07.255950 22759 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:07.266773 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:07.270365 23156 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.270946 23156 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.271292 23156 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.273865 22959 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:07.274570 22959 ts_tablet_manager.cc:1403] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.003s
I20260709 13:11:07.279000 22959 raft_consensus.cc:359] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.279919 22959 raft_consensus.cc:385] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.280287 22959 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.280154 23156 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.282753 22959 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.284381 23156 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.290977 22967 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.291525 22967 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.292419 22959 ts_tablet_manager.cc:1434] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:11:07.295148 22967 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.296725 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.297461 22914 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:07.299182 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:07.300158 22967 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.300594 22967 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.300989 22967 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:07.308133 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63252.4us
I20260709 13:11:07.310837 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.311507 22915 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:07.293411 23322 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.315136 22967 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.315204 23322 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.319650 23322 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:07.317376 22960 tablet_bootstrap.cc:654] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.311091 22959 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:07.289407 23318 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:11:07.308120 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:07.323778 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:07.324045 22762 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.331625 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:07.332394 22915 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.333292 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:07.333215 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:07.333977 22839 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:07.335750 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:07.336359 22840 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.328888 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:07.328510 22841 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 0.
I20260709 13:11:07.344012 23318 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:11:07.344204 22880 leader_election.cc:343] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.344709 23318 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:07.346961 23318 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.354724 22967 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.354601 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:07.357729 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:07.357896 23322 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:07.358453 22912 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.358596 23322 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.359000 23322 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.359877 22915 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
W20260709 13:11:07.367597 22712 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:11:07.396200 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57357.9us
I20260709 13:11:07.413364 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:07.422492 23156 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:07.423117 23156 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:07.423980 23156 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.425576 22840 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:07.435604 22912 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:07.463265 22762 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:07.470183 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:07.490686 22959 tablet_bootstrap.cc:654] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.475760 23331 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:07.486610 23322 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.493482 23331 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:07.495432 23322 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:07.502521 22960 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:07.503355 22960 ts_tablet_manager.cc:1403] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.256s	user 0.027s	sys 0.019s
I20260709 13:11:07.494791 23331 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.505827 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:07.508533 22880 leader_election.cc:343] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:07.508725 22967 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.509430 22967 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.511458 22967 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.534153 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:07.534989 22840 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.602802 22662 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:07.609014 22662 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:07.607494 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.611622 22960 raft_consensus.cc:359] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.612409 22960 raft_consensus.cc:385] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.612715 22960 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.617631 22960 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.615311 22664 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:07.625332 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:07.632112 22840 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:07.639292 22909 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:07.642776 22960 ts_tablet_manager.cc:1434] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.139s	user 0.015s	sys 0.004s
I20260709 13:11:07.644039 22960 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: Bootstrap starting.
I20260709 13:11:07.644721 22959 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:07.647521 22959 ts_tablet_manager.cc:1403] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.337s	user 0.004s	sys 0.023s
I20260709 13:11:07.650506 22960 tablet_bootstrap.cc:654] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.651372 22959 raft_consensus.cc:359] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.652082 22959 raft_consensus.cc:385] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.652343 22959 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.659832 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:07.663556 23396 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:07.664094 23396 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:07.664520 23394 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:07.665923 22960 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: No bootstrap required, opened a new log
I20260709 13:11:07.664933 23396 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.666489 22960 ts_tablet_manager.cc:1403] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.008s
I20260709 13:11:07.669584 22960 raft_consensus.cc:359] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.670357 22960 raft_consensus.cc:385] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.670751 22960 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.673393 22730 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:07.674713 23331 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.672421 22960 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.678794 22960 ts_tablet_manager.cc:1434] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:11:07.674341 22959 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.687562 22959 ts_tablet_manager.cc:1434] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.001s
I20260709 13:11:07.704751 23156 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:07.706866 23331 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:07.708187 22959 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:07.712690 22661 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:07.758685 22959 tablet_bootstrap.cc:654] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.771590 22959 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:07.775414 22959 ts_tablet_manager.cc:1403] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.003s
I20260709 13:11:07.778805 23394 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:07.785400 22959 raft_consensus.cc:359] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.786329 22959 raft_consensus.cc:385] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.786720 22959 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.787850 22959 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.792205 22959 ts_tablet_manager.cc:1434] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 13:11:07.793273 22959 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Bootstrap starting.
I20260709 13:11:07.811182 22959 tablet_bootstrap.cc:654] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:07.813019 23156 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:07.821457 22959 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: No bootstrap required, opened a new log
I20260709 13:11:07.822082 22959 ts_tablet_manager.cc:1403] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.008s
I20260709 13:11:07.825371 22959 raft_consensus.cc:359] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.826133 22959 raft_consensus.cc:385] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:07.826399 22959 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Initialized, Role: FOLLOWER
I20260709 13:11:07.827083 22959 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.829169 22959 ts_tablet_manager.cc:1434] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:07.905818 22967 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.906440 22967 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.908440 22967 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.909476 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.909406 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:07.910306 22840 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 1: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:07.912824 22730 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:07.914002 22967 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:07.922857 22967 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:07.959089 23394 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:07.983430 23386 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:07.990563 23386 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.991115 23386 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:07.993863 23386 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:07.995664 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:07.996780 22909 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:07.996955 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:07.997912 22762 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 0.
I20260709 13:11:07.998919 22807 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:08.000180 23394 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.000586 23394 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.000931 23394 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.014845 23394 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:08.017799 23394 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:08.017983 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:08.018416 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:08.018841 22766 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.018936 22909 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.027238 22909 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:08.027810 22766 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:08.029179 22807 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:08.030385 23386 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.032178 23386 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:08.033021 23386 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:08.046865 22663 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 0 to 1, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.069437 22967 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.070025 22967 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:08.072013 22967 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:08.073032 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:08.073606 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:08.074205 22840 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:08.074592 22909 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 0.
I20260709 13:11:08.075498 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:08.076293 23331 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:08.076678 23331 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.076694 22967 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.077180 23331 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.085765 23331 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:08.095250 23331 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:08.096829 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:08.097631 22836 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.103384 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:08.104228 22915 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:08.105638 23331 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:11:08.113911 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:08.115165 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:08.116473 23331 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:08.117899 23331 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:08.118839 23331 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:08.119630 22836 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:08.132246 22663 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 0 to 1, leader changed from <none> to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.180522 23393 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.192162 23396 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.447114 23385 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.458865 23156 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.544188 23247 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:08.566331 23195 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:08.715982 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:08.753870 23420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:08.757673 23423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:08.758342 23421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:08.761216 21636 server_base.cc:1061] running on GCE node
I20260709 13:11:08.768103 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:08.768519 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:08.768774 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602668768750 us; error 0 us; skew 500 ppm
I20260709 13:11:08.769484 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:08.773113 21636 webserver.cc:533] Webserver started at http://127.21.33.4:36155/ using document root <none> and password file <none>
I20260709 13:11:08.773756 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:08.774003 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:08.774360 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:08.775949 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-3-root/instance:
uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
format_stamp: "Formatted at 2026-07-09 13:11:08 on dist-test-slave-p8rg"
I20260709 13:11:08.781996 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:11:08.786769 23428 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:08.787786 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:11:08.788147 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-3-root
uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
format_stamp: "Formatted at 2026-07-09 13:11:08 on dist-test-slave-p8rg"
I20260709 13:11:08.788476 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:08.823514 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:08.825238 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:08.826987 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:08.831833 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:08.832159 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:08.832523 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:08.832757 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:08.988385 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.4:37663
I20260709 13:11:08.988535 23490 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.4:37663 every 8 connection(s)
I20260709 13:11:08.999678 21636 auto_leader_rebalancer-test.cc:384] add a tserver: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:09.021376 23491 heartbeater.cc:344] Connected to a master server at 127.21.33.62:44143
I20260709 13:11:09.022111 23491 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:09.022998 23491 heartbeater.cc:507] Master 127.21.33.62:44143 requested a full tablet report, sending...
I20260709 13:11:09.025290 22661 ts_manager.cc:194] Registered new tserver with Master: 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:09.027396 22661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58552
I20260709 13:11:09.536631 22762 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:09.544929 22914 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.547055 22967 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.556041 22836 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.558418 22967 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.559862 23504 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.563066 22914 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:09.580318 22730 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11f984a583c84fbe9ef7b3c817b06c44. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:09.588018 22661 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.582290 22836 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.618865 22840 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:09.629683 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.632575 23385 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:11:09.640067 22762 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.641551 23385 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.648221 23496 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.650208 22909 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:09.664038 22805 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0670a9005da848cd93427b2d2b4aa41d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:09.687107 22766 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:09.675683 22664 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:09.704069 22730 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6da5381f5554dc69fb15991d351fb20. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:09.705647 22909 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.707376 23497 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.707662 22840 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.710282 23497 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:09.710040 22766 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.733472 23504 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.741773 22840 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.746202 22909 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.765833 22766 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:09.774104 22664 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:09.797314 22730 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9951bee4cd6b4966bef18c1e0d25eaca. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:09.797889 22836 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.800259 22915 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:09.800369 23519 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.806398 23519 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:09.828584 22967 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.830637 22836 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.832774 22915 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.851658 22663 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:09.982179 23524 ts_tablet_manager.cc:933] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:09.986501 23524 tablet_copy_client.cc:323] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:10.003042 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet 11f984a583c84fbe9ef7b3c817b06c44 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:10.003885 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet 11f984a583c84fbe9ef7b3c817b06c44 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-11f984a583c84fbe9ef7b3c817b06c44
I20260709 13:11:10.013674 22776 tablet_copy_source_session.cc:215] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:10.018312 23524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f984a583c84fbe9ef7b3c817b06c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:10.030706 23491 heartbeater.cc:499] Master 127.21.33.62:44143 was elected leader, sending a full tablet report...
I20260709 13:11:10.036198 23524 tablet_copy_client.cc:806] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:10.036948 23524 tablet_copy_client.cc:670] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:10.042348 23524 tablet_copy_client.cc:538] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:10.053581 23524 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:10.085783 23524 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:10.087168 23524 tablet_bootstrap.cc:492] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:10.087990 23524 ts_tablet_manager.cc:1403] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.019s
I20260709 13:11:10.091727 23524 raft_consensus.cc:359] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.092667 23524 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:10.093357 23524 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.097538 23524 ts_tablet_manager.cc:1434] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:11:10.101081 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-11f984a583c84fbe9ef7b3c817b06c44 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:10.101768 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-11f984a583c84fbe9ef7b3c817b06c44 on tablet 11f984a583c84fbe9ef7b3c817b06c44 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:10.145884 23524 ts_tablet_manager.cc:933] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:10.149904 23524 tablet_copy_client.cc:323] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:10.162546 22851 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet 0670a9005da848cd93427b2d2b4aa41d from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:10.163239 22851 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet 0670a9005da848cd93427b2d2b4aa41d from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-0670a9005da848cd93427b2d2b4aa41d
I20260709 13:11:10.164987 23531 ts_tablet_manager.cc:933] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:10.173081 22851 tablet_copy_source_session.cc:215] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:10.174405 23531 tablet_copy_client.cc:323] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:10.177214 23524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0670a9005da848cd93427b2d2b4aa41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:10.178474 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet a6da5381f5554dc69fb15991d351fb20 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:10.179083 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet a6da5381f5554dc69fb15991d351fb20 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-a6da5381f5554dc69fb15991d351fb20
I20260709 13:11:10.195137 22776 tablet_copy_source_session.cc:215] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:10.206169 23531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6da5381f5554dc69fb15991d351fb20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:10.208040 23524 tablet_copy_client.cc:806] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:10.208770 23524 tablet_copy_client.cc:670] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:10.218086 23524 tablet_copy_client.cc:538] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:10.228428 23531 tablet_copy_client.cc:806] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:10.231485 23524 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:10.230675 23531 tablet_copy_client.cc:670] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:10.238924 23531 tablet_copy_client.cc:538] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:10.250885 23531 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:10.288834 23531 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:10.290231 23531 tablet_bootstrap.cc:492] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:10.291143 23531 ts_tablet_manager.cc:1403] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.011s
I20260709 13:11:10.294668 23531 raft_consensus.cc:359] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.295640 23531 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:10.296412 23531 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.298996 23531 ts_tablet_manager.cc:1434] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:11:10.301011 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-a6da5381f5554dc69fb15991d351fb20 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:10.301640 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-a6da5381f5554dc69fb15991d351fb20 on tablet a6da5381f5554dc69fb15991d351fb20 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:10.324779 23524 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:10.325819 23524 tablet_bootstrap.cc:492] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:10.326488 23524 ts_tablet_manager.cc:1403] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.096s	user 0.039s	sys 0.041s
I20260709 13:11:10.328935 23524 raft_consensus.cc:359] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.329638 23524 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:10.330265 23524 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.332943 23524 ts_tablet_manager.cc:1434] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:10.334921 22851 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-0670a9005da848cd93427b2d2b4aa41d received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:10.335338 22851 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-0670a9005da848cd93427b2d2b4aa41d on tablet 0670a9005da848cd93427b2d2b4aa41d with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:10.453343 23524 ts_tablet_manager.cc:933] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:10.472689 23466 raft_consensus.cc:1217] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:10.507501 23524 tablet_copy_client.cc:323] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:10.522998 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet 9951bee4cd6b4966bef18c1e0d25eaca from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:10.523888 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet 9951bee4cd6b4966bef18c1e0d25eaca from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-9951bee4cd6b4966bef18c1e0d25eaca
I20260709 13:11:10.585336 23466 raft_consensus.cc:1217] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:10.617700 22776 tablet_copy_source_session.cc:215] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:10.623515 23466 raft_consensus.cc:1217] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:10.624703 23524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9951bee4cd6b4966bef18c1e0d25eaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:10.723944 23524 tablet_copy_client.cc:806] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:10.724815 23524 tablet_copy_client.cc:670] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:10.729808 23524 tablet_copy_client.cc:538] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:10.772663 23524 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:10.794869 23524 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:10.795856 23524 tablet_bootstrap.cc:492] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:10.796566 23524 ts_tablet_manager.cc:1403] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.009s
I20260709 13:11:10.800081 23524 raft_consensus.cc:359] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.801294 23524 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:10.801927 23524 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:10.804482 23524 ts_tablet_manager.cc:1434] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:11:10.807485 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-9951bee4cd6b4966bef18c1e0d25eaca received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:10.808036 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-9951bee4cd6b4966bef18c1e0d25eaca on tablet 9951bee4cd6b4966bef18c1e0d25eaca with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:10.885000 22967 raft_consensus.cc:1064] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:10.888264 22967 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:10.895618 23466 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:10.896772 22836 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:10.897403 23542 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:10.898634 22914 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:10.898752 23504 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:10.900348 22967 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:10.921620 22967 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:10.925237 23466 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:10.926532 22914 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:10.967605 23466 raft_consensus.cc:1217] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:10.973395 22661 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:10.995832 22840 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.047775 22764 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.059425 23466 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.062649 23504 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:11.066256 22912 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.076938 23519 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:11.081650 23497 raft_consensus.cc:1064] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:11.085870 23519 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.085870 23466 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.092229 23497 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.093840 22912 raft_consensus.cc:2955] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.115043 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 11f984a583c84fbe9ef7b3c817b06c44 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:11.122467 23465 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.124614 22836 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.126843 22914 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.129155 22967 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.131978 22664 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.143275 23542 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.153877 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet 11f984a583c84fbe9ef7b3c817b06c44 with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:11.157722 23556 tablet_replica.cc:333] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:11.161562 23497 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.174844 23542 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.177541 23465 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.182808 23540 raft_consensus.cc:1064] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:11.197755 23540 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.211778 23466 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.214499 23556 raft_consensus.cc:2243] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:11.218146 23537 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.219698 22912 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.238600 22840 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.251889 23556 raft_consensus.cc:2272] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:11.251889 22915 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.257009 22766 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.258937 22661 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:11.261826 23385 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.265386 23496 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:11.275053 23556 ts_tablet_manager.cc:1916] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:11.297369 22759 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.307183 23536 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.312209 23466 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.314011 23519 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:11.320700 22759 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.320736 23556 ts_tablet_manager.cc:1929] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:11.325412 23556 log.cc:1199] T 11f984a583c84fbe9ef7b3c817b06c44 P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/11f984a583c84fbe9ef7b3c817b06c44
I20260709 13:11:11.338478 23465 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.345700 22967 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.347280 22912 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.357056 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a6da5381f5554dc69fb15991d351fb20 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:11.361655 22840 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.359018 23466 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.367740 23542 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:11.367154 22662 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.378312 22836 raft_consensus.cc:2955] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.379282 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet 11f984a583c84fbe9ef7b3c817b06c44 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:11.406044 22663 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.420042 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet a6da5381f5554dc69fb15991d351fb20 with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:11.423300 22836 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.425944 23562 tablet_replica.cc:333] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:11.430298 23466 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.431360 23562 raft_consensus.cc:2243] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:11.432381 23562 raft_consensus.cc:2272] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:11.432548 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.435408 23540 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:11.436896 23562 ts_tablet_manager.cc:1916] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:11.437590 23539 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:11.444118 23537 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.452678 22912 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.462908 22967 raft_consensus.cc:1064] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:11.466584 22967 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.468119 23466 raft_consensus.cc:2955] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.477331 23562 ts_tablet_manager.cc:1929] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:11.478940 23562 log.cc:1199] T a6da5381f5554dc69fb15991d351fb20 P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/a6da5381f5554dc69fb15991d351fb20
I20260709 13:11:11.480640 22661 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.482069 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet a6da5381f5554dc69fb15991d351fb20 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:11.495937 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet 0670a9005da848cd93427b2d2b4aa41d with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:11.498767 22912 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.498768 23465 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.511049 23542 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:11.513736 23497 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.516265 22840 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:11.550192 23542 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:11.542906 23564 tablet_replica.cc:333] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:11.557255 23564 raft_consensus.cc:2243] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:11.558092 23564 raft_consensus.cc:2272] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:11.564117 23564 ts_tablet_manager.cc:1916] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:11:11.616641 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 118474us
I20260709 13:11:11.617808 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0670a9005da848cd93427b2d2b4aa41d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:11.617007 23497 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.621245 22840 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.636685 22912 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.642617 23465 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.653987 23564 ts_tablet_manager.cc:1929] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:11.654597 23564 log.cc:1199] T 0670a9005da848cd93427b2d2b4aa41d P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/0670a9005da848cd93427b2d2b4aa41d
I20260709 13:11:11.656672 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet 0670a9005da848cd93427b2d2b4aa41d (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:11.660609 22661 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:11.677593 22762 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:11.714812 23465 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.716605 22967 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:11.719880 22915 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:11.726346 22967 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:11.729007 23519 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.736907 23466 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.742201 22912 raft_consensus.cc:2955] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.746317 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9951bee4cd6b4966bef18c1e0d25eaca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:11.758515 22661 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:11.769384 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet 9951bee4cd6b4966bef18c1e0d25eaca with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:11.770421 23556 tablet_replica.cc:333] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:11.771720 23556 raft_consensus.cc:2243] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:11.772606 23556 raft_consensus.cc:2272] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:11.776862 23556 ts_tablet_manager.cc:1916] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:11.799686 23556 ts_tablet_manager.cc:1929] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:11.800040 23556 log.cc:1199] T 9951bee4cd6b4966bef18c1e0d25eaca P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/9951bee4cd6b4966bef18c1e0d25eaca
I20260709 13:11:11.801743 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet 9951bee4cd6b4966bef18c1e0d25eaca (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:12.852134 22840 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:12.859802 22915 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:12.860554 22805 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a537de05aee48178205ec0e7e0d436e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:12.861167 22766 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:12.861259 23537 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:12.863020 23385 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:12.873425 23385 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:12.888988 22762 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:12.892761 22915 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:12.901988 22840 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:12.906518 22662 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:12.918031 22805 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26c26954cc1042d884cd8813a9346a27. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:12.920521 22912 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:12.922794 22762 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:12.922658 23576 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:12.924662 23537 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:12.935444 23385 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:12.936465 22912 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:12.943224 22762 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:12.964109 22663 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:12.969842 22762 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:12.988009 22730 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 485fd0c9273e482692ddc7589cb09f02. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:12.991200 22912 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:12.992540 23497 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:12.993331 22833 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:12.994668 23497 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.001797 23504 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:13.004079 22912 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:13.005986 22833 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:13.025786 22661 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:13.074806 22915 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.087759 22833 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:13.089224 22762 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:13.090729 23570 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:13.091954 23495 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:13.106554 23570 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:13.112205 22879 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f12b50ff391746348739bd5f7a08a44a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:13.113646 22762 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:13.123971 22833 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:13.128638 22663 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.311470 23597 ts_tablet_manager.cc:933] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:13.313706 23597 tablet_copy_client.cc:323] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:13.315798 22851 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet 26c26954cc1042d884cd8813a9346a27 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:13.316361 22851 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet 26c26954cc1042d884cd8813a9346a27 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-26c26954cc1042d884cd8813a9346a27
I20260709 13:11:13.330911 22851 tablet_copy_source_session.cc:215] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:13.384383 23599 ts_tablet_manager.cc:933] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:13.335274 23597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c26954cc1042d884cd8813a9346a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:13.398808 23599 tablet_copy_client.cc:323] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:13.400852 22851 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet 2a537de05aee48178205ec0e7e0d436e from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:13.401437 22851 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet 2a537de05aee48178205ec0e7e0d436e from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-2a537de05aee48178205ec0e7e0d436e
I20260709 13:11:13.411288 22851 tablet_copy_source_session.cc:215] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:13.431774 23599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a537de05aee48178205ec0e7e0d436e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:13.435796 23597 tablet_copy_client.cc:806] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:13.444614 23597 tablet_copy_client.cc:670] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:13.449276 23597 tablet_copy_client.cc:538] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:13.452903 23599 tablet_copy_client.cc:806] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:13.453651 23599 tablet_copy_client.cc:670] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:13.458467 23599 tablet_copy_client.cc:538] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:13.465278 23597 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:13.468011 23599 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:13.498672 23599 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:13.499689 23599 tablet_bootstrap.cc:492] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:13.499581 23597 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:13.500555 23599 ts_tablet_manager.cc:1403] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.013s
I20260709 13:11:13.500691 23597 tablet_bootstrap.cc:492] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:13.501590 23597 ts_tablet_manager.cc:1403] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.012s
I20260709 13:11:13.504397 23599 raft_consensus.cc:359] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.504788 23597 raft_consensus.cc:359] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.505439 23599 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:13.505810 23597 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:13.506253 23599 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.506579 23597 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.526895 23599 ts_tablet_manager.cc:1434] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.014s
I20260709 13:11:13.532441 23597 ts_tablet_manager.cc:1434] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.014s
I20260709 13:11:13.532589 22851 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-2a537de05aee48178205ec0e7e0d436e received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:13.538746 22851 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-2a537de05aee48178205ec0e7e0d436e on tablet 2a537de05aee48178205ec0e7e0d436e with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:13.540067 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-26c26954cc1042d884cd8813a9346a27 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:13.540529 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-26c26954cc1042d884cd8813a9346a27 on tablet 26c26954cc1042d884cd8813a9346a27 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:13.608045 23599 ts_tablet_manager.cc:933] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:13.610065 23599 tablet_copy_client.cc:323] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:13.612337 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet 485fd0c9273e482692ddc7589cb09f02 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:13.612910 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet 485fd0c9273e482692ddc7589cb09f02 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-485fd0c9273e482692ddc7589cb09f02
I20260709 13:11:13.615219 23597 ts_tablet_manager.cc:933] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:13.617202 23597 tablet_copy_client.cc:323] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:11:13.621312 22776 tablet_copy_source_session.cc:215] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:13.627454 23599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fd0c9273e482692ddc7589cb09f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:13.637863 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet f12b50ff391746348739bd5f7a08a44a from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:11:13.638522 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet f12b50ff391746348739bd5f7a08a44a from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-f12b50ff391746348739bd5f7a08a44a
I20260709 13:11:13.646533 22925 tablet_copy_source_session.cc:215] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:13.655949 23597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f12b50ff391746348739bd5f7a08a44a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:13.659485 23599 tablet_copy_client.cc:806] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:13.660320 23599 tablet_copy_client.cc:670] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:13.666545 23599 tablet_copy_client.cc:538] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:13.677727 23597 tablet_copy_client.cc:806] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:13.678774 23597 tablet_copy_client.cc:670] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:13.683593 23597 tablet_copy_client.cc:538] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:13.690192 23599 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:13.708455 23597 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:13.727823 23599 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:13.729563 23599 tablet_bootstrap.cc:492] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:13.730294 23599 ts_tablet_manager.cc:1403] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.010s
I20260709 13:11:13.734558 23599 raft_consensus.cc:359] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.736984 23599 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:13.739588 23599 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.747499 23599 ts_tablet_manager.cc:1434] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:11:13.749294 23597 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:13.751021 23597 tablet_bootstrap.cc:492] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:13.752125 23597 ts_tablet_manager.cc:1403] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.019s
I20260709 13:11:13.753978 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-485fd0c9273e482692ddc7589cb09f02 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:13.754489 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-485fd0c9273e482692ddc7589cb09f02 on tablet 485fd0c9273e482692ddc7589cb09f02 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:13.757932 23597 raft_consensus.cc:359] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.760416 23597 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:13.761169 23597 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:13.770771 23597 ts_tablet_manager.cc:1434] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.005s
I20260709 13:11:13.772625 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-f12b50ff391746348739bd5f7a08a44a received from {username='slave'} at 127.0.0.1:51142
I20260709 13:11:13.773144 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-f12b50ff391746348739bd5f7a08a44a on tablet f12b50ff391746348739bd5f7a08a44a with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:13.808394 23465 raft_consensus.cc:1217] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:13.924789 23465 raft_consensus.cc:1217] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:14.026335 23465 raft_consensus.cc:1217] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:14.233534 23465 raft_consensus.cc:1217] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:14.412639 23575 raft_consensus.cc:1064] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:14.419092 23575 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.430879 23618 raft_consensus.cc:1064] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:14.440040 23618 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.457895 23465 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.460785 23619 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:11:14.468067 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44235.4us
I20260709 13:11:14.483771 22915 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.492509 22766 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.496762 23615 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:14.502556 23615 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:11:14.503914 22915 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.507198 23465 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.511363 23618 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:14.514618 23618 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:14.524474 22766 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.530177 23575 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:14.541360 23618 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.547791 23497 raft_consensus.cc:1064] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:14.550540 23497 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.552786 23575 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.586897 22661 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.594849 22764 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.600010 23465 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.604343 22836 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.606492 22912 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.609952 22766 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.613770 23466 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.623446 22915 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.634142 23465 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.631618 22915 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.651657 22663 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.654304 23631 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:14.657238 23635 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:14.675638 23631 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:14.729902 23612 raft_consensus.cc:1064] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:14.734458 23612 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.776485 23634 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.781646 22912 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.835585 23466 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.837929 22764 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.842095 23464 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.849043 23593 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:14.855304 22833 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:14.864528 22839 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.868358 23645 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:11:14.870496 23593 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:14.877480 22662 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.874402 22833 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.893859 22836 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.898960 23612 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.907114 23466 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.910799 23465 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:14.911599 23464 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:14.913667 23575 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:14.918533 22914 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:14.913661 23620 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:14.915096 22759 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:14.932440 23618 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:14.935770 23617 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:14.935463 23537 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.943390 22759 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:14.946550 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26c26954cc1042d884cd8813a9346a27 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:14.950902 23463 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:14.954708 23634 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:14.955036 23575 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.965481 23631 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.967283 22836 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.942075 23465 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.978569 22662 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:14.993645 22833 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:14.999528 23634 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:14.986315 22764 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.021054 22833 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.022918 22661 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.982923 23464 raft_consensus.cc:2955] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:11:15.033005 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78673us
I20260709 13:11:14.977383 23463 raft_consensus.cc:2955] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.095665 22766 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.114158 22915 raft_consensus.cc:2955] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.122469 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2a537de05aee48178205ec0e7e0d436e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:15.123674 22664 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.128520 22664 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.137176 22908 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:15.156971 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet 26c26954cc1042d884cd8813a9346a27 with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:15.157368 23463 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:15.166874 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 485fd0c9273e482692ddc7589cb09f02 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:15.169427 23593 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:15.186021 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet 485fd0c9273e482692ddc7589cb09f02 with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:15.186946 22905 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a537de05aee48178205ec0e7e0d436e with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:15.190543 23650 tablet_replica.cc:333] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:15.200481 23650 raft_consensus.cc:2243] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:15.201407 23650 raft_consensus.cc:2272] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:15.201908 22764 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:15.206286 23650 ts_tablet_manager.cc:1916] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:15.206998 23649 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.218510 23593 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:15.224360 22759 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.229696 22651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f12b50ff391746348739bd5f7a08a44a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:15.229040 23465 raft_consensus.cc:2955] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:15.241163 23651 tablet_replica.cc:333] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:15.249732 23650 ts_tablet_manager.cc:1929] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:15.250362 23650 log.cc:1199] T 26c26954cc1042d884cd8813a9346a27 P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/26c26954cc1042d884cd8813a9346a27
I20260709 13:11:15.254036 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet 26c26954cc1042d884cd8813a9346a27 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:15.262743 22663 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:15.275128 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet f12b50ff391746348739bd5f7a08a44a with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:15.278312 23652 tablet_replica.cc:333] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:15.279070 23651 raft_consensus.cc:2243] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:15.279604 23652 raft_consensus.cc:2243] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:15.280105 23651 raft_consensus.cc:2272] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:15.280536 23652 raft_consensus.cc:2272] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:15.284188 23652 ts_tablet_manager.cc:1916] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:15.284243 23651 ts_tablet_manager.cc:1916] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:15.298278 23652 ts_tablet_manager.cc:1929] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:15.298840 23652 log.cc:1199] T f12b50ff391746348739bd5f7a08a44a P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/f12b50ff391746348739bd5f7a08a44a
I20260709 13:11:15.300478 23651 ts_tablet_manager.cc:1929] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:15.300611 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet f12b50ff391746348739bd5f7a08a44a (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:15.301061 23651 log.cc:1199] T 485fd0c9273e482692ddc7589cb09f02 P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/485fd0c9273e482692ddc7589cb09f02
I20260709 13:11:15.302925 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet 485fd0c9273e482692ddc7589cb09f02 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:15.303696 23651 tablet_replica.cc:333] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:15.304652 23651 raft_consensus.cc:2243] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:15.305315 23651 raft_consensus.cc:2272] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:15.308084 23651 ts_tablet_manager.cc:1916] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:15.321008 23651 ts_tablet_manager.cc:1929] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:15.321412 23651 log.cc:1199] T 2a537de05aee48178205ec0e7e0d436e P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/2a537de05aee48178205ec0e7e0d436e
I20260709 13:11:15.322963 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet 2a537de05aee48178205ec0e7e0d436e (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:16.376107 22912 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:16.386147 22833 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:16.387399 23645 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.390400 22766 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:16.390439 22879 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09deb1f2792d4748b024f33e42736814. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:16.391850 23645 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.396276 23657 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.399008 22833 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.416253 22759 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.418464 22766 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:16.422508 22662 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:16.437435 22730 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f79e81f486b4e7dbf2d845891b6b9c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:16.437692 22912 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:16.440693 22833 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:16.443436 23656 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.449957 23656 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:16.461036 23655 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.463014 22833 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.468627 22915 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.495515 22833 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:16.494554 22663 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.507184 22915 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:16.507719 22766 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:16.507951 22805 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e19799c88cba4c9194a9308860a76da7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:16.508759 23653 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.509865 23537 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.523070 23653 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.531133 22915 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.536388 22766 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.559424 22912 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:16.567656 22662 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:16.575838 22879 consensus_peers.cc:597] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4dd4e5f26303415aa699a9fd74ac17e8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:16.576339 22833 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:16.576792 22766 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:16.578504 23657 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:16.579870 23645 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:16.587570 23649 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.591737 22836 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.611949 22759 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:16.615455 22663 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:16.929831 23676 ts_tablet_manager.cc:933] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:16.936757 23676 tablet_copy_client.cc:323] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:11:16.938838 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet 09deb1f2792d4748b024f33e42736814 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:11:16.939394 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet 09deb1f2792d4748b024f33e42736814 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-09deb1f2792d4748b024f33e42736814
I20260709 13:11:16.947537 23678 ts_tablet_manager.cc:933] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:16.948957 22925 tablet_copy_source_session.cc:215] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:16.952777 23676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09deb1f2792d4748b024f33e42736814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:16.954398 23678 tablet_copy_client.cc:323] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:16.956073 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet e19799c88cba4c9194a9308860a76da7 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:16.956564 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet e19799c88cba4c9194a9308860a76da7 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-e19799c88cba4c9194a9308860a76da7
I20260709 13:11:16.964423 22850 tablet_copy_source_session.cc:215] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:16.967972 23678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19799c88cba4c9194a9308860a76da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:16.974318 23676 tablet_copy_client.cc:806] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:16.974970 23676 tablet_copy_client.cc:670] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:16.979482 23676 tablet_copy_client.cc:538] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:16.985102 23678 tablet_copy_client.cc:806] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:16.985810 23678 tablet_copy_client.cc:670] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:16.990377 23676 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:16.991799 23678 tablet_copy_client.cc:538] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:17.001538 23678 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:17.041965 23676 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:17.042997 23676 tablet_bootstrap.cc:492] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:17.043717 23676 ts_tablet_manager.cc:1403] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.054s	user 0.028s	sys 0.019s
I20260709 13:11:17.046895 23676 raft_consensus.cc:359] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.047780 23676 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:17.048460 23676 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.055018 23678 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:17.056264 23678 tablet_bootstrap.cc:492] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:17.057207 23678 ts_tablet_manager.cc:1403] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.056s	user 0.026s	sys 0.003s
I20260709 13:11:17.058532 23676 ts_tablet_manager.cc:1434] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.008s
I20260709 13:11:17.061735 23678 raft_consensus.cc:359] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.062786 23678 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:17.063966 23678 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.067013 23678 ts_tablet_manager.cc:1434] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:17.072603 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-09deb1f2792d4748b024f33e42736814 received from {username='slave'} at 127.0.0.1:51142
I20260709 13:11:17.073514 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-09deb1f2792d4748b024f33e42736814 on tablet 09deb1f2792d4748b024f33e42736814 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:17.073892 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-e19799c88cba4c9194a9308860a76da7 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:17.074539 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-e19799c88cba4c9194a9308860a76da7 on tablet e19799c88cba4c9194a9308860a76da7 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:17.078804 23678 ts_tablet_manager.cc:933] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:17.094513 23678 tablet_copy_client.cc:323] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:17.099035 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet 4f79e81f486b4e7dbf2d845891b6b9c0 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:17.099637 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet 4f79e81f486b4e7dbf2d845891b6b9c0 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-4f79e81f486b4e7dbf2d845891b6b9c0
I20260709 13:11:17.115236 22776 tablet_copy_source_session.cc:215] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:17.119328 23678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f79e81f486b4e7dbf2d845891b6b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:17.135434 23678 tablet_copy_client.cc:806] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:17.136126 23678 tablet_copy_client.cc:670] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:17.140094 23678 tablet_copy_client.cc:538] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:17.147626 23678 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:17.167898 23678 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:17.168730 23678 tablet_bootstrap.cc:492] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:17.169315 23678 ts_tablet_manager.cc:1403] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.001s
I20260709 13:11:17.171695 23678 raft_consensus.cc:359] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.172447 23678 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:17.173105 23678 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.176451 23678 ts_tablet_manager.cc:1434] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:11:17.181316 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-4f79e81f486b4e7dbf2d845891b6b9c0 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:17.182961 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-4f79e81f486b4e7dbf2d845891b6b9c0 on tablet 4f79e81f486b4e7dbf2d845891b6b9c0 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:17.198405 23678 ts_tablet_manager.cc:933] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:17.201926 23678 tablet_copy_client.cc:323] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:11:17.203527 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet 4dd4e5f26303415aa699a9fd74ac17e8 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:11:17.204100 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet 4dd4e5f26303415aa699a9fd74ac17e8 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-4dd4e5f26303415aa699a9fd74ac17e8
I20260709 13:11:17.210491 22925 tablet_copy_source_session.cc:215] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:17.213711 23678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dd4e5f26303415aa699a9fd74ac17e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:17.230780 23678 tablet_copy_client.cc:806] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:17.231382 23678 tablet_copy_client.cc:670] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:17.235611 23678 tablet_copy_client.cc:538] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:17.246388 23678 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:17.264892 23678 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:17.265812 23678 tablet_bootstrap.cc:492] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:17.266461 23678 ts_tablet_manager.cc:1403] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.011s
I20260709 13:11:17.269181 23678 raft_consensus.cc:359] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.269984 23678 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:17.270569 23678 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:17.273034 23678 ts_tablet_manager.cc:1434] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:11:17.274880 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-4dd4e5f26303415aa699a9fd74ac17e8 received from {username='slave'} at 127.0.0.1:51142
I20260709 13:11:17.275286 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-4dd4e5f26303415aa699a9fd74ac17e8 on tablet 4dd4e5f26303415aa699a9fd74ac17e8 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:17.433892 23465 raft_consensus.cc:1217] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:17.524538 23465 raft_consensus.cc:1217] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:17.532384 23465 raft_consensus.cc:1217] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:17.590349 23465 raft_consensus.cc:1217] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:17.831852 23653 raft_consensus.cc:1064] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:17.834324 23653 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:17.845474 23465 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:17.847539 22915 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:17.847815 22766 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:17.848429 23575 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.849339 23653 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:17.852299 23537 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:17.870599 23537 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:17.872489 22915 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:17.894915 22662 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:17.872756 22759 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:17.900058 23465 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:17.969194 23649 raft_consensus.cc:1064] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:17.971169 22836 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:17.972553 23649 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:17.980734 22764 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:17.982260 23465 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:17.987591 23693 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:17.991205 23653 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:18.006448 23463 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.003724 23575 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.018337 22759 raft_consensus.cc:2955] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.033160 22661 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.046530 23463 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.050024 23675 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:18.054147 22836 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.056020 22764 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.057930 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet e19799c88cba4c9194a9308860a76da7 with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:18.058990 23675 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:18.060406 23657 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:18.082588 23698 raft_consensus.cc:1064] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:18.084872 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e19799c88cba4c9194a9308860a76da7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:18.090945 23700 tablet_replica.cc:333] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:18.091454 23698 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:18.107683 23675 raft_consensus.cc:1064] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:18.111114 23675 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:18.122627 23465 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.124552 22912 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.125960 23656 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:18.129146 23698 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:18.133744 22836 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.139174 23700 raft_consensus.cc:2243] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.140645 23656 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:11:18.166221 23700 raft_consensus.cc:2272] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.190111 23465 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.194231 23649 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:18.207480 23700 ts_tablet_manager.cc:1916] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:18.212442 23649 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.215162 22766 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.261601 22836 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.261569 23463 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.265537 22766 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:18.270416 23708 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:18.280508 23708 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:18.295704 22661 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.312489 23675 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.321230 22766 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.322408 23700 ts_tablet_manager.cc:1929] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:18.323102 23700 log.cc:1199] T e19799c88cba4c9194a9308860a76da7 P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/e19799c88cba4c9194a9308860a76da7
I20260709 13:11:18.325309 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet e19799c88cba4c9194a9308860a76da7 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
W20260709 13:11:18.348201 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 155415us
I20260709 13:11:18.352658 23654 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.375789 23463 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.381580 22661 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.390431 23465 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.419137 22661 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.434501 22833 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.459609 22839 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.464866 22912 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:11:18.467069 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78351.9us
I20260709 13:11:18.473987 22833 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.529793 22908 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:18.536775 23463 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:18.536461 22764 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:18.538291 23657 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:18.538722 22833 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:18.545079 23675 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:18.553655 23649 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.555328 23463 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.556650 23465 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:18.558809 23713 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:18.563489 22839 raft_consensus.cc:2955] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.567889 23656 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.573868 22651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 09deb1f2792d4748b024f33e42736814 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:18.575126 22912 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:18.576764 23698 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:18.580655 23465 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.590570 22664 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.594215 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4f79e81f486b4e7dbf2d845891b6b9c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:18.596974 22912 raft_consensus.cc:2955] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.613427 22662 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:18.634910 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f79e81f486b4e7dbf2d845891b6b9c0 with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:18.636834 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet 09deb1f2792d4748b024f33e42736814 with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:18.662564 23716 tablet_replica.cc:333] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:18.663640 23716 raft_consensus.cc:2243] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.664445 23716 raft_consensus.cc:2272] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.676859 23716 ts_tablet_manager.cc:1916] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:18.681875 23715 tablet_replica.cc:333] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:18.683389 23715 raft_consensus.cc:2243] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.684312 23715 raft_consensus.cc:2272] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.688163 23715 ts_tablet_manager.cc:1916] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:18.723646 23715 ts_tablet_manager.cc:1929] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:18.724227 23715 log.cc:1199] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/4f79e81f486b4e7dbf2d845891b6b9c0
I20260709 13:11:18.725962 23716 ts_tablet_manager.cc:1929] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:18.727833 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet 4f79e81f486b4e7dbf2d845891b6b9c0 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:18.726464 23716 log.cc:1199] T 09deb1f2792d4748b024f33e42736814 P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/09deb1f2792d4748b024f33e42736814
I20260709 13:11:18.732161 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet 09deb1f2792d4748b024f33e42736814 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:18.906185 22908 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:18.911906 22833 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:18.912817 23465 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:18.914309 23649 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:18.915607 23675 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:18.925552 23649 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.931607 22839 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.934294 23465 raft_consensus.cc:2955] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.951263 22662 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:18.952668 22651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4dd4e5f26303415aa699a9fd74ac17e8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:18.980072 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet 4dd4e5f26303415aa699a9fd74ac17e8 with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:18.981233 23716 tablet_replica.cc:333] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:18.982622 23716 raft_consensus.cc:2243] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:18.987785 23716 raft_consensus.cc:2272] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:18.992033 23716 ts_tablet_manager.cc:1916] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:19.008798 23716 ts_tablet_manager.cc:1929] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:19.009168 23716 log.cc:1199] T 4dd4e5f26303415aa699a9fd74ac17e8 P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/4dd4e5f26303415aa699a9fd74ac17e8
I20260709 13:11:19.010681 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet 4dd4e5f26303415aa699a9fd74ac17e8 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:20.058596 22764 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:20.065728 22839 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:20.066049 22908 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:20.066905 22730 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: adf76c1fde374ccbbb4a0edda126b851. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.067894 23656 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.069205 23713 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.099092 23717 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.102339 22839 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.103744 22908 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.123119 22764 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:20.126791 22664 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:20.143224 22730 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e36250f751be4e7480619d5807252ac0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.145895 22908 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:20.146776 22839 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:20.147612 23713 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.148950 23717 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.165337 23656 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.172950 22839 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.188469 22912 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.199927 22662 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.207859 22839 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:20.250406 22912 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:20.252497 22759 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:20.253476 22805 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b86ad75e17d742e2926f8af53f082682. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.256096 23653 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.260177 23731 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.267542 23653 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.270762 22759 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.275125 22912 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.290498 22839 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:20.289436 22662 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:20.306895 22805 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f55844ac3c9a4ceaa35b3d5bdadad40e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.316083 22912 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:20.317874 23575 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:20.324628 22759 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:20.327764 23731 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:20.332304 23653 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.334506 22912 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.340224 22759 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.361431 22662 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:20.641407 23741 ts_tablet_manager.cc:933] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:20.667261 23741 tablet_copy_client.cc:323] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:20.670169 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet adf76c1fde374ccbbb4a0edda126b851 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:20.671128 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet adf76c1fde374ccbbb4a0edda126b851 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-adf76c1fde374ccbbb4a0edda126b851
I20260709 13:11:20.690969 23743 ts_tablet_manager.cc:933] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:20.739369 22776 tablet_copy_source_session.cc:215] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:20.741631 23743 tablet_copy_client.cc:323] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:20.744182 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet b86ad75e17d742e2926f8af53f082682 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:20.744997 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet b86ad75e17d742e2926f8af53f082682 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-b86ad75e17d742e2926f8af53f082682
I20260709 13:11:20.756150 23741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf76c1fde374ccbbb4a0edda126b851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.767315 22850 tablet_copy_source_session.cc:215] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:20.772336 23743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b86ad75e17d742e2926f8af53f082682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.775377 23745 ts_tablet_manager.cc:933] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:20.789296 23745 tablet_copy_client.cc:323] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:20.792408 23747 ts_tablet_manager.cc:933] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:20.793989 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet e36250f751be4e7480619d5807252ac0 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:20.798308 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet e36250f751be4e7480619d5807252ac0 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-e36250f751be4e7480619d5807252ac0
I20260709 13:11:20.804626 23743 tablet_copy_client.cc:806] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:20.805538 23743 tablet_copy_client.cc:670] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:20.810984 23743 tablet_copy_client.cc:538] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:20.823485 23743 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:20.834872 23741 tablet_copy_client.cc:806] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:20.835561 23741 tablet_copy_client.cc:670] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:20.848426 23741 tablet_copy_client.cc:538] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:20.862668 23747 tablet_copy_client.cc:323] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:20.864864 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet f55844ac3c9a4ceaa35b3d5bdadad40e from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:20.865571 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet f55844ac3c9a4ceaa35b3d5bdadad40e from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-f55844ac3c9a4ceaa35b3d5bdadad40e
I20260709 13:11:20.871521 23743 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:20.875962 23743 tablet_bootstrap.cc:492] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:20.877030 22850 tablet_copy_source_session.cc:215] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:20.877197 23743 ts_tablet_manager.cc:1403] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.018s
I20260709 13:11:20.881532 23747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55844ac3c9a4ceaa35b3d5bdadad40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.882279 23743 raft_consensus.cc:359] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:20.885555 23743 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:20.904206 22776 tablet_copy_source_session.cc:215] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:20.886521 23743 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:20.913111 23741 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:20.919668 23745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e36250f751be4e7480619d5807252ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:20.970201 23747 tablet_copy_client.cc:806] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:20.975761 23747 tablet_copy_client.cc:670] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:20.992748 23747 tablet_copy_client.cc:538] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:21.007755 23745 tablet_copy_client.cc:806] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:21.008527 23745 tablet_copy_client.cc:670] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:21.027330 23745 tablet_copy_client.cc:538] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:21.040758 23743 ts_tablet_manager.cc:1434] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.163s	user 0.016s	sys 0.043s
I20260709 13:11:21.049744 23463 raft_consensus.cc:1217] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:21.085748 23747 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:21.086163 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-b86ad75e17d742e2926f8af53f082682 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:21.087183 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-b86ad75e17d742e2926f8af53f082682 on tablet b86ad75e17d742e2926f8af53f082682 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:21.092801 23741 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:21.117749 23741 tablet_bootstrap.cc:492] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:21.118248 23745 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:21.118659 23741 ts_tablet_manager.cc:1403] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.216s	user 0.023s	sys 0.021s
I20260709 13:11:21.125466 23741 raft_consensus.cc:359] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:21.126543 23741 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:21.127302 23741 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:21.134320 23741 ts_tablet_manager.cc:1434] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.005s
I20260709 13:11:21.142234 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-adf76c1fde374ccbbb4a0edda126b851 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:21.144979 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-adf76c1fde374ccbbb4a0edda126b851 on tablet adf76c1fde374ccbbb4a0edda126b851 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:21.181944 23747 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:21.183109 23747 tablet_bootstrap.cc:492] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:21.183856 23747 ts_tablet_manager.cc:1403] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.098s	user 0.032s	sys 0.025s
I20260709 13:11:21.187341 23747 raft_consensus.cc:359] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:21.188268 23747 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:21.188987 23747 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:21.196417 23747 ts_tablet_manager.cc:1434] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:11:21.198239 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-f55844ac3c9a4ceaa35b3d5bdadad40e received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:21.198716 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-f55844ac3c9a4ceaa35b3d5bdadad40e on tablet f55844ac3c9a4ceaa35b3d5bdadad40e with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:21.226949 23745 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:21.228086 23745 tablet_bootstrap.cc:492] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:21.228863 23745 ts_tablet_manager.cc:1403] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.116s	user 0.019s	sys 0.009s
I20260709 13:11:21.232203 23745 raft_consensus.cc:359] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:21.233080 23745 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:21.233770 23745 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:21.236676 23745 ts_tablet_manager.cc:1434] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:21.239346 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-e36250f751be4e7480619d5807252ac0 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:21.239802 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-e36250f751be4e7480619d5807252ac0 on tablet e36250f751be4e7480619d5807252ac0 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:21.331434 23465 raft_consensus.cc:1217] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:21.570866 23463 raft_consensus.cc:1217] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:21.676589 23731 raft_consensus.cc:1064] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:21.680052 23731 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:21.712203 23465 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.714071 23738 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.715950 22759 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.718662 22909 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.724977 23653 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.726495 23465 raft_consensus.cc:1217] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:21.728271 23653 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:21.756928 23740 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.759194 23465 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.783595 23738 raft_consensus.cc:1064] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:21.786864 23738 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:21.811024 23463 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.813030 22909 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.813560 23731 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:21.818231 23731 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.824302 22766 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.826825 22664 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.826911 23758 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:11:21.840022 22759 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.845366 22909 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.876802 23738 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.879169 22759 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.947773 23465 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.957223 22839 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:21.967437 23463 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:21.971289 22661 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:21.969233 23738 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:11:21.997243 23731 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.007880 23465 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.008917 22912 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.019032 22909 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.018628 22839 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:22.022727 23738 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:22.030598 22759 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.033156 23653 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:22.035607 23463 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.035905 23753 raft_consensus.cc:1064] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:22.038301 23758 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:22.042955 23753 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:22.049760 23740 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.067626 22759 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.088359 22662 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.093683 23465 raft_consensus.cc:2955] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.096662 22909 raft_consensus.cc:2955] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.111650 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b86ad75e17d742e2926f8af53f082682 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:22.117242 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f55844ac3c9a4ceaa35b3d5bdadad40e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:22.126641 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet b86ad75e17d742e2926f8af53f082682 with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:22.130095 22661 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.143962 22914 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.144184 23462 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.145777 23753 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:22.146719 22839 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.147862 23753 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:22.150007 23713 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:22.201699 23752 raft_consensus.cc:1064] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:22.204927 23752 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:22.214380 23717 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.232471 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet f55844ac3c9a4ceaa35b3d5bdadad40e with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:22.237224 23465 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.255829 22912 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.259682 23779 tablet_replica.cc:333] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:22.264483 23774 tablet_replica.cc:333] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:22.287806 22662 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.290818 23774 raft_consensus.cc:2243] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:22.295892 23774 raft_consensus.cc:2272] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:22.304603 23774 ts_tablet_manager.cc:1916] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:22.305122 23465 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.322264 23752 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:22.328055 23779 raft_consensus.cc:2243] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:22.328935 23779 raft_consensus.cc:2272] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:22.332064 22912 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.335947 23713 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:22.340399 23779 ts_tablet_manager.cc:1916] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:22.361613 22839 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.387874 23717 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:22.391420 22841 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.396512 23713 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.407896 23463 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.421048 23774 ts_tablet_manager.cc:1929] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:22.421861 23774 log.cc:1199] T b86ad75e17d742e2926f8af53f082682 P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/b86ad75e17d742e2926f8af53f082682
I20260709 13:11:22.424338 22759 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:22.427858 23779 ts_tablet_manager.cc:1929] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:22.428457 23779 log.cc:1199] T f55844ac3c9a4ceaa35b3d5bdadad40e P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/f55844ac3c9a4ceaa35b3d5bdadad40e
I20260709 13:11:22.431607 22914 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.432792 23463 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.433997 22661 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.437211 22833 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.437983 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet b86ad75e17d742e2926f8af53f082682 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:22.440514 23780 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:22.443110 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet f55844ac3c9a4ceaa35b3d5bdadad40e (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:22.453697 23753 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.470665 23465 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.478139 22759 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:22.482558 23752 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:22.497586 23463 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.499739 23713 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:22.505612 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet adf76c1fde374ccbbb4a0edda126b851 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:22.507817 22914 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:22.509438 23780 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:22.510989 22841 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.512427 23780 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.519938 23465 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.524863 22836 raft_consensus.cc:2955] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.519901 22660 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.539721 22914 raft_consensus.cc:2955] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.554106 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet adf76c1fde374ccbbb4a0edda126b851 with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:22.555759 23779 tablet_replica.cc:333] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:22.556896 23779 raft_consensus.cc:2243] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:22.559188 23779 raft_consensus.cc:2272] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:22.562386 22662 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:22.579715 23779 ts_tablet_manager.cc:1916] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:22.581925 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e36250f751be4e7480619d5807252ac0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:22.602471 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet e36250f751be4e7480619d5807252ac0 with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:22.621596 23779 ts_tablet_manager.cc:1929] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:22.622182 23779 log.cc:1199] T adf76c1fde374ccbbb4a0edda126b851 P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/adf76c1fde374ccbbb4a0edda126b851
I20260709 13:11:22.624792 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet adf76c1fde374ccbbb4a0edda126b851 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:22.635468 23785 tablet_replica.cc:333] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:22.636850 23785 raft_consensus.cc:2243] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:22.638679 23785 raft_consensus.cc:2272] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:22.643137 23785 ts_tablet_manager.cc:1916] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:22.658497 23785 ts_tablet_manager.cc:1929] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:22.658922 23785 log.cc:1199] T e36250f751be4e7480619d5807252ac0 P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/e36250f751be4e7480619d5807252ac0
I20260709 13:11:22.660775 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet e36250f751be4e7480619d5807252ac0 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:23.692765 22762 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:23.704524 22730 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f2aaf3c1715e4d29954bd6ee38914a7a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:23.707250 22836 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:23.707351 22914 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:23.709092 23713 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:23.710191 23753 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:23.742041 23753 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:23.745285 22914 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:23.747464 22836 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:23.767150 22661 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:23.769846 22836 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:23.778504 22914 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:23.782155 23740 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:23.783586 22805 consensus_peers.cc:597] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6a04bfd7f184dd7a86718230ea0680e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:23.791419 22762 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:23.793991 23731 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:23.796628 23770 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:23.806454 22914 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:23.837024 22766 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:23.879725 22730 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b88bd2b1b1d4ced918486ea751fbe25. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:23.893991 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 54995.6us
I20260709 13:11:23.902740 22841 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:23.907568 23789 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:23.907826 22662 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:23.910079 22766 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:23.911856 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 87470.2us
I20260709 13:11:23.926098 22914 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:23.939775 23753 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.004159 23789 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.011564 22915 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.039958 22836 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.041592 22660 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.053929 22914 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.069380 22833 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:24.074354 22762 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:24.069916 22879 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82f76532c7d04012a58fce30b62b344d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:24.076669 23797 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.077627 23729 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.107324 23729 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.112013 22764 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.111884 22836 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:24.134819 22662 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:24.179143 23808 ts_tablet_manager.cc:933] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:24.181823 23808 tablet_copy_client.cc:323] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:24.184173 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet f2aaf3c1715e4d29954bd6ee38914a7a from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:24.184741 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet f2aaf3c1715e4d29954bd6ee38914a7a from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-f2aaf3c1715e4d29954bd6ee38914a7a
I20260709 13:11:24.191237 22776 tablet_copy_source_session.cc:215] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:24.195369 23808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2aaf3c1715e4d29954bd6ee38914a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.217116 23808 tablet_copy_client.cc:806] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:24.217840 23808 tablet_copy_client.cc:670] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:24.223259 23808 tablet_copy_client.cc:538] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:24.238739 23808 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:24.266459 23808 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:24.268558 23808 tablet_bootstrap.cc:492] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:24.269814 23808 ts_tablet_manager.cc:1403] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.031s	user 0.023s	sys 0.008s
I20260709 13:11:24.273756 23808 raft_consensus.cc:359] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.274902 23808 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:24.275664 23808 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.278357 23811 ts_tablet_manager.cc:933] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:24.303849 23808 ts_tablet_manager.cc:1434] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.017s
I20260709 13:11:24.306463 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-f2aaf3c1715e4d29954bd6ee38914a7a received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:24.307024 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-f2aaf3c1715e4d29954bd6ee38914a7a on tablet f2aaf3c1715e4d29954bd6ee38914a7a with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:24.307126 23811 tablet_copy_client.cc:323] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:24.309381 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet d6a04bfd7f184dd7a86718230ea0680e from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:24.309995 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet d6a04bfd7f184dd7a86718230ea0680e from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-d6a04bfd7f184dd7a86718230ea0680e
I20260709 13:11:24.320763 22850 tablet_copy_source_session.cc:215] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:24.325486 23811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6a04bfd7f184dd7a86718230ea0680e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.357777 23811 tablet_copy_client.cc:806] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:24.358402 23811 tablet_copy_client.cc:670] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:24.363807 23811 tablet_copy_client.cc:538] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:24.371699 23808 ts_tablet_manager.cc:933] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:24.373579 23808 tablet_copy_client.cc:323] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:24.375020 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet 0b88bd2b1b1d4ced918486ea751fbe25 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:24.375478 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet 0b88bd2b1b1d4ced918486ea751fbe25 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-0b88bd2b1b1d4ced918486ea751fbe25
I20260709 13:11:24.377183 23811 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:24.383184 22776 tablet_copy_source_session.cc:215] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:24.390691 23808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b88bd2b1b1d4ced918486ea751fbe25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.416093 23811 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:24.417248 23811 tablet_bootstrap.cc:492] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:24.419123 23811 ts_tablet_manager.cc:1403] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.042s	user 0.029s	sys 0.009s
I20260709 13:11:24.422003 23808 tablet_copy_client.cc:806] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:24.422686 23808 tablet_copy_client.cc:670] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:24.425457 23811 raft_consensus.cc:359] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.427309 23811 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:24.428092 23811 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.430852 23811 ts_tablet_manager.cc:1434] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:11:24.434682 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-d6a04bfd7f184dd7a86718230ea0680e received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:24.435202 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-d6a04bfd7f184dd7a86718230ea0680e on tablet d6a04bfd7f184dd7a86718230ea0680e with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:24.436559 23808 tablet_copy_client.cc:538] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:24.451251 23808 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:24.479214 23808 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:24.480327 23808 tablet_bootstrap.cc:492] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:24.481053 23808 ts_tablet_manager.cc:1403] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.004s
I20260709 13:11:24.484133 23808 raft_consensus.cc:359] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.485014 23808 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:24.485697 23808 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.488510 23808 ts_tablet_manager.cc:1434] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:11:24.490438 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-0b88bd2b1b1d4ced918486ea751fbe25 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:24.491094 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-0b88bd2b1b1d4ced918486ea751fbe25 on tablet 0b88bd2b1b1d4ced918486ea751fbe25 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:24.609854 23808 ts_tablet_manager.cc:933] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:24.612947 23808 tablet_copy_client.cc:323] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:11:24.616494 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet 82f76532c7d04012a58fce30b62b344d from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:11:24.617168 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet 82f76532c7d04012a58fce30b62b344d from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-82f76532c7d04012a58fce30b62b344d
I20260709 13:11:24.626565 22925 tablet_copy_source_session.cc:215] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:24.631623 23808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f76532c7d04012a58fce30b62b344d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:24.650358 23808 tablet_copy_client.cc:806] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:24.651036 23808 tablet_copy_client.cc:670] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:24.661190 23808 tablet_copy_client.cc:538] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:24.669229 23808 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:24.676903 23463 raft_consensus.cc:1217] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:24.704025 23808 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:24.706301 23808 tablet_bootstrap.cc:492] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:24.707073 23808 ts_tablet_manager.cc:1403] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.006s
I20260709 13:11:24.711354 23808 raft_consensus.cc:359] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.712373 23808 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:24.712962 23808 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:24.714931 23808 ts_tablet_manager.cc:1434] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:24.717815 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-82f76532c7d04012a58fce30b62b344d received from {username='slave'} at 127.0.0.1:51142
I20260709 13:11:24.718587 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-82f76532c7d04012a58fce30b62b344d on tablet 82f76532c7d04012a58fce30b62b344d with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:24.857553 23463 raft_consensus.cc:1217] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:24.900023 23463 raft_consensus.cc:1217] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:25.056452 23463 raft_consensus.cc:1217] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:25.102900 23790 raft_consensus.cc:1064] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:25.106765 23790 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.122430 23463 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.123394 22912 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.125792 23790 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.128209 22836 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.130800 23789 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.133093 23789 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.169436 23789 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.175278 22841 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.172334 23463 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.210602 22660 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.233896 22915 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.262075 22764 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.267357 23462 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.268767 23825 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:25.273646 22915 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.275172 23789 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:25.277931 23790 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.283975 23462 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.293877 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f2aaf3c1715e4d29954bd6ee38914a7a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:25.293226 22914 raft_consensus.cc:2955] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.306814 22662 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.314181 23825 raft_consensus.cc:1064] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:25.319898 23825 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.336720 23463 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.338423 23789 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:25.338840 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet f2aaf3c1715e4d29954bd6ee38914a7a with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:25.343858 22915 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.352487 23831 tablet_replica.cc:333] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:25.354001 23831 raft_consensus.cc:2243] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:25.361687 23790 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.363654 23831 raft_consensus.cc:2272] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:25.373960 22836 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.377363 23825 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.401566 23831 ts_tablet_manager.cc:1916] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:25.428308 23825 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.432364 23740 raft_consensus.cc:1064] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:25.435266 22914 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.444196 22836 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.444952 23740 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.457999 23462 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.458181 22915 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.459460 23831 ts_tablet_manager.cc:1929] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:25.460039 23831 log.cc:1199] T f2aaf3c1715e4d29954bd6ee38914a7a P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/f2aaf3c1715e4d29954bd6ee38914a7a
I20260709 13:11:25.459826 23801 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.460281 23462 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.464148 23801 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:25.466274 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet f2aaf3c1715e4d29954bd6ee38914a7a (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:25.468232 22662 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.477116 22764 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.482921 23801 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.488248 22764 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.512948 23462 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.518055 23821 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:25.521339 22836 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.540019 23821 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:25.549770 23825 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.550575 23835 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.552747 23463 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.556619 22914 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.558736 23462 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.569566 22836 raft_consensus.cc:2955] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.578971 22766 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.588029 22662 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.610803 22841 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.620615 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0b88bd2b1b1d4ced918486ea751fbe25 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:25.636654 22662 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.643678 23841 raft_consensus.cc:1064] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:25.643728 22914 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.647598 23801 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:25.643828 23462 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.654675 23801 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:25.657856 23841 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.665162 23770 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.670367 22912 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.678171 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b88bd2b1b1d4ced918486ea751fbe25 with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:25.693717 23465 raft_consensus.cc:2955] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.699335 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6a04bfd7f184dd7a86718230ea0680e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:25.715283 22841 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.715554 23463 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.721503 22766 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:25.723294 23784 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:25.729727 23797 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:25.732158 23797 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:25.733971 22661 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.739410 23843 tablet_replica.cc:333] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:25.740934 23843 raft_consensus.cc:2243] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:25.742293 23843 raft_consensus.cc:2272] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:25.751209 23843 ts_tablet_manager.cc:1916] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:25.759967 23842 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.770486 23465 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.772951 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet d6a04bfd7f184dd7a86718230ea0680e with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:25.777664 23843 ts_tablet_manager.cc:1929] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:25.778389 23843 log.cc:1199] T 0b88bd2b1b1d4ced918486ea751fbe25 P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/0b88bd2b1b1d4ced918486ea751fbe25
I20260709 13:11:25.780669 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet 0b88bd2b1b1d4ced918486ea751fbe25 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:25.795078 23848 tablet_replica.cc:333] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:25.798895 22664 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.806664 22764 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.822472 22836 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.860442 23848 raft_consensus.cc:2243] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:25.863576 23848 raft_consensus.cc:2272] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:25.869658 23848 ts_tablet_manager.cc:1916] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:25.873109 22912 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:25.888216 23462 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.888219 22836 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:25.890725 23842 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:25.892805 23842 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:25.898725 23784 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.902647 23462 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:25.913173 22651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82f76532c7d04012a58fce30b62b344d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:25.958582 22662 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.911876 22841 raft_consensus.cc:2955] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:26.009089 23848 ts_tablet_manager.cc:1929] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:26.010900 23848 log.cc:1199] T d6a04bfd7f184dd7a86718230ea0680e P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/d6a04bfd7f184dd7a86718230ea0680e
I20260709 13:11:26.020402 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet 82f76532c7d04012a58fce30b62b344d with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:26.021822 23848 tablet_replica.cc:333] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:26.022949 23848 raft_consensus.cc:2243] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:26.023785 23848 raft_consensus.cc:2272] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:26.027825 23848 ts_tablet_manager.cc:1916] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:26.028257 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet d6a04bfd7f184dd7a86718230ea0680e (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:26.067468 23848 ts_tablet_manager.cc:1929] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:26.068199 23848 log.cc:1199] T 82f76532c7d04012a58fce30b62b344d P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/82f76532c7d04012a58fce30b62b344d
I20260709 13:11:26.072278 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet 82f76532c7d04012a58fce30b62b344d (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:27.116999 22914 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.127140 22841 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:27.127411 22879 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38001bb9b85f4814aff1beccb3ae9acb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.127492 22764 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.128747 23849 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.129621 23797 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.139591 23842 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.141589 22841 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.141378 22764 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.162997 22662 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.164299 22837 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.183889 22914 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.184667 22764 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:27.185923 22805 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17f68c054a034150b6ed9a31489ec653. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.186599 23851 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:27.187822 23850 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.210295 22914 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.206314 23850 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.217461 22764 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.231261 22762 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.242112 22664 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:27.253237 22730 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a71fa884e652460c841521a01aa8b5a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.253715 22837 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.254997 22915 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.255529 23821 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:27.257349 23790 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.269459 23790 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.272490 22915 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.276520 22837 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.288440 22841 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.291782 22661 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:27.315924 22805 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 659f33da809544df8f5fc087bbe7cc17. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.316598 22914 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.319060 23850 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.321004 22762 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.322274 23850 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.336448 23834 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.339679 22762 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.343567 22914 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:27.355755 22664 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:27.508477 23871 ts_tablet_manager.cc:933] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:27.511026 23871 tablet_copy_client.cc:323] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:11:27.513406 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet 38001bb9b85f4814aff1beccb3ae9acb from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:11:27.514120 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet 38001bb9b85f4814aff1beccb3ae9acb from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-38001bb9b85f4814aff1beccb3ae9acb
I20260709 13:11:27.522039 22925 tablet_copy_source_session.cc:215] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:27.526085 23871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38001bb9b85f4814aff1beccb3ae9acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:27.547236 23871 tablet_copy_client.cc:806] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:27.548048 23871 tablet_copy_client.cc:670] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:27.552028 23871 tablet_copy_client.cc:538] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:27.563198 23871 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:27.587112 23871 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:27.588050 23871 tablet_bootstrap.cc:492] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:27.588845 23871 ts_tablet_manager.cc:1403] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.008s
I20260709 13:11:27.592132 23871 raft_consensus.cc:359] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.593082 23871 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:27.593724 23871 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.597312 23871 ts_tablet_manager.cc:1434] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:11:27.599473 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-38001bb9b85f4814aff1beccb3ae9acb received from {username='slave'} at 127.0.0.1:51142
I20260709 13:11:27.599946 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-38001bb9b85f4814aff1beccb3ae9acb on tablet 38001bb9b85f4814aff1beccb3ae9acb with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:27.660382 23871 ts_tablet_manager.cc:933] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:27.663410 23871 tablet_copy_client.cc:323] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.1:45095
I20260709 13:11:27.665876 22776 tablet_copy_service.cc:140] P acfda9b350734def9b6c621f66a5f247: Received BeginTabletCopySession request for tablet a71fa884e652460c841521a01aa8b5a0 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:58100)
I20260709 13:11:27.666484 22776 tablet_copy_service.cc:161] P acfda9b350734def9b6c621f66a5f247: Beginning new tablet copy session on tablet a71fa884e652460c841521a01aa8b5a0 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:58100: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-a71fa884e652460c841521a01aa8b5a0
I20260709 13:11:27.675274 22776 tablet_copy_source_session.cc:215] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:27.686195 23871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71fa884e652460c841521a01aa8b5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:27.686241 23876 ts_tablet_manager.cc:933] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:27.690402 23876 tablet_copy_client.cc:323] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:27.692114 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet 17f68c054a034150b6ed9a31489ec653 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:27.692625 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet 17f68c054a034150b6ed9a31489ec653 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-17f68c054a034150b6ed9a31489ec653
I20260709 13:11:27.700423 22850 tablet_copy_source_session.cc:215] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:27.708477 23876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f68c054a034150b6ed9a31489ec653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:27.709061 23871 tablet_copy_client.cc:806] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:27.710210 23871 tablet_copy_client.cc:670] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:27.715392 23871 tablet_copy_client.cc:538] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:27.728302 23871 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:27.731912 23876 tablet_copy_client.cc:806] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:27.732682 23876 tablet_copy_client.cc:670] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:27.756330 23876 tablet_copy_client.cc:538] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:27.758102 23871 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:27.759203 23871 tablet_bootstrap.cc:492] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:27.760001 23871 ts_tablet_manager.cc:1403] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.032s	user 0.020s	sys 0.011s
I20260709 13:11:27.763605 23871 raft_consensus.cc:359] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.764586 23871 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:27.765265 23871 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.771595 23876 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:27.773170 23871 ts_tablet_manager.cc:1434] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:11:27.775252 22776 tablet_copy_service.cc:342] P acfda9b350734def9b6c621f66a5f247: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-a71fa884e652460c841521a01aa8b5a0 received from {username='slave'} at 127.0.0.1:58100
I20260709 13:11:27.775723 22776 tablet_copy_service.cc:434] P acfda9b350734def9b6c621f66a5f247: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-a71fa884e652460c841521a01aa8b5a0 on tablet a71fa884e652460c841521a01aa8b5a0 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:27.798677 23876 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:27.800972 23876 tablet_bootstrap.cc:492] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:27.801975 23876 ts_tablet_manager.cc:1403] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.019s
I20260709 13:11:27.818397 23876 raft_consensus.cc:359] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.819458 23876 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:27.820444 23876 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:27.831265 23876 ts_tablet_manager.cc:1434] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 13:11:27.835060 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-17f68c054a034150b6ed9a31489ec653 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:27.835585 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-17f68c054a034150b6ed9a31489ec653 on tablet 17f68c054a034150b6ed9a31489ec653 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:27.894059 23462 raft_consensus.cc:1217] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:27.913097 23876 ts_tablet_manager.cc:933] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:27.934655 23876 tablet_copy_client.cc:323] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
I20260709 13:11:27.937165 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet 659f33da809544df8f5fc087bbe7cc17 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:27.937744 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet 659f33da809544df8f5fc087bbe7cc17 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-659f33da809544df8f5fc087bbe7cc17
I20260709 13:11:27.944386 22850 tablet_copy_source_session.cc:215] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:27.948184 23876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659f33da809544df8f5fc087bbe7cc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:27.975445 23876 tablet_copy_client.cc:806] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:27.976351 23876 tablet_copy_client.cc:670] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:27.980386 23876 tablet_copy_client.cc:538] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:27.990836 23876 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:28.044345 23876 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:28.046819 23876 tablet_bootstrap.cc:492] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:28.047557 23876 ts_tablet_manager.cc:1403] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.057s	user 0.029s	sys 0.019s
I20260709 13:11:28.050707 23876 raft_consensus.cc:359] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:28.051563 23876 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:28.052202 23876 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:28.054757 23876 ts_tablet_manager.cc:1434] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:11:28.056535 22850 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-659f33da809544df8f5fc087bbe7cc17 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:28.056983 22850 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-659f33da809544df8f5fc087bbe7cc17 on tablet 659f33da809544df8f5fc087bbe7cc17 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:28.206060 23462 raft_consensus.cc:1217] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:28.280412 23462 raft_consensus.cc:1217] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:11:28.428296 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45218.7us
W20260709 13:11:28.439610 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64095.1us
I20260709 13:11:28.500499 23462 raft_consensus.cc:1217] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:28.502570 23883 raft_consensus.cc:1064] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:28.505395 23883 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.523006 23462 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.527482 22837 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.529466 23887 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.531054 22762 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.531610 23887 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.535173 23880 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.558048 23880 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.561200 23465 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.576674 22841 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.581450 22661 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.584893 23867 raft_consensus.cc:1064] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:28.588058 23867 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.581946 22762 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.613095 22914 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.614029 22837 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.615448 23465 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.617549 23884 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.619997 23821 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.620963 23866 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:28.646219 23821 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.650696 22837 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.651362 22914 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.666973 23465 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.674809 22912 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.678427 22664 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.681947 23462 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:28.684625 22762 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:28.684664 23883 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:28.686474 23887 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:28.729229 23834 raft_consensus.cc:1064] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:28.746132 23842 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.750468 23463 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.758759 23834 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.760185 22651 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 38001bb9b85f4814aff1beccb3ae9acb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:28.760066 22759 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.766484 22762 raft_consensus.cc:2955] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.782457 22660 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:28.787305 23462 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:28.794144 23821 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:28.799741 22912 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:28.808351 22912 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.809921 23884 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:28.811589 22762 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.811587 23834 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.816573 23462 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.817811 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet 38001bb9b85f4814aff1beccb3ae9acb with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:28.817865 23801 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.813129 23463 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:28.812983 23888 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.833212 23801 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.858659 23903 tablet_replica.cc:333] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:28.870718 22662 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New cstate: current_term: 1 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.877384 22650 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a71fa884e652460c841521a01aa8b5a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:28.873656 22914 raft_consensus.cc:2955] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.872326 23801 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.886063 23903 raft_consensus.cc:2243] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:28.887085 23903 raft_consensus.cc:2272] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:28.889184 22912 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.892585 23903 ts_tablet_manager.cc:1916] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:28.905761 22830 tablet_service.cc:1558] Processing DeleteTablet for tablet a71fa884e652460c841521a01aa8b5a0 with delete_type TABLET_DATA_TOMBSTONED (TS 776baf137f8745dd866379ca15ab8cee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39290
I20260709 13:11:28.911902 22759 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.927359 23462 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.928803 22662 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:28.936268 23903 ts_tablet_manager.cc:1929] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:28.936866 23903 log.cc:1199] T 38001bb9b85f4814aff1beccb3ae9acb P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/38001bb9b85f4814aff1beccb3ae9acb
I20260709 13:11:28.938818 23903 tablet_replica.cc:333] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:11:28.938742 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet 38001bb9b85f4814aff1beccb3ae9acb (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:28.940023 23903 raft_consensus.cc:2243] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:28.940923 23903 raft_consensus.cc:2272] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:28.947397 23903 ts_tablet_manager.cc:1916] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:28.960494 23885 raft_consensus.cc:1064] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:28.967676 23885 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.979815 22837 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:28.993492 23903 ts_tablet_manager.cc:1929] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:28.999676 23903 log.cc:1199] T a71fa884e652460c841521a01aa8b5a0 P 776baf137f8745dd866379ca15ab8cee: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/wals/a71fa884e652460c841521a01aa8b5a0
I20260709 13:11:29.002640 23465 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:29.003472 22649 catalog_manager.cc:5028] TS 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): tablet a71fa884e652460c841521a01aa8b5a0 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:29.004417 23801 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.005129 22914 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:29.008888 23851 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.012637 23834 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.020697 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:29.029206 23465 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.045011 22915 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:29.047958 22759 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:29.047277 22915 raft_consensus.cc:2955] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.048410 23462 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:29.049898 23850 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.052193 23850 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.058655 23885 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.068460 22660 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 3 to 4, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.072603 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 17f68c054a034150b6ed9a31489ec653 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:29.092123 22752 tablet_service.cc:1558] Processing DeleteTablet for tablet 17f68c054a034150b6ed9a31489ec653 with delete_type TABLET_DATA_TOMBSTONED (TS acfda9b350734def9b6c621f66a5f247 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46946
I20260709 13:11:29.098305 23911 tablet_replica.cc:333] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:11:29.099551 23911 raft_consensus.cc:2243] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:29.100399 23911 raft_consensus.cc:2272] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:29.103853 23911 ts_tablet_manager.cc:1916] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:29.114773 23850 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.118383 22759 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.136247 23911 ts_tablet_manager.cc:1929] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:29.137915 23911 log.cc:1199] T 17f68c054a034150b6ed9a31489ec653 P acfda9b350734def9b6c621f66a5f247: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/wals/17f68c054a034150b6ed9a31489ec653
I20260709 13:11:29.139928 23465 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.140867 22664 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 reported cstate change: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.144517 22650 catalog_manager.cc:5028] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): tablet 17f68c054a034150b6ed9a31489ec653 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:29.148602 22914 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.161175 22837 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.179396 22762 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:29.181655 23885 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.191742 23851 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.199244 22759 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.203258 23465 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:29.203626 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 659f33da809544df8f5fc087bbe7cc17 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:29.205896 23834 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:29.213056 23465 raft_consensus.cc:2955] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:11:29.222996 22664 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index 3 to 4, VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) evicted. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:29.238072 22899 tablet_service.cc:1558] Processing DeleteTablet for tablet 659f33da809544df8f5fc087bbe7cc17 with delete_type TABLET_DATA_TOMBSTONED (TS 6545453e74344bb2957dd3729c5b3870 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43260
I20260709 13:11:29.243580 23914 tablet_replica.cc:333] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:11:29.245002 23914 raft_consensus.cc:2243] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:29.246954 23914 raft_consensus.cc:2272] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:29.252354 23914 ts_tablet_manager.cc:1916] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:29.266182 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:29.266844 22837 raft_consensus.cc:606] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:29.269559 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:29.270061 22837 raft_consensus.cc:606] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:29.270632 23914 ts_tablet_manager.cc:1929] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:29.271087 23914 log.cc:1199] T 659f33da809544df8f5fc087bbe7cc17 P 6545453e74344bb2957dd3729c5b3870: Deleting WAL directory at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/wals/659f33da809544df8f5fc087bbe7cc17
I20260709 13:11:29.272737 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:29.272858 22651 catalog_manager.cc:5028] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): tablet 659f33da809544df8f5fc087bbe7cc17 (table test-workload [id=bdcf5bfb06a44f0d9ccbe49dac135e6f]) successfully deleted
I20260709 13:11:29.273262 22837 raft_consensus.cc:606] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:29.275489 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:29.276077 22837 raft_consensus.cc:606] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:29.294216 22759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
mode: GRACEFUL
new_leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:44588
I20260709 13:11:29.294768 22759 raft_consensus.cc:606] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Received request to transfer leadership to TS 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:29.298599 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:29.299403 22837 raft_consensus.cc:606] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:29.300770 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:29.302233 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:29.354059 23888 raft_consensus.cc:993] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: : Instructing follower 74c0a02901bc45c891e42d0d1bbbf6a2 to start an election
I20260709 13:11:29.354715 23886 raft_consensus.cc:1081] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Signalling peer 74c0a02901bc45c891e42d0d1bbbf6a2 to start an election
I20260709 13:11:29.356331 23465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0"
dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:43702
I20260709 13:11:29.356906 23465 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.357229 23465 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.362567 23465 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.365828 23465 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:29.366539 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:29.369313 22914 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.367926 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:29.374292 22759 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.374650 22759 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:29.375423 22759 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.377882 22759 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.379530 23801 raft_consensus.cc:993] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.381054 23851 raft_consensus.cc:993] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.381669 22914 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:29.383087 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:29.384395 23885 raft_consensus.cc:1081] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.385762 23874 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.387198 23834 raft_consensus.cc:1081] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.391821 23874 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:29.392879 23874 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.394481 22912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:43274
I20260709 13:11:29.395061 22912 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.395422 22912 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.396024 22759 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:29.399037 22915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:43274
I20260709 13:11:29.399667 22915 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.400009 22915 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.408707 22912 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.409396 23885 raft_consensus.cc:993] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 74c0a02901bc45c891e42d0d1bbbf6a2 to start an election
I20260709 13:11:29.410233 23885 raft_consensus.cc:1081] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 74c0a02901bc45c891e42d0d1bbbf6a2 to start an election
I20260709 13:11:29.412263 23465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27"
dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:43708
I20260709 13:11:29.411579 22664 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.412930 23465 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.413353 23465 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.416090 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:29.416711 22762 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.420621 22915 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.423388 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:29.423808 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:29.423960 22759 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.424383 22837 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.424791 22837 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:29.424552 23465 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.425664 22837 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.427027 22837 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.427474 22762 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.424364 22915 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:29.429968 22912 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:29.430562 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:29.430922 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:29.431139 22836 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.431839 22836 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:29.432526 23465 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:29.432662 22836 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.433859 22836 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.432621 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:29.437125 22841 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.437235 22837 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.437474 22841 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:29.438282 22841 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.439579 22841 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.439618 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:29.440523 23855 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.444363 23842 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.445698 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:29.446638 22912 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.449353 22836 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.453598 22841 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:29.455062 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:29.456241 23874 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.456774 23842 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:29.457152 23874 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:29.460937 22759 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.461531 23855 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:29.466957 23855 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.458511 23874 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:29.461364 23842 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.474660 22912 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:29.490360 22664 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.495885 22660 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.497658 22660 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.535195 23917 raft_consensus.cc:993] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.535713 23801 raft_consensus.cc:1081] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.537256 22912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:43274
I20260709 13:11:29.537788 22912 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.538043 22912 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.542891 23917 raft_consensus.cc:993] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.543437 23801 raft_consensus.cc:1081] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:29.544104 22912 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.544899 22915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:43274
I20260709 13:11:29.545480 22915 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.547053 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:29.547441 22915 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.547724 22759 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.549002 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:29.549568 22841 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.549851 22841 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:29.550594 22841 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.553200 22841 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.556553 22759 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.559046 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:29.560071 23924 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.560685 23924 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:29.561599 23924 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.565352 22912 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:29.567467 22915 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.570358 22841 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.572726 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:29.573307 22764 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.573625 22915 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:29.574589 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:29.575127 22841 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.575435 22841 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:29.576263 22841 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.577360 22841 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.584728 22841 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.585816 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:29.581229 22764 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:29.587406 23842 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.589350 23842 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:29.590302 23842 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:29.592124 22660 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.610833 22660 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.883246 22912 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:29.885284 23920 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:29.886910 22764 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:29.889544 23920 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:29.905790 22836 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:29.912951 23842 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:29.948127 22837 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:29.950084 23920 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:29.950791 22915 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:29.962778 23920 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:30.015120 22762 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.025435 23923 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:30.040505 22837 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.051509 22766 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.052143 23842 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:30.057050 23924 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:30.015259 22759 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.162062 22839 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.173568 22766 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.174422 23855 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:30.193164 23924 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:30.226846 22836 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.229386 23842 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:30.229818 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 220517us
W20260709 13:11:30.242858 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50514.7us
W20260709 13:11:30.225867 23944 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:11:30.253485 23923 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:11:30.320957 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:30.324326 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 71184.5us
W20260709 13:11:30.327198 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77745.5us
W20260709 13:11:30.447188 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 102429us
W20260709 13:11:30.482249 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 189401us
W20260709 13:11:30.499464 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 158656us
I20260709 13:11:30.512274 22836 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:30.554535 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:30.619360 22805 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f666ea4542d4951919f36de0b43b972. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:30.623579 22764 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:30.626181 23960 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:30.629738 22912 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:30.632701 23957 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:30.642803 23960 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:30.742520 22836 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:30.731382 22909 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:30.748718 22758 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:31.458724 23973 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:11:31.699566 22758 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.947s	user 0.007s	sys 0.017s
I20260709 13:11:31.827622 23976 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:31.828553 23976 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:31.829970 23974 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:31.831707 23974 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:31.832304 23976 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:31.833841 23950 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:31.834079 23974 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:31.834622 23950 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:31.837065 23950 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:31.844611 23975 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:31.845233 23975 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:31.856794 22664 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:31.860764 23975 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:31.890384 22716 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 1.198s	user 0.021s	sys 0.013s
W20260709 13:11:31.890880 22716 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 1.199s	user 0.022s	sys 0.014s
I20260709 13:11:31.910866 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1652) took 1069 ms. Trace:
I20260709 13:11:31.911463 22914 rpcz_store.cc:276] 0709 13:11:30.840976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.909977 (+1069001us) service_pool.cc:224] Handling call
0709 13:11:31.910271 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:31.910442 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:31.910464 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:31.910499 (+    35us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:31.910525 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:31.910586 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:31.910669 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:31.940022 22757 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:31.950747 23963 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:31.951848 23963 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:31.983484 23950 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:31.984447 23950 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.465438 23985 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:32.466710 23985 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.479005 23988 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.496937 23986 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.498066 23986 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:32.501844 23984 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.502539 23984 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.504925 23984 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:32.505722 23801 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.480078 23988 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.617061 23974 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:32.519510 23986 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:32.617177 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:32.617739 23960 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.629635 23801 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.650605 23995 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:32.651624 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:32.642835 23963 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:32.656006 22841 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:32.668227 23801 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:32.674065 23801 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.657662 23974 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.670394 23950 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:32.665271 23963 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:32.679786 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.028s	sys 0.056s
W20260709 13:11:32.682338 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.029s	sys 0.056s
I20260709 13:11:32.703670 23801 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.706316 23998 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.706457 23983 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:32.682643 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 783710us
W20260709 13:11:32.692653 22805 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af39123de9434cc0b451f4b03d305310. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:32.712589 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 156953us
I20260709 13:11:32.687579 23984 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.651680 23995 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.727360 23995 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:32.728127 23995 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:32.737879 23991 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:32.738987 23991 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:32.728781 23995 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:32.674528 23960 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:32.744761 23993 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:32.745205 23801 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:32.707544 23990 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:32.730156 23950 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:32.712518 23963 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:32.748548 23985 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:32.797801 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48579.8us
W20260709 13:11:32.800612 22878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.042s	sys 0.045s
I20260709 13:11:32.798000 24000 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:32.807192 22878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.042s	sys 0.046s
I20260709 13:11:32.747877 23984 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:32.802217 23988 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:32.800094 24003 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:32.707283 23998 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:33.103045 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 203221us
I20260709 13:11:32.798744 23993 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:32.815413 23999 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.104502 23997 ts_tablet_manager.cc:933] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:33.010262 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1280 after lost signal to thread 21636
I20260709 13:11:33.103899 23991 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.125389 24005 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.126071 24005 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.134603 24005 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.134898 23996 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:33.135517 24005 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.135572 23996 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.136456 24005 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.010247 24004 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:32.804661 24000 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.109828 23967 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:33.120734 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002d11c0 after lost signal to thread 21639
I20260709 13:11:32.799057 23983 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.108050 24002 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.157620 23974 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:32.822152 23990 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.153935 23992 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.122713 23950 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:33.291303 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec180 after lost signal to thread 22698
I20260709 13:11:33.318611 24009 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:33.105692 23999 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.323465 23990 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.309546 23974 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:33.302848 23995 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.402113 23998 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:33.326222 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec0c0 after lost signal to thread 22712
I20260709 13:11:33.107089 24003 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.291018 24004 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:33.620577 24007 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:11:33.319775 24009 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.646286 23993 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.649020 23997 tablet_copy_client.cc:323] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
W20260709 13:11:33.650486 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.014s
I20260709 13:11:33.389091 23990 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.393797 23960 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.458133 24005 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:33.654547 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.006s	sys 0.015s
I20260709 13:11:33.176879 23967 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.405066 23984 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.391983 24013 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.454083 23963 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.454329 23974 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.392936 23992 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.676736 24004 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.683809 23992 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.684612 23992 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:33.677932 24015 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.324972 24002 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.664251 23993 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.668979 23960 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.668725 24005 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.681520 24004 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.735488 24014 ts_tablet_manager.cc:933] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.665735 23994 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:33.744571 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1623) took 1810 ms. Trace:
I20260709 13:11:33.745478 24006 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.750046 24000 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.751464 23983 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.681130 23963 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.725109 23996 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.687755 23990 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.764633 23967 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.765973 23999 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.766501 23994 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.780680 23950 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.680936 24013 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.776223 24006 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.758831 23960 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.755404 24005 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.774115 23999 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.788674 23999 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.815819 23974 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.822830 23994 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.754426 24004 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.825755 24006 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.757882 23993 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.828707 24009 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:33.833010 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:33.833972 22841 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:33.843479 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 274) took 1040 ms. Trace:
I20260709 13:11:33.844486 23461 rpcz_store.cc:276] 0709 13:11:32.803024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.803441 (+   417us) service_pool.cc:224] Handling call
0709 13:11:33.099700 (+296259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.724372 (+624672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:33.728213 (+  3841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:33.739847 (+ 11634us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:33.739915 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.843003 (+103088us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54002c7754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:33.843091 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:33.843350 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307806976}
I20260709 13:11:33.846585 23967 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.849552 23967 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.847222 23986 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.853142 23986 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.855872 23986 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:33.848233 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:33.861063 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:33.861888 22836 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:33.863755 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet 2f666ea4542d4951919f36de0b43b972 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:33.864794 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet 2f666ea4542d4951919f36de0b43b972 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-2f666ea4542d4951919f36de0b43b972
I20260709 13:11:33.871644 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:33.875873 23982 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.879135 23979 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.880191 23979 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.881978 24006 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:33.882985 24006 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.885849 24006 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.876722 23982 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:33.748559 22761 rpcz_store.cc:276] 0709 13:11:31.934246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.952925 (+ 18679us) service_pool.cc:224] Handling call
0709 13:11:33.743589 (+1790664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.743898 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:33.743940 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:33.744000 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:33.744056 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.744143 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:33.744325 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:33.858243 23967 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:33.889928 23981 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:33.803098 23963 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:33.679772 23487 debug-util.cc:398] Leaking SignalData structure 0x7b080034cee0 after lost signal to thread 22790
I20260709 13:11:33.898278 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 267) took 1981 ms. Trace:
I20260709 13:11:33.898546 23924 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:33.685258 23992 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:33.760872 24015 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:34.537537 24007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.000s	sys 0.005s
I20260709 13:11:34.801153 23463 rpcz_store.cc:276] 0709 13:11:31.917105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.923691 (+  6586us) service_pool.cc:224] Handling call
0709 13:11:33.897209 (+1973518us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:33.897453 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.897730 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:33.897762 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:33.897800 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:33.897834 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.897896 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:33.898017 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25366912}
I20260709 13:11:34.624567 23886 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:33.888139 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:34.774178 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.039s	sys 0.078s
W20260709 13:11:34.753225 24014 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.003s	sys 0.006s
W20260709 13:11:36.224885 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.771s	user 0.041s	sys 0.080s
W20260709 13:11:36.225003 24014 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 2.488s	user 0.004s	sys 0.007s
I20260709 13:11:36.225826 24014 tablet_copy_client.cc:323] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.2:41075
W20260709 13:11:34.808810 22731 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.013s	sys 0.006s
I20260709 13:11:34.778616 23990 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:36.227460 22731 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.339s	user 0.014s	sys 0.006s
I20260709 13:11:34.790719 24002 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.228112 24002 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.228494 24002 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.228870 24002 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:34.787853 24021 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:34.816952 23981 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.230470 24021 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.232033 23981 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.232777 24021 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:34.818364 23982 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:34.821305 24015 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:34.825577 24005 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.234163 24005 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.234670 24005 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:34.827035 23992 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.236871 24005 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.233263 24021 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:34.796658 24006 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.238399 24021 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.238827 24006 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.240545 24021 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.241544 24006 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:34.815654 24007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.001s	sys 0.007s
I20260709 13:11:34.822156 23999 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:34.829077 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.013s	sys 0.016s
W20260709 13:11:36.242872 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.352s	user 0.014s	sys 0.016s
I20260709 13:11:34.811303 23924 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:34.770642 23960 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:34.819332 23963 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:34.623370 23979 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:34.817914 23886 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.252537 23886 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:34.827711 23487 debug-util.cc:398] Leaking SignalData structure 0x7b080034c680 after lost signal to thread 22865
I20260709 13:11:34.831148 24019 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.257683 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:36.257463 24019 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.258340 23462 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:36.257095 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:36.259958 23464 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:36.260386 24019 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.260294 24002 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:35.540871 24030 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:35.543318 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.013s	sys 0.016s
I20260709 13:11:35.546708 24031 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.261310 24030 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.261950 24031 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.262463 24002 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.264276 24031 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.264787 24030 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.064414 22850 tablet_copy_source_session.cc:215] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:11:36.083935 22804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.915s	user 0.009s	sys 0.023s
W20260709 13:11:36.265663 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.616s	user 0.013s	sys 0.018s
W20260709 13:11:36.266129 22804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.100s	user 0.010s	sys 0.023s
W20260709 13:11:36.102284 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.036s	sys 0.063s
I20260709 13:11:36.267079 22850 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:46858 (request call id 37) took 2404 ms. Trace:
W20260709 13:11:36.267385 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.036s	sys 0.063s
I20260709 13:11:36.267506 22850 rpcz_store.cc:276] 0709 13:11:33.863191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.863402 (+   211us) service_pool.cc:224] Handling call
0709 13:11:36.266942 (+2403540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.103708 24032 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.110335 24004 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.118464 23993 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.122917 24003 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.122814 24033 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.269024 24032 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.269869 24033 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.271102 24015 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.271610 24032 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.271682 24015 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.272323 24032 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.128438 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.324s	user 0.072s	sys 0.130s
W20260709 13:11:36.274164 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.470s	user 0.073s	sys 0.130s
I20260709 13:11:36.274423 24015 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:36.274587 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 3.47062e+06us
I20260709 13:11:36.275038 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:36.275758 23462 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:36.132995 24013 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.274626 23460 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:36.279464 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:36.133437 24034 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.138409 24035 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:36.139482 24036 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.138530 23961 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.140341 22878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.758s	user 0.056s	sys 0.084s
I20260709 13:11:36.140652 24037 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.153831 22880 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.605s (SENT) [suppressed 4 similar messages]
I20260709 13:11:36.157109 24038 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.163688 24040 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.172267 24039 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.176178 22730 connection.cc:256] RPC call timeout handler was delayed by -2.8440140119998887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:11:36.177057 23431 connection.cc:256] RPC call timeout handler was delayed by -1.3111856409999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:11:36.177081 24042 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.178006 24041 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.186060 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.797s	user 0.072s	sys 0.073s
W20260709 13:11:36.213559 22806 connection.cc:256] RPC call timeout handler was delayed by -1.7233053119999795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:11:36.308107 22806 connection.cc:256] RPC call timeout handler was delayed by -1.6909053119999271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:11:36.308722 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4398053119999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
W20260709 13:11:36.309053 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3174053119998916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:11:36.309330 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2503590399999212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
I20260709 13:11:36.217943 22664 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 240) took 5397 ms. Trace:
W20260709 13:11:36.309631 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2116590399999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 13:11:36.220875 22660 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 260) took 4314 ms. Trace:
I20260709 13:11:36.243506 23981 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.243979 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:36.244663 23982 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.247735 23924 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.249940 23979 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.253870 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:36.307489 24037 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:36.265393 23487 debug-util.cc:398] Leaking SignalData structure 0x7b080034c6c0 after lost signal to thread 22866
I20260709 13:11:36.268427 23990 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.310554 22660 rpcz_store.cc:276] 0709 13:11:31.906307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.912964 (+  6657us) service_pool.cc:224] Handling call
0709 13:11:36.220641 (+4307677us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:11:36.280521 24035 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.272990 24032 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.297667 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:36.326828 23801 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.283515 24015 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.289729 24033 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.298806 24030 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.299799 24031 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.301041 23886 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.280419 24034 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.295737 24036 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:36.344446 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 190529us
W20260709 13:11:36.305027 22878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.980s	user 0.057s	sys 0.084s
W20260709 13:11:36.214432 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 477703us
I20260709 13:11:36.316013 23463 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
W20260709 13:11:35.833123 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800098120 after lost signal to thread 21636
I20260709 13:11:36.318226 24044 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.353220 22807 connection.cc:256] RPC call timeout handler was delayed by -2.0037297220000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 13:11:36.327579 23801 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:36.333840 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 2.44579e+06us
I20260709 13:11:36.301204 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:36.318178 24049 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:36.320379 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.932s	user 0.073s	sys 0.073s
W20260709 13:11:36.358286 22807 connection.cc:256] RPC call timeout handler was delayed by -1.8057297220000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:11:36.333527 24015 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.338083 23960 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:36.354873 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800172900 after lost signal to thread 21639
I20260709 13:11:36.320627 23982 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.324363 23979 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.383276 24053 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.327054 23981 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.341301 24033 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.380259 23460 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:36.357201 24031 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.324730 24042 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.388237 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:36.378288 23487 debug-util.cc:398] Leaking SignalData structure 0x7b080030e360 after lost signal to thread 22939
I20260709 13:11:36.384634 23985 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.322197 23961 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:36.394150 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72219.8us
I20260709 13:11:36.380761 24015 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.358742 23886 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.380385 23924 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.412492 22914 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:36.353799 24030 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.322230 24039 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.384199 23957 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.345840 23990 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:36.434703 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96998us
I20260709 13:11:36.309723 22664 rpcz_store.cc:276] 0709 13:11:30.820604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.876850 (+ 56246us) service_pool.cc:224] Handling call
0709 13:11:36.217806 (+5340956us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:31.917298 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:31.935682 (+ 18384us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:31.935738 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:11:31.935823 (+    85us) tablet.cc:662] Decoding operations
0709 13:11:31.936333 (+   510us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:31.936449 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:31.936486 (+    37us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:31.936797 (+   311us) tablet.cc:701] Row locks acquired
0709 13:11:31.936858 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:11:31.937084 (+   226us) write_op.cc:270] Start()
0709 13:11:31.937256 (+   172us) write_op.cc:276] Timestamp: P: 1783602691937002 usec, L: 0
0709 13:11:32.618664 (+681408us) op_driver.cc:348] REPLICATION: starting
0709 13:11:32.690314 (+ 71650us) log.cc:844] Serialized 591 byte log entry
0709 13:11:34.777343 (+2087029us) op_driver.cc:464] REPLICATION: finished
0709 13:11:34.830755 (+ 53412us) write_op.cc:301] APPLY: starting
0709 13:11:34.831077 (+   322us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:34.831219 (+   142us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:34.831384 (+   165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:34.832057 (+   673us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:34.832626 (+   569us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:34.832699 (+    73us) write_op.cc:312] APPLY: finished
0709 13:11:34.833161 (+   462us) log.cc:844] Serialized 23 byte log entry
0709 13:11:34.833436 (+   275us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:34.833906 (+   470us) write_op.cc:454] Released schema lock
0709 13:11:34.833967 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:34.834392 (+   425us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:34.834433 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":52677,"apply.run_cpu_time_us":2415,"apply.run_wall_time_us":3987,"num_ops":1,"prepare.queue_time_us":55815,"prepare.run_cpu_time_us":174395,"prepare.run_wall_time_us":1192022,"raft.queue_time_us":985601,"raft.run_cpu_time_us":13344,"raft.run_wall_time_us":46919,"replication_time_us":2158494,"thread_start_us":2912077,"threads_started":4,"wal-append.queue_time_us":461811}]]}
I20260709 13:11:36.354827 24038 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.321784 24040 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:36.449057 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 151489us
I20260709 13:11:36.442044 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:36.392107 24044 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:36.438634 22787 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9800 after lost signal to thread 22698
I20260709 13:11:36.359473 24041 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.413117 24053 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.418110 23960 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.469744 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.471153 23801 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.417276 24033 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.476231 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.479883 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:36.466420 22787 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9640 after lost signal to thread 22712
W20260709 13:11:36.471035 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb8c0 after lost signal to thread 23490
I20260709 13:11:36.487996 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.488605 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.488767 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.448925 23924 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.445046 24051 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.477020 23801 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:11:36.450862 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:36.505679 22841 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:36.530947 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.531936 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:36.540110 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 100358us
I20260709 13:11:36.490275 24047 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.532171 22913 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:36.477029 24053 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.454761 23985 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.538174 24051 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.539423 24054 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.461079 24052 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.481040 24046 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.539867 22910 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:36.455449 23957 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.541922 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.503796 24048 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.582060 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.614651 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.388998 24049 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.541428 24043 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.541396 24047 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.617609 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.535988 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.638882 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.662513 24057 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.663552 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.654469 24059 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.617022 24035 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.679934 23961 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.680780 23961 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.681248 23961 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:36.659519 22787 debug-util.cc:398] Leaking SignalData structure 0x7b080030e0a0 after lost signal to thread 22790
I20260709 13:11:36.655282 24032 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:36.682726 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 200980us
I20260709 13:11:36.658175 24052 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:36.740942 22787 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8340 after lost signal to thread 22791
I20260709 13:11:36.654542 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.826414 22832 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:36.640552 24054 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.830325 23886 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.830991 23957 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.831136 23886 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:36.684342 23981 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.740650 23961 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.663724 24039 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.839648 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.840056 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.654973 24058 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.655438 24043 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:36.810408 24032 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:36.810308 24057 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:36.866189 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:36.856841 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800321be0 after lost signal to thread 22865
I20260709 13:11:36.872164 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.879719 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.883675 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.884785 24038 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.853118 24043 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:36.869335 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800322e20 after lost signal to thread 22866
I20260709 13:11:36.859618 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.895785 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.876713 23981 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:36.724472 24048 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.655094 24046 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.841421 24058 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.851536 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:11:36.877594 24033 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.818426 24059 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:36.905978 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.907269 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:36.913537 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.915573 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:36.874943 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:36.900687 23990 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.902431 23960 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.907491 24037 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.915455 24042 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.932996 24042 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.933336 24057 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.933621 24042 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.905725 22909 raft_consensus.cc:606] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:11:36.942035 24052 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.905126 23924 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:36.942281 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.903986 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.917721 24039 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.907616 24051 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.915225 23985 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.913954 24049 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.902114 24032 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:36.945645 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800267780 after lost signal to thread 22939
I20260709 13:11:36.925024 24034 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.943478 24042 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.949081 24039 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:36.950344 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.957669 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.962435 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:36.902549 23981 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:36.963673 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:36.917838 24054 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:36.946445 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:36.961311 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800341120 after lost signal to thread 22940
I20260709 13:11:36.953192 24058 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:36.983304 22787 debug-util.cc:398] Leaking SignalData structure 0x7b08003094e0 after lost signal to thread 23490
W20260709 13:11:36.983750 22787 debug-util.cc:398] Leaking SignalData structure 0x7b0800313ac0 after lost signal to thread 23491
I20260709 13:11:36.985229 23982 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:36.985795 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:36.993556 24047 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:36.996477 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.031s	sys 0.064s
W20260709 13:11:37.019157 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.033s	sys 0.067s
I20260709 13:11:37.020150 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.020984 22834 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:37.021189 24059 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.028903 24048 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.034801 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.035048 24041 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:37.019829 22732 connection.cc:256] RPC call timeout handler was delayed by -2.2661852789999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 13:11:37.029182 24036 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:37.000535 23431 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.085s	sys 0.139s
W20260709 13:11:37.049852 23431 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.085s	sys 0.139s
W20260709 13:11:37.050416 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 743254us
I20260709 13:11:36.964819 24039 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:37.000765 22805 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.022s	sys 0.044s
W20260709 13:11:37.055202 22805 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.026s	sys 0.044s
W20260709 13:11:37.056104 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 593565us
I20260709 13:11:37.071663 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.080598 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.083666 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.085841 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.085639 24042 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.086535 22765 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.089648 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.091140 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.091430 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.086390 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.049015 23997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f666ea4542d4951919f36de0b43b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:37.098922 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.110726 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.121116 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.122550 22765 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.123828 22850 tablet_copy_service.cc:140] P 776baf137f8745dd866379ca15ab8cee: Received BeginTabletCopySession request for tablet af39123de9434cc0b451f4b03d305310 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:46858)
I20260709 13:11:37.124078 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.124469 22850 tablet_copy_service.cc:161] P 776baf137f8745dd866379ca15ab8cee: Beginning new tablet copy session on tablet af39123de9434cc0b451f4b03d305310 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:46858: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-af39123de9434cc0b451f4b03d305310
I20260709 13:11:37.124779 22761 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.116392 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.123863 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.127259 22914 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:37.127382 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.128769 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.100593 24042 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.112226 24048 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.137092 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.139451 22837 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:37.110065 24043 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.148335 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.149271 22837 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.155052 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.155758 23461 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.157644 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.113904 23981 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.158862 23981 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.159039 24043 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.162789 24032 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.164049 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.164790 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.165994 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.140964 24048 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:37.167160 24048 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.167496 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.168372 22837 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.169013 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.170162 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.170490 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.170833 22765 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.169350 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.144937 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.197956 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.199126 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
W20260709 13:11:37.201274 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.007s	sys 0.035s
W20260709 13:11:37.201957 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.007s	sys 0.035s
W20260709 13:11:37.202338 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 748029us
W20260709 13:11:37.204037 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 396562us
W20260709 13:11:37.198948 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 368415us
W20260709 13:11:37.207798 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 292350us
I20260709 13:11:37.204577 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.224633 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.242266 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.213339 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.209764 22832 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:37.244066 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:37.224861 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.229574 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.249802 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:37.250272 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 119109us
I20260709 13:11:37.218168 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.251874 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.254739 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:37.263208 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.247725 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:37.244597 24053 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.009s
I20260709 13:11:37.244773 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.268940 24046 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.270109 24030 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.272802 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.248253 22850 tablet_copy_source_session.cc:215] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:37.262832 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.252311 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:37.368162 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:37.359812 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 97109.1us
I20260709 13:11:37.369050 22765 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.266727 22911 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.373855 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.374625 22757 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:37.378121 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.378860 22762 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:37.379246 23924 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.390417 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.390848 23924 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:37.379205 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.399827 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.400426 22766 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.393586 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.393932 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.394457 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.379223 23459 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:37.396149 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.397037 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.411638 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.395366 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.379276 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.413290 22759 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.416195 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:37.391799 24015 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.939s	user 0.017s	sys 0.018s
I20260709 13:11:37.421799 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.424861 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.425654 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.426385 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.426537 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:37.299155 24053 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.401315 23924 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.434859 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.435046 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.435686 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.440119 22766 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:37.466229 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.488991 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.458500 23979 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.436915 22760 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.542950 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.543452 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:37.432893 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.544234 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:37.549777 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68686us
I20260709 13:11:37.550518 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:37.572201 22837 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.550751 22911 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:37.552843 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.482460 24069 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.553102 23924 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:37.555310 23961 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.697s	user 0.005s	sys 0.009s
I20260709 13:11:37.558372 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.558221 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.551841 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:37.559708 24048 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.541983 22838 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:37.541658 24056 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:37.565725 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 153142us
I20260709 13:11:37.478761 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:37.482733 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56309.7us
I20260709 13:11:37.432866 24015 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.576088 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.582178 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.582441 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:37.583734 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.584444 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.585947 24070 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.587486 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.587836 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.586429 24070 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:37.598664 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.599484 22759 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.600528 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.601770 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.603148 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.604837 23924 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:37.605846 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.609409 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.610143 22765 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.611757 22836 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.613828 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.616814 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.617996 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.624739 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.625754 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.598690 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.627902 22841 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:37.580628 23961 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.598686 24070 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.631356 23961 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:37.586374 24044 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.635010 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.574035 24039 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.617266 24053 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:37.615882 24069 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.632085 24072 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.657433 24072 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:37.658130 24072 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.631705 23924 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.638765 22758 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.646461 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.647718 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.648837 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.661804 23924 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:37.660566 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.667696 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.644980 24069 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:37.642832 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58565.1us
I20260709 13:11:37.665207 23924 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.660446 24074 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.678079 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:37.678622 24014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af39123de9434cc0b451f4b03d305310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:37.679721 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.666675 24073 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.674192 24056 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.682435 24056 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:37.682348 24073 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:37.682991 24056 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.683104 24073 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.684096 22837 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
W20260709 13:11:37.686075 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57192us
I20260709 13:11:37.674120 24069 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.683537 24076 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.681084 24074 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:37.701076 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:37.699810 24074 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.702754 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.704183 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:37.705024 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:37.711581 23981 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 13:11:37.712936 22764 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:37.716109 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.718838 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:37.719509 22832 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.713183 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:37.724103 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:37.738912 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:37.742765 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:37.745857 24068 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.726698 24048 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:37.757108 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.757928 23465 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:37.762476 22664 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 238) took 1277 ms. Trace:
I20260709 13:11:37.762970 22664 rpcz_store.cc:276] 0709 13:11:36.485593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.491451 (+  5858us) service_pool.cc:224] Handling call
0709 13:11:37.762355 (+1270904us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:11:37.777853 24039 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.789423 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:37.790535 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.791141 22907 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.792618 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:37.794560 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:37.795271 23462 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:37.764914 24074 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:37.749259 24073 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:37.792970 24070 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:37.800984 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58338us
I20260709 13:11:37.814971 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:37.816396 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:37.815078 24070 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:37.813069 24073 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:37.817543 24039 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.818382 24039 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:37.818842 24039 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:37.766070 24076 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.821192 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:37.822588 24039 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:37.823621 24074 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:37.824098 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:37.831449 24072 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:37.837225 24072 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:37.837764 24076 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:37.838361 24076 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:37.839377 24076 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:38.844043 24079 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:11:38.866171 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800197940 after lost signal to thread 21636
W20260709 13:11:38.866811 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800197980 after lost signal to thread 21639
I20260709 13:11:37.825618 24069 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:39.536844 24069 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:39.715487 22907 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.924s	user 0.018s	sys 0.010s
I20260709 13:11:39.716125 22907 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
W20260709 13:11:39.717031 22907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 47) took 1928 ms (client timeout 2153 ms). Trace:
W20260709 13:11:39.717710 22907 rpcz_store.cc:269] 0709 13:11:37.788361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.789000 (+   639us) service_pool.cc:224] Handling call
0709 13:11:39.716919 (+1927919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:40.008195 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1694) took 2308 ms. Trace:
I20260709 13:11:40.008829 22911 rpcz_store.cc:276] 0709 13:11:37.699628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.700592 (+   964us) service_pool.cc:224] Handling call
0709 13:11:37.701224 (+   632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:37.701654 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:37.701690 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:37.710445 (+  8755us) log.cc:844] Serialized 492 byte log entry
0709 13:11:37.753231 (+ 42786us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:37.769535 (+ 16304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.800039 (+ 30504us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.007748 (+2207709us) raft_consensus.cc:1685] finished
0709 13:11:40.007772 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.008062 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17664,"thread_start_us":15427,"threads_started":1,"wal-append.queue_time_us":534483,"wal-append.run_cpu_time_us":3791,"wal-append.run_wall_time_us":1198389}
W20260709 13:11:40.013870 22880 connection.cc:256] RPC call timeout handler was delayed by -2.1885546949999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
W20260709 13:11:40.014696 22880 connection.cc:256] RPC call timeout handler was delayed by -2.1378146389999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 13:11:40.015041 22880 connection.cc:256] RPC call timeout handler was delayed by -2.1225146389999736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 13:11:40.016727 22880 connection.cc:256] RPC call timeout handler was delayed by -1.9110146389999727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 13:11:40.017156 22880 connection.cc:256] RPC call timeout handler was delayed by -1.7268546949999291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:11:40.017545 22880 connection.cc:256] RPC call timeout handler was delayed by -1.7130431759999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 13:11:40.017887 22880 connection.cc:256] RPC call timeout handler was delayed by -1.6851431759999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 13:11:40.018221 22880 connection.cc:256] RPC call timeout handler was delayed by -1.6068431759999839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:11:40.018567 22880 connection.cc:256] RPC call timeout handler was delayed by -1.3863431759999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 13:11:40.018951 22880 connection.cc:256] RPC call timeout handler was delayed by -1.2819431759999134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
W20260709 13:11:40.019284 22880 connection.cc:256] RPC call timeout handler was delayed by -1.2369431759999543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 13:11:40.021102 24085 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.022065 24086 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.021867 24085 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.022644 24086 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.024187 24085 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:40.024750 24085 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.025316 24086 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:40.025249 24085 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:40.028179 24085 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:40.032665 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.317s	user 0.033s	sys 0.066s
W20260709 13:11:40.033209 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.319s	user 0.033s	sys 0.066s
W20260709 13:11:40.033464 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 2.31917e+06us
I20260709 13:11:40.035038 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.036594 24086 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.037348 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.037817 24086 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:40.038607 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:40.039460 24086 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.039803 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.040292 22907 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.040784 24086 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:40.041479 24086 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:40.041853 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.042464 22907 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.042882 24086 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.043761 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.044409 22907 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.044412 24086 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.045583 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.046535 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.061376 22915 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.062783 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.063552 22915 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.064597 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.066129 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.067040 22915 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.068002 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.068877 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.045225 24085 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.048241 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.071231 22907 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.072295 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.072798 22907 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:40.051420 23924 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 2.378s	user 0.017s	sys 0.033s
I20260709 13:11:40.073377 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.073948 24085 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.074193 22907 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.075253 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.075459 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:40.076041 22805 connection.cc:256] RPC call timeout handler was delayed by -1.2259597209999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 13:11:40.076514 22805 connection.cc:256] RPC call timeout handler was delayed by -1.1404597209999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
I20260709 13:11:40.076546 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:46252
W20260709 13:11:40.076939 22805 connection.cc:256] RPC call timeout handler was delayed by -1.0182008579998865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
I20260709 13:11:40.070339 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.077315 22907 raft_consensus.cc:606] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Received request to transfer leadership to TS 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:40.077796 22915 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:40.074393 24085 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.080137 22912 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.076644 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.081872 22911 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.082789 23924 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:40.084093 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 36) took 2856 ms (client timeout 1749 ms). Trace:
W20260709 13:11:40.084717 22764 rpcz_store.cc:269] 0709 13:11:37.227336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.227521 (+   185us) service_pool.cc:224] Handling call
0709 13:11:40.083854 (+2856333us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:40.083977 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033095552}
I20260709 13:11:40.086598 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.087309 22837 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.088302 22909 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.095408 24085 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.097870 24085 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:40.099037 22837 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:40.101147 24031 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.103775 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 800) took 3292 ms. Trace:
I20260709 13:11:40.104362 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.104239 22758 rpcz_store.cc:276] 0709 13:11:36.811180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.817411 (+  6231us) service_pool.cc:224] Handling call
0709 13:11:36.831172 (+ 13761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.665203 (+834031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.103459 (+2438256us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400369054. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:40.103664 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840935808}
I20260709 13:11:40.105073 22914 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:40.114174 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 43) took 2981 ms (client timeout 1546 ms). Trace:
W20260709 13:11:40.114822 22838 rpcz_store.cc:269] 0709 13:11:37.132945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.134476 (+  1531us) service_pool.cc:224] Handling call
0709 13:11:40.113842 (+2979366us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:40.114048 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742762752}
I20260709 13:11:40.115423 24042 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.116060 24042 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.118983 24042 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.119635 24042 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.120153 24042 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:40.120605 24056 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 2.437s	user 0.012s	sys 0.024s
I20260709 13:11:40.122205 24056 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.127566 24056 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.131196 24042 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.140898 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.143675 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.144474 22913 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:40.145336 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1645) took 2417 ms (client timeout 2073 ms). Trace:
W20260709 13:11:40.145802 22758 rpcz_store.cc:269] 0709 13:11:37.729756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.145165 (+2415409us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:40.145267 (+   102us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:11:40.146921 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.147442 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.148252 22765 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.149652 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.150264 22758 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:40.150833 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:40.152734 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:40.153887 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.154443 22765 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:40.157823 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.158270 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.158565 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.158857 22764 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.159392 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.159679 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.159961 22766 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:40.160226 22764 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:40.160729 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.160989 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.161271 22766 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.162079 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.162130 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.162698 22764 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.162724 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.163128 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.163635 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.164220 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.171129 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 298) took 2381 ms. Trace:
I20260709 13:11:40.171617 23460 rpcz_store.cc:276] 0709 13:11:37.789699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.797788 (+  8089us) service_pool.cc:224] Handling call
0709 13:11:37.802342 (+  4554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.833838 (+ 31496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:37.833878 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:37.833940 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:37.833972 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.170843 (+2336871us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b540039e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:40.170908 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.171035 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71775488}
I20260709 13:11:40.174467 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.175139 22914 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.175941 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.176496 22914 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.176585 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.177299 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.177743 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:40.177910 22914 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.178864 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.179090 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.180071 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.180709 22761 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.180742 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.182413 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.182988 22914 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.184566 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:40.185034 22732 connection.cc:256] RPC call timeout handler was delayed by -3.0803096969999615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 13:11:40.183027 22757 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:40.186578 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.186688 24085 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:40.187069 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.187459 24085 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.187728 22758 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:40.185214 22913 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.190359 24085 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:40.186683 23430 connection.cc:256] RPC call timeout handler was delayed by -1.9448966840000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 13:11:40.192389 23430 connection.cc:256] RPC call timeout handler was delayed by -1.8170966840000347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
I20260709 13:11:40.195114 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:40.196157 24092 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.196530 24092 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:40.186625 24043 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.196873 24092 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:40.192776 23430 connection.cc:256] RPC call timeout handler was delayed by -1.3985966840000401s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
I20260709 13:11:40.197219 24043 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.198347 24088 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.198833 24088 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.200529 24039 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.201423 24039 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.203373 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.203975 24039 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.206032 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.207095 24085 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.207437 24088 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.208139 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.209089 24085 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.209487 24085 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.209853 24085 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.214211 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.211822 24092 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.217274 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.218288 22758 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:40.220374 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.218204 24092 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:40.221786 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.222436 22759 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:40.223640 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.223939 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.224370 22766 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:40.206485 24043 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:40.226786 24085 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:40.230229 22841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 39) took 3125 ms (client timeout 1688 ms). Trace:
W20260709 13:11:40.230724 22841 rpcz_store.cc:269] 0709 13:11:37.104703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.104835 (+   132us) service_pool.cc:224] Handling call
0709 13:11:40.230026 (+3125191us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b540019f0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:40.230155 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173578112}
W20260709 13:11:40.232678 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 2.49022e+06us
I20260709 13:11:40.235358 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.237856 24088 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.238025 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 851) took 3075 ms. Trace:
I20260709 13:11:40.238425 24088 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.238399 22836 rpcz_store.cc:276] 0709 13:11:37.163109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.165893 (+  2784us) service_pool.cc:224] Handling call
0709 13:11:37.166252 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.612354 (+446102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.237709 (+2625355us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:40.237928 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023970560}
I20260709 13:11:40.240760 24088 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.241336 24088 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.230139 24085 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:40.246361 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.247092 22910 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.249744 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:40.250241 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.249773 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.250926 22761 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:40.251345 22762 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
W20260709 13:11:40.251748 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 157216us
I20260709 13:11:40.252311 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.252727 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.253330 22757 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:40.255841 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.257258 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.259732 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43708 (request call id 275) took 7130 ms. Trace:
I20260709 13:11:40.260239 24014 rpcz_store.cc:276] 0709 13:11:33.129283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.129492 (+   209us) service_pool.cc:224] Handling call
0709 13:11:33.736016 (+606524us) ts_tablet_manager.cc:934] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
0709 13:11:37.676731 (+3940715us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:11:40.227029 (+2550298us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:40.227154 (+   125us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:11:40.258674 (+ 31520us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:11:40.259122 (+   448us) tablet_replica.cc:206] Creating consensus instance
0709 13:11:40.259538 (+   416us) ts_tablet_manager.cc:989] Replica registered.
0709 13:11:40.259617 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":315,"tablet-copy.queue_time_us":328474,"thread_start_us":2647124,"threads_started":2}
W20260709 13:11:40.269165 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.496s	user 0.016s	sys 0.024s
W20260709 13:11:40.269837 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.497s	user 0.017s	sys 0.024s
I20260709 13:11:40.275261 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:40.276283 24036 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.276696 24036 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:40.277072 24036 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.278692 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.456s	user 0.078s	sys 0.196s
W20260709 13:11:40.279194 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.457s	user 0.078s	sys 0.196s
W20260709 13:11:40.279507 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 2.45834e+06us
I20260709 13:11:40.261107 23924 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:40.283788 22695 debug-util.cc:398] Leaking SignalData structure 0x7b08000345e0 after lost signal to thread 22698
W20260709 13:11:40.285739 22732 connection.cc:256] RPC call timeout handler was delayed by -2.66990969699998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
W20260709 13:11:40.286264 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7304897640000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:11:40.286571 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7304897640000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:11:40.286845 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6989897639999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 13:11:40.287117 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6269897639999726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 13:11:40.287395 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5954897640000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:11:40.287658 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5783897640000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:11:40.287917 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5567897639999728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 13:11:40.288180 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5225897640000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:11:40.288444 22732 connection.cc:256] RPC call timeout handler was delayed by -1.486589763999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 13:11:40.288710 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4811897639999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:11:40.288957 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4802897639999628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
I20260709 13:11:40.291514 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:40.289152 22664 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:11:40.289214 22732 connection.cc:256] RPC call timeout handler was delayed by -1.45148976400003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 13:11:40.292646 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4370897640000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 13:11:40.292946 22732 connection.cc:256] RPC call timeout handler was delayed by -1.37498976400002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 13:11:40.293190 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3641897640000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:11:40.293455 22732 connection.cc:256] RPC call timeout handler was delayed by -1.200389763999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
W20260709 13:11:40.293740 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1976897639999606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 13:11:40.293999 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1490897639999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
W20260709 13:11:40.294255 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1391897639999797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 13:11:40.294516 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1373897639999768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 13:11:40.294780 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1058897640000396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260709 13:11:40.297197 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:40.301263 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:40.302536 24088 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:40.303117 24088 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:40.303984 24088 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.307159 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.307981 23460 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.311386 24015 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:40.312422 23997 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 2.564s	user 0.017s	sys 0.019s
I20260709 13:11:40.312677 24031 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.313046 24031 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:40.313413 24031 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.313721 23997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43708 (request call id 272) took 7801 ms. Trace:
I20260709 13:11:40.314146 23997 rpcz_store.cc:276] 0709 13:11:32.511909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.584356 (+ 72447us) service_pool.cc:224] Handling call
0709 13:11:33.122465 (+538109us) ts_tablet_manager.cc:934] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
0709 13:11:36.837032 (+3714567us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:11:37.747729 (+910697us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:37.747856 (+   127us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:11:40.312857 (+2565001us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:11:40.313208 (+   351us) tablet_replica.cc:206] Creating consensus instance
0709 13:11:40.313575 (+   367us) ts_tablet_manager.cc:989] Replica registered.
0709 13:11:40.313656 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":336,"tablet-copy.queue_time_us":121873,"thread_start_us":313938,"threads_started":2}
I20260709 13:11:40.315471 23997 tablet_copy_client.cc:806] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
W20260709 13:11:40.315980 22695 debug-util.cc:398] Leaking SignalData structure 0x7b08000346a0 after lost signal to thread 22712
I20260709 13:11:40.317422 23997 tablet_copy_client.cc:670] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:40.319412 24040 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.319778 24036 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.320715 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.322101 24036 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.325335 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.324333 22664 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:40.336210 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:40.338433 23462 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:40.329243 22695 debug-util.cc:398] Leaking SignalData structure 0x7b08002fccc0 after lost signal to thread 22790
W20260709 13:11:40.346268 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800320f00 after lost signal to thread 22791
I20260709 13:11:40.346005 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:40.347610 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800329640 after lost signal to thread 22865
I20260709 13:11:40.346015 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:40.348618 23465 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:40.349038 22695 debug-util.cc:398] Leaking SignalData structure 0x7b08003668a0 after lost signal to thread 22939
I20260709 13:11:40.350652 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:40.352195 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:40.352886 23461 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.335803 24040 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:40.349357 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800366960 after lost signal to thread 22940
W20260709 13:11:40.355373 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 145947us
I20260709 13:11:40.361303 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.361383 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.365077 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.365406 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.368286 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.377490 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:40.381201 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 91366.1us
I20260709 13:11:40.372680 23886 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.384507 23886 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.384898 23886 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.385627 22805 leader_election.cc:336] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.668s (SENT)
I20260709 13:11:40.355589 24014 tablet_copy_client.cc:806] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:40.393242 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.391196 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.395730 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:40.339408 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:40.398051 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68491.6us
I20260709 13:11:40.398725 24070 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:40.372695 24090 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.399475 24070 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:40.400509 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.402644 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.386302 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:40.403895 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.372434 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.393993 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:40.407316 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:40.408589 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 198719us
W20260709 13:11:40.411227 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48412us
I20260709 13:11:40.412222 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:40.390769 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119446us
I20260709 13:11:40.413076 22760 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:40.413867 24003 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.371325 24040 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:11:40.405691 24036 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.413856 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.408293 24014 tablet_copy_client.cc:670] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:40.400736 24070 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.399947 24090 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.405295 24056 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:40.414453 24003 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:40.410769 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800375200 after lost signal to thread 23490
I20260709 13:11:40.420552 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.422787 24065 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:40.442886 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74709.3us
I20260709 13:11:40.423596 24031 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:40.414638 22832 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 2.687s	user 0.027s	sys 0.018s
I20260709 13:11:40.418371 24094 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:40.443701 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:40.444746 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 62574.1us
I20260709 13:11:40.448403 23801 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:40.445897 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.442653 24003 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.446517 22832 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:40.421541 24036 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.466347 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.467157 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.445436 24056 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.467824 22836 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
W20260709 13:11:40.417622 23431 connection.cc:256] RPC call timeout handler was delayed by -2.3101238609999655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 13:11:40.468842 23431 connection.cc:256] RPC call timeout handler was delayed by -1.8278221070000882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 13:11:40.469281 23431 connection.cc:256] RPC call timeout handler was delayed by -1.7414221070000622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
I20260709 13:11:40.465553 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:40.469647 23431 connection.cc:256] RPC call timeout handler was delayed by -1.5641221070001166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 13:11:40.470125 23431 connection.cc:256] RPC call timeout handler was delayed by -1.5233225289999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:11:40.470505 23431 connection.cc:256] RPC call timeout handler was delayed by -1.5206225289999793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
I20260709 13:11:40.470671 24056 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.470995 24087 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.470125 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.466504 24036 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.471982 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:40.440819 24090 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.470391 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.481552 22833 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.481814 23461 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.485371 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5612915000000385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
I20260709 13:11:40.485631 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:40.485826 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4244915000000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 13:11:40.486238 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3758914999999661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:11:40.486582 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0246778509999785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 13:11:40.486887 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.490274 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.490859 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.492331 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.493757 22837 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.494459 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.531s (SENT)
I20260709 13:11:40.494861 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.491621 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.496232 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.498298 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.475339 23990 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:40.470956 23431 connection.cc:256] RPC call timeout handler was delayed by -1.490922529000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:11:40.500828 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.489331 23924 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:11:40.506105 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.469569 24094 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:40.482709 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 868) took 2775 ms (client timeout 1554 ms). Trace:
I20260709 13:11:40.491305 24065 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.492725 24087 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.516687 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
W20260709 13:11:40.508829 22806 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.567s (SENT)
I20260709 13:11:40.520900 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.501411 23985 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.491633 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.511439 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.501618 24085 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.526666 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:40.523690 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80084us
I20260709 13:11:40.528075 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:40.528700 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.529317 22837 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.500737 23431 connection.cc:256] RPC call timeout handler was delayed by -1.31632252899999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
I20260709 13:11:40.534447 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.534943 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.525651 22841 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
W20260709 13:11:40.535183 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.579s (SENT)
W20260709 13:11:40.521452 22805 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 2.249s (SENT)
I20260709 13:11:40.537099 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:40.537966 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:40.540259 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.622s (SENT)
I20260709 13:11:40.540817 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.526115 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.504217 23990 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.575373 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.553238 24086 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.516153 24106 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.523880 23985 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.592670 23985 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.536336 24031 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.552680 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:40.547428 22834 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.536059 23431 connection.cc:256] RPC call timeout handler was delayed by -1.3028225289999682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
I20260709 13:11:40.544437 22838 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.566718 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:40.570181 22806 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.698s (SENT)
I20260709 13:11:40.607683 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.563771 24040 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:40.543290 24103 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:40.609845 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.846s (SENT)
I20260709 13:11:40.610111 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.610445 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.565223 24107 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.572158 24108 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.553597 24015 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.573673 24096 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:40.573570 23460 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:40.553013 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 143477us
I20260709 13:11:40.576320 22837 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:40.585132 22835 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.592144 23990 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.595659 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:40.625983 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:40.517753 22832 rpcz_store.cc:269] 0709 13:11:37.707630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.718548 (+ 10918us) service_pool.cc:224] Handling call
0709 13:11:40.481973 (+2763425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:40.544486 24105 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.564826 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.611652 24040 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:40.629415 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.630461 23462 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:40.544297 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:40.612375 22806 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.866s (SENT)
I20260709 13:11:40.607468 22841 raft_consensus.cc:2376] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2. Re-sending same reply.
I20260709 13:11:40.613812 24107 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:40.619580 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 91600.6us
I20260709 13:11:40.594816 24086 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.622071 23465 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:40.604080 23431 connection.cc:256] RPC call timeout handler was delayed by -1.2077221070001087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260709 13:11:40.634048 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:40.640820 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.982s (SENT)
I20260709 13:11:40.614614 24053 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.614746 24108 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.637300 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:40.638069 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:40.641264 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.621765 24104 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:40.667946 23801 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.668538 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:40.672964 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.073s (SENT)
W20260709 13:11:40.674957 22806 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.170s (SENT)
W20260709 13:11:40.639068 23431 connection.cc:256] RPC call timeout handler was delayed by -1.1537478610000562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 13:11:40.665719 24088 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:40.676225 23431 connection.cc:256] RPC call timeout handler was delayed by -1.1300225290000299s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
I20260709 13:11:40.676468 24088 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:40.667044 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 132864us
W20260709 13:11:40.677685 22806 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.231s (SENT)
I20260709 13:11:40.665972 24108 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.690518 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.690532 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.691617 23462 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.641990 24107 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.636415 24085 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:40.639267 24086 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.663113 24092 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:40.693779 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67902.5us
W20260709 13:11:40.697904 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 190910us
I20260709 13:11:40.699142 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:40.689962 24088 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.700122 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:40.674646 24066 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.702477 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.702626 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.703107 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.703897 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.705554 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.667215 24036 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.708935 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.709070 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.710872 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.709530 24116 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.713805 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.696295 24092 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:40.700702 24103 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.715643 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.723408 22837 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:40.696468 24113 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.714619 23924 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.725668 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.726575 22837 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:40.736011 24107 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.738855 24107 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.748543 23997 tablet_copy_client.cc:538] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:40.749256 24014 tablet_copy_client.cc:538] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:40.712179 24065 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.710078 24085 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:40.718252 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:40.767421 23460 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:40.700428 24040 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.722150 24102 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.769953 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:40.781608 24112 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:11:40.781170 24085 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.762408 23461 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:40.722365 24116 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:40.782935 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72336.6us
I20260709 13:11:40.723737 24092 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.790477 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.797715 22833 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:40.760984 24065 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.809360 24065 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:40.806092 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.504s (SENT)
I20260709 13:11:40.806103 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:11:40.811599 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.507s (SENT)
W20260709 13:11:40.812543 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57856.6us
I20260709 13:11:40.812405 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.782985 23924 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.768868 24114 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.788935 24102 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:40.824690 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.827770 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.820945 24014 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
W20260709 13:11:40.832415 23431 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.715s (SENT)
I20260709 13:11:40.806591 24116 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.804533 24094 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.814455 23460 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.837669 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:40.840816 23431 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.734s (SENT)
I20260709 13:11:40.841197 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.841893 24118 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:40.842224 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.749s (SENT)
W20260709 13:11:40.843920 23431 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.941s (SENT)
W20260709 13:11:40.844951 23431 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.063s (SENT)
W20260709 13:11:40.845762 23431 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.102s (SENT)
I20260709 13:11:40.846326 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.847118 24118 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:40.847489 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.232s (SENT)
I20260709 13:11:40.847900 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:40.848492 24118 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.851365 24065 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.825387 24043 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.830688 24102 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.857493 22909 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.838924 24114 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:40.859783 24036 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.861414 24114 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:40.861996 24087 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:40.862423 24087 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:40.862458 24114 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.863034 23886 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.820984 24086 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.821126 24088 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.853951 24070 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:40.863464 24087 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:40.875818 24087 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.882779 24105 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:40.892554 24090 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.896382 24108 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.899240 24108 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.900079 24116 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:40.876471 23886 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.963792 24086 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:40.971244 24088 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:40.972926 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.976150 24065 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:40.980389 22914 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.987193 22832 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:41.013214 22832 raft_consensus.cc:606] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:40.987468 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.015301 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.016840 24116 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.018432 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:40.999295 22907 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.999882 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.001387 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.002239 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.002787 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.010368 22908 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.988404 24090 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:41.016921 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.027249 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.027356 22841 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:41.029103 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.030439 22841 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:41.035351 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.050038 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:41.050889 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:41.052007 22763 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.040308 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.060844 22762 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:40.996989 23985 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.072003 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.077387 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.078670 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:41.082294 24070 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.088230 22841 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:41.094976 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.095079 24118 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 13:11:41.093891 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 340871us
I20260709 13:11:41.117990 24070 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:11:41.119315 22834 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.009s
I20260709 13:11:41.120229 22834 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:41.122861 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.124267 22907 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:41.119863 24070 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.126328 23997 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:11:41.133916 23462 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:41.135188 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.136293 22841 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:41.145387 24003 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.663s	user 0.007s	sys 0.011s
I20260709 13:11:41.145999 24003 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:41.146273 24118 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:41.147298 23985 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:41.158643 22841 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:41.162426 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.163578 24040 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.164912 23458 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.169411 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:41.182367 22836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:41.183563 22836 raft_consensus.cc:606] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:41.185678 24096 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.187356 22906 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:41.190596 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 384392us
I20260709 13:11:41.192170 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:41.175735 24092 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.195186 22758 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.204253 24132 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.205858 24132 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.206723 24132 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.210373 24092 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:41.210593 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.212128 24136 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.212955 24136 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:41.219367 24103 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:41.220888 24066 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.221405 24066 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:41.221083 24136 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.222790 24066 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.224177 24066 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.224614 24134 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:41.225461 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:41.226567 24090 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.226840 22664 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 129) took 1048 ms. Trace:
I20260709 13:11:41.227949 24090 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:41.228104 24118 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:41.231549 24003 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:41.231629 24118 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:41.233534 24118 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.229967 24090 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.236125 24131 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:41.239842 23886 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.240391 23886 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:41.240746 24090 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.243058 23886 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.236234 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.245285 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:41.247236 24131 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.177683 24102 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.255188 24131 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.255836 24131 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.262717 24134 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.263792 24134 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:41.264353 23886 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.265056 24134 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.268882 24110 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:41.270594 24110 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.271047 24110 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:41.274930 24110 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.278082 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.278635 24110 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.276646 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:41.282523 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.283505 22906 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:41.228029 22664 rpcz_store.cc:276] 0709 13:11:40.178481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.178666 (+   185us) service_pool.cc:224] Handling call
0709 13:11:41.226502 (+1047836us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:40.451886 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:40.569661 (+117775us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:40.574892 (+  5231us) write_op.cc:435] Acquired schema lock
0709 13:11:40.578227 (+  3335us) tablet.cc:662] Decoding operations
0709 13:11:40.666908 (+ 88681us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:40.668862 (+  1954us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:40.670704 (+  1842us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:11:40.707999 (+ 37295us) tablet.cc:701] Row locks acquired
0709 13:11:40.710631 (+  2632us) write_op.cc:260] PREPARE: finished
0709 13:11:40.724732 (+ 14101us) write_op.cc:270] Start()
0709 13:11:40.751929 (+ 27197us) write_op.cc:276] Timestamp: P: 1783602700722853 usec, L: 0
0709 13:11:40.755032 (+  3103us) op_driver.cc:348] REPLICATION: starting
0709 13:11:40.837458 (+ 82426us) log.cc:844] Serialized 907 byte log entry
0709 13:11:40.905491 (+ 68033us) op_driver.cc:464] REPLICATION: finished
0709 13:11:41.172253 (+266762us) write_op.cc:301] APPLY: starting
0709 13:11:41.173124 (+   871us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:41.173367 (+   243us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:41.175869 (+  2502us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:41.181731 (+  5862us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:41.182175 (+   444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:11:41.182238 (+    63us) write_op.cc:312] APPLY: finished
0709 13:11:41.186782 (+  4544us) log.cc:844] Serialized 29 byte log entry
0709 13:11:41.188527 (+  1745us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:41.188692 (+   165us) write_op.cc:454] Released schema lock
0709 13:11:41.188748 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:41.189787 (+  1039us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:41.189820 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":264918,"apply.run_cpu_time_us":5592,"apply.run_wall_time_us":17787,"num_ops":2,"prepare.queue_time_us":65203,"prepare.run_cpu_time_us":16458,"prepare.run_wall_time_us":443291,"raft.queue_time_us":1343,"raft.run_cpu_time_us":12168,"raft.run_wall_time_us":257937,"replication_time_us":150284,"thread_start_us":246718,"threads_started":4,"wal-append.queue_time_us":63037}]]}
I20260709 13:11:41.288394 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.286006 22760 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:41.301630 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.302243 22765 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.302649 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:41.302791 22765 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:41.313189 22763 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:41.314095 24116 raft_consensus.cc:2764] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:11:41.315014 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.316964 22765 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.319298 22765 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:41.319325 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78239.8us
I20260709 13:11:41.321070 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:41.322081 22762 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:41.334475 24087 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.335556 24087 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:41.338881 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:41.334553 23461 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.321385 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:41.343093 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.341221 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.345283 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:41.341224 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.342079 22766 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:41.350057 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.343508 24087 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.367614 22765 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:41.368970 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63565.2us
I20260709 13:11:41.377633 24040 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.378259 24040 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.378666 24040 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.379381 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.397960 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.401486 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.403430 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.404517 24116 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:41.404994 24116 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:41.405422 24116 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.406025 24102 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.399375 23985 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:41.410501 23985 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:41.412688 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:41.413210 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.404773 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.412760 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.425468 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.426467 22914 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:41.427392 22841 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.430191 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.430374 22907 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.430974 22836 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.443810 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.421866 24086 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.446189 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.446513 22840 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.447223 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:41.448971 22909 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:41.457589 24102 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.458465 22838 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.457692 24086 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.460623 23458 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.481029 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.481065 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.458494 24102 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.493816 24102 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:41.509452 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.507375 24116 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.512037 24116 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:41.487737 23985 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:41.513769 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58714.2us
I20260709 13:11:41.517076 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:41.500885 22907 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.520907 22765 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
mode: GRACEFUL
new_leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:44588
I20260709 13:11:41.521757 22765 raft_consensus.cc:606] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Received request to transfer leadership to TS 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:41.520351 22835 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.552907 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:41.553665 24139 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:11:41.555083 24043 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.555567 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:41.558053 24088 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.559303 24088 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:41.558142 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:41.524228 23458 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.527094 23985 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:41.568064 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45078.7us
W20260709 13:11:41.569671 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.570883 24086 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:41.525874 24031 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.569260 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.578239 22908 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:41.581645 24043 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:41.583171 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.588810 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:41.590104 23886 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.595317 23886 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.595748 23886 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.595629 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.573745 24031 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.574996 23985 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:41.611057 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 116249us
I20260709 13:11:41.597234 24130 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:11:41.598106 24116 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.614643 24031 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.638451 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.639642 22909 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:41.640226 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:41.640909 22912 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.581804 24132 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.659806 24132 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:41.670960 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48648.9us
I20260709 13:11:41.672221 24086 raft_consensus.cc:993] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: : Instructing follower 74c0a02901bc45c891e42d0d1bbbf6a2 to start an election
I20260709 13:11:41.673134 24136 raft_consensus.cc:1081] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Signalling peer 74c0a02901bc45c891e42d0d1bbbf6a2 to start an election
I20260709 13:11:41.675338 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:41.678611 22915 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:41.681314 24102 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.683801 24102 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.688283 24092 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.689007 24092 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:41.689445 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:41.690120 24092 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.691375 24116 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.739337 24116 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.691982 22907 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:41.708230 24143 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:11:41.745667 24040 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.746567 24043 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.748097 24040 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.750348 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:41.615187 23985 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.770208 24031 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.719283 24105 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:46844 (request call id 10) took 10879 ms. Trace:
I20260709 13:11:41.770970 24053 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:41.773699 24132 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:41.724575 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 166487us
I20260709 13:11:41.777912 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:41.784339 22836 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:41.796418 22912 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:41.713147 24131 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:41.806748 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.790043 24132 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:41.797756 24031 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.824766 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:41.821261 24142 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.299s	user 0.005s	sys 0.003s
I20260709 13:11:41.825587 24132 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.837019 24031 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.833528 24094 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:11:41.840234 24104 raft_consensus.cc:993] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:41.833714 24086 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 13:11:41.844638 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 237040us
I20260709 13:11:41.847400 24043 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.853232 24031 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:41.824065 24116 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.828998 24053 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 13:11:41.863915 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70646us
W20260709 13:11:41.867558 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 279065us
I20260709 13:11:41.862969 24094 raft_consensus.cc:1081] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:41.876029 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.882184 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:41.888258 24031 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.886580 24070 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:11:41.885900 24120 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.891817 24120 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:41.885742 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:41.900264 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:41.901388 24131 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.902292 24131 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:41.902693 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:41.902823 24131 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:41.903894 24031 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.904506 24031 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:41.904991 24031 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.905301 24148 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.905663 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:41.906795 24066 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:41.907202 24066 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.907598 24066 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.893399 24120 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.911124 24120 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.917026 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:41.922488 24116 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:41.925015 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:41.930995 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:41.932046 24116 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.932531 22836 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:41.932783 24116 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.934278 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:41.935559 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:41.925521 24014 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:11:41.785573 24105 rpcz_store.cc:276] 0709 13:11:30.735000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.736940 (+  1940us) service_pool.cc:224] Handling call
0709 13:11:30.803411 (+ 66471us) log.cc:844] Serialized 492 byte log entry
0709 13:11:31.372106 (+568695us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b54000d19a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6a142 00007f68a1a4aeaa 00007f68a1a51d53 00007f68a1a51cd9 00007f68a1a51c69 00007f68a1a51c31 00007f68a1a50dfd 000000000038f354 000000000038f199 00007f689ea04475
0709 13:11:31.806782 (+434676us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18003066a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a71b26 00007f68a1a6b49a 00007f68a1a6a714
0709 13:11:31.843039 (+ 36257us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b54000d19a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6a142 00007f68a1a4aeaa 00007f68a1a51d53 00007f68a1a51cd9 00007f68a1a51c69 00007f68a1a51c31 00007f68a1a50dfd 000000000038f354 000000000038f199 00007f689ea04475
0709 13:11:31.879037 (+ 35998us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b54000d19a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a71b26 00007f68a1a6b49a 00007f68a1a6a714 00007f68a1a4aeaa 00007f68a1a51d53 00007f68a1a51cd9 00007f68a1a51c69 00007f68a1a51c31 00007f68a1a50dfd 000000000038f354
0709 13:11:41.608037 (+9729000us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2004,"raft.queue_time_us":3520,"raft.run_cpu_time_us":66671,"raft.run_wall_time_us":2419549,"spinlock_wait_cycles":16742784,"thread_start_us":84586,"threads_started":1,"wal-append.queue_time_us":387011,"wal-append.run_cpu_time_us":3984,"wal-append.run_wall_time_us":1359778}
W20260709 13:11:41.951414 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65754.9us
I20260709 13:11:41.962374 22839 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.957162 24053 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:11:41.903543 24131 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.965742 24090 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:41.971524 24090 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:41.972528 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.690s (SENT) [suppressed 63 similar messages]
I20260709 13:11:41.951256 24043 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:11:41.982968 23458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8"
dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:40710
I20260709 13:11:41.983630 24104 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:41.984012 23458 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:41.984294 24104 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:41.984658 23458 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:41.989210 22912 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:41.964421 22835 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:41.959781 24102 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:41.966475 24116 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:41.949541 23985 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:41.972884 24090 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:41.973440 23886 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:42.004347 23886 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:42.004906 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.014225 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:41.995362 24102 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:42.027007 22914 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.028023 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.031010 22911 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:42.028816 24096 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.048339 22915 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:11:42.055047 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60346.9us
I20260709 13:11:42.026006 24014 tablet_bootstrap.cc:492] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:42.033039 24088 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:11:42.062726 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:41.995939 24104 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.067606 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.068506 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.071619 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.076382 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.077519 22915 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:42.078781 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.079643 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.080576 22913 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:42.081595 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.081873 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.046499 24102 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.082728 22912 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:42.083914 22909 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.073406 24031 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.073951 22765 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:42.091089 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.091804 22765 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:42.103567 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.107553 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.076229 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.133638 24014 ts_tablet_manager.cc:1403] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 1.313s	user 0.038s	sys 0.034s
I20260709 13:11:42.124341 24031 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:42.139357 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.139906 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.140568 22764 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:42.140718 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.141547 22762 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:42.143115 23458 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.146629 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.147403 22764 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.147673 22762 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:42.122704 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.123428 23985 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:42.146580 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.111524 24110 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.124811 22841 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.149953 24110 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:42.127764 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:42.151325 24110 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.152550 24110 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.106734 24066 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.153709 24014 raft_consensus.cc:359] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.085428 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.132112 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.132987 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.111403 23886 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:42.159394 24116 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.160125 22834 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:42.159893 24096 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.159484 24066 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:42.160655 24102 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.159418 22758 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:42.150017 22832 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.162950 24102 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:42.163496 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.165045 22762 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.165946 22914 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.167177 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.168051 22758 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:42.116111 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.169771 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.170604 22765 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:42.130826 22909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:43274
I20260709 13:11:42.171811 22909 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:42.171900 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.172297 22909 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.172149 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.147882 22764 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:42.173622 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68002.3us
W20260709 13:11:42.174274 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.174782 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.175770 22765 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:42.121675 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.157965 22759 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:42.124014 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:42.178606 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57016us
I20260709 13:11:42.181668 24087 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:42.183183 24132 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.185688 24132 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:42.184774 22764 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.186872 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.187410 22764 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.189980 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.192559 24131 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.193291 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.191663 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.194909 24131 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:42.195886 22836 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.195731 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:42.196977 23886 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:42.197669 23886 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:42.198840 23886 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.199219 23886 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:42.196453 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.202107 24087 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:42.202574 22766 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.161463 24066 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.203768 22759 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.203934 24066 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:42.204540 22763 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.205354 24066 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.206804 24066 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.206598 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.200268 23886 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.208263 23886 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.209344 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.211078 24156 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:42.211585 22837 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.212219 24156 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.213491 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.214979 24156 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:42.183199 24014 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:42.218267 22837 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.219023 24014 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:42.168821 22908 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:42.223512 22760 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.225956 22837 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.226560 23458 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:42.225989 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:42.222122 24014 ts_tablet_manager.cc:1434] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.003s
W20260709 13:11:42.231077 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51291.3us
I20260709 13:11:42.230643 22760 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.236588 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.217096 23997 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:42.241544 22914 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.243371 22765 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.246388 23985 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.246853 22910 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.247880 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.248519 24155 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.249821 22833 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:42.256214 22765 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.259804 23458 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.259706 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:42.260152 22760 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.261264 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:42.268462 22837 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:42.269781 24155 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.272683 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:42.274324 24137 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.274830 24137 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:42.275506 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:42.276268 24137 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.270967 23997 tablet_bootstrap.cc:492] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:42.276464 24131 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.278019 24103 raft_consensus.cc:993] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: : Instructing follower 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:42.298257 24103 raft_consensus.cc:1081] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Signalling peer 6545453e74344bb2957dd3729c5b3870 to start an election
I20260709 13:11:42.286971 24003 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:42.298792 22660 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:42.287920 24105 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.308619 24105 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:42.288110 23985 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:42.309442 24003 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.309942 24003 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:42.310221 23985 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.300946 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:42.310609 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.311344 22915 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:42.288717 24151 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.313714 24151 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:42.293031 22841 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 3. Ops: []
I20260709 13:11:42.294283 23997 ts_tablet_manager.cc:1403] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 1.174s	user 0.027s	sys 0.024s
I20260709 13:11:42.277485 24137 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:42.316066 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 54578.4us
I20260709 13:11:42.314872 24151 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:42.322425 24151 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.316789 24155 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:11:42.318156 24157 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.235s	user 0.006s	sys 0.000s
I20260709 13:11:42.318696 24105 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.310612 23985 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:42.327713 24132 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:42.329015 24131 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:42.329869 24031 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.331902 22832 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:42.333777 24104 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.354264 24134 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.354583 24131 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.355422 24131 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:42.335734 24043 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:42.343503 24102 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:11:42.357936 24116 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.358198 22760 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.311200 24003 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.358387 24116 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:42.359632 24105 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.359381 24116 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.360759 24116 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.354436 24132 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.358100 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.361680 24132 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:42.362339 24155 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.362658 24132 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.364338 24132 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.338533 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:42.359532 24003 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.366756 24149 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:42.326838 22660 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:42.257612 22909 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.342047 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.349704 24031 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:42.351470 22851 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-af39123de9434cc0b451f4b03d305310 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:42.351944 22765 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.352128 23985 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.352470 24136 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:42.352780 24088 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:42.355494 24134 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:42.344256 22762 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:42.374382 22909 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:42.376662 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:42.382387 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:42.383795 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.383735 22660 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:42.391355 22765 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:42.392498 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.375425 24031 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.367252 24149 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:42.356868 24131 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:42.409314 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.393648 24031 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:42.413704 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.397944 22763 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.393044 22851 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-af39123de9434cc0b451f4b03d305310 on tablet af39123de9434cc0b451f4b03d305310 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:42.395037 23985 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.395879 24149 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.418069 22906 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:42.395617 22760 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:42.395119 23431 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.540s (SENT)
I20260709 13:11:42.428124 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.405563 24110 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:42.431977 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111643us
W20260709 13:11:42.425397 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.399223 24092 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:42.431506 22764 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:42.431000 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 324823us
I20260709 13:11:42.392846 22660 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 3, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 3 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:42.398138 22759 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
W20260709 13:11:42.439415 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.442204 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.393347 24170 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:11:42.453820 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.455633 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:42.459861 24053 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.460338 24053 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:42.460877 24092 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:42.461212 24053 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.462098 24092 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.397267 24134 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.465703 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.466566 24134 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.476902 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:42.486075 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 91002.2us
I20260709 13:11:42.452521 24131 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.506501 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:42.487425 24110 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:42.490108 24103 raft_consensus.cc:2764] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:11:42.493801 23886 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.496613 23997 raft_consensus.cc:359] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:42.538887 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.513458 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.558555 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:42.560098 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49305.5us
I20260709 13:11:42.561069 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:42.509958 24110 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:42.493613 24102 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:11:42.548445 23997 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:42.527350 24053 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.588690 24110 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.630635 24149 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.633999 24149 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:42.634817 24149 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.640259 23997 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:42.581669 24090 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.553896 24086 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:42.610230 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.611187 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107259us
I20260709 13:11:42.613847 24164 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.608024 23985 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:42.586782 23886 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:42.657964 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.659693 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.660955 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.663586 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:42.664178 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.661659 23886 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:42.664911 23886 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.650998 23985 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.678162 24103 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:42.686128 24162 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.682092 24066 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:42.687333 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.687633 24151 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:42.678635 23985 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.704268 24110 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.647782 24102 consensus_queue.cc:1243] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:11:42.708371 24110 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:42.716120 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.721287 24149 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:42.721956 24149 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:42.729923 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.736995 24149 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:42.738653 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75191.4us
I20260709 13:11:42.777323 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.778694 22836 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:42.792158 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.794407 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.795452 24087 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.016s	sys 0.058s
W20260709 13:11:42.795919 24087 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.016s	sys 0.058s
I20260709 13:11:42.781797 24105 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.817354 24149 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:42.871706 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.871533 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98521.5us
I20260709 13:11:42.873081 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:42.884235 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 377845us
I20260709 13:11:42.885403 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.903219 24105 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:42.909639 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.883958 24104 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.876251 24170 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:42.916033 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.913344 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.041s	sys 0.045s
W20260709 13:11:42.916231 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 355288us
W20260709 13:11:42.945334 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.041s	sys 0.045s
I20260709 13:11:42.883701 24164 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:42.945773 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 601751us
I20260709 13:11:42.885097 23886 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:42.952024 22878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.012s
W20260709 13:11:42.952436 22878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.006s	sys 0.012s
I20260709 13:11:42.954259 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:42.937989 24105 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:42.956851 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71568.3us
W20260709 13:11:42.961385 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.012s	sys 0.120s
W20260709 13:11:42.961724 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.012s	sys 0.120s
I20260709 13:11:42.947199 24104 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:42.955581 24181 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:42.957985 24179 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:42.963284 24165 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.956738 24164 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:42.989663 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.003s	sys 0.018s
I20260709 13:11:42.963632 24180 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:42.948318 24170 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.974303 24179 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:42.964406 24105 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:42.988277 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:43.004590 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:42.990046 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.003s	sys 0.018s
W20260709 13:11:42.994354 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.004s	sys 0.015s
W20260709 13:11:43.013383 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.004s	sys 0.015s
W20260709 13:11:43.015107 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68529.7us
I20260709 13:11:43.017966 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:11:43.018566 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 646304us
I20260709 13:11:43.016072 24118 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.020191 24118 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.020597 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.021307 22839 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.021633 22839 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.022626 24118 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.022460 22839 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.023373 22839 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.023190 24118 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:42.986781 23985 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:43.028708 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 563200us
I20260709 13:11:43.030776 22915 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.009575 24165 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.036501 22908 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:43274
I20260709 13:11:43.037072 22908 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:43.037442 22908 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.038579 22840 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:43.040066 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52024.9us
I20260709 13:11:43.040902 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:43.041522 22836 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:43.044231 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45674.3us
I20260709 13:11:42.992391 24092 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
W20260709 13:11:43.048765 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:43.052712 24177 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:11:43.054224 24175 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:11:43.058452 23461 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:43.059885 22838 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:43.036105 24031 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.060900 24179 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.048650 24184 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.070307 24155 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:43.030323 24181 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:43.073688 24155 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:43.081238 24031 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:43.082037 24178 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:11:43.082904 24151 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:43.083954 24151 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.084331 24151 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.084614 23985 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.086205 24104 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.076030 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.090524 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.094058 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.094689 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.096706 24156 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:43.097678 24164 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.070853 24165 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:11:43.099253 22759 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:43.099550 23997 ts_tablet_manager.cc:1434] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.780s	user 0.010s	sys 0.018s
W20260709 13:11:43.099993 23997 ts_tablet_manager.cc:1467] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Tablet startup took 780ms
I20260709 13:11:43.077538 23886 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.077191 24134 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.082576 24031 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.099354 24165 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.104307 22762 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:43.106196 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.106441 22759 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:43.110953 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:43.118152 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:43.118587 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:43.119086 22764 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:43.122768 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:43.123207 24197 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:43.124069 22759 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:43.123778 24197 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.119472 24087 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.126969 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.124107 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.085178 24151 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.107499 24165 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.163229 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.165192 22915 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:43.135910 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:43.135262 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.168092 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.168727 22841 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.168859 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.172246 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.130666 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:43.173801 24163 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:11:43.174167 22758 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.175127 24163 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.175428 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.171165 24086 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.176920 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:43.133915 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.176172 22910 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:43.125823 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.129242 24179 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.180213 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.180589 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.204233 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:43.150027 24137 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.824s	user 0.010s	sys 0.003s
I20260709 13:11:43.180833 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:43.184741 24170 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.176390 22912 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:43.175518 24163 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.205300 24086 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.220781 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.220517 24163 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.221937 24163 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.222143 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.223526 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.224653 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.225725 24189 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.225864 22906 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:43.199227 24149 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.131223 24105 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.227816 24118 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:43.228390 24118 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:43.193229 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 79064.2us
I20260709 13:11:43.229708 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:43.230685 24118 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:43.231351 24118 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.231981 24118 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:11:43.142865 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.130970 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.234301 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.128628 24087 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.131510 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.204878 23985 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:43.238883 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.239073 24165 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.136047 24151 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.240538 24165 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.240603 22906 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.131615 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:43.242125 24086 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.243104 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.136677 22762 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:43.244686 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 56518.4us
I20260709 13:11:43.185110 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.195515 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:43.146401 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 97395.5us
W20260709 13:11:43.248555 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59499us
W20260709 13:11:43.197356 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 70532.9us
I20260709 13:11:43.198069 24136 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:11:43.129650 24197 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.201952 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.202144 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.111747 24164 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:11:43.205400 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.239579 22838 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.225221 22913 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.252640 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.136654 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:43.253998 22914 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:43.128203 24103 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:43.278723 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82686.7us
I20260709 13:11:43.259132 22841 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:43.278803 22836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:43.264148 22840 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:43.142822 22908 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.280022 22836 raft_consensus.cc:606] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.280527 22836 raft_consensus.cc:614] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:43.280584 22909 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.128912 22760 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:43.128562 24188 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.110620 24181 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.292366 24086 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:43.276093 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.277514 22766 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:43.238246 24087 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.293907 24191 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:43.283740 22841 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:43.302343 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.220530 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:43.316484 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.793s (SENT)
I20260709 13:11:43.270356 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:43.278761 22834 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.334061 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.335675 24164 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.338562 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:43.342701 23886 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.344636 24173 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.380715 22839 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:43.133455 24155 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.383185 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.290328 23985 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:11:43.346187 23463 raft_consensus.cc:1217] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:43.349097 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:11:43.349941 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72592.9us
I20260709 13:11:43.354391 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:43.355510 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 103009us
I20260709 13:11:43.394634 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.358682 22908 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:43.395704 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:43.364931 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:43.388161 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54245.6us
W20260709 13:11:43.390033 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:43.397725 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.389477 24197 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
W20260709 13:11:43.390653 24053 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.003s
W20260709 13:11:43.406232 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 114808us
I20260709 13:11:43.412629 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.414323 23466 raft_consensus.cc:1217] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:11:43.414712 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98305.5us
I20260709 13:11:43.415645 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.330381 24187 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 13:11:43.359213 24164 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.390476 24118 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:43.428534 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:43.407961 24132 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.406911 24188 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.406174 24162 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:43.391126 23457 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.407761 23463 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:43.410563 24173 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.406389 24156 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:11:43.449394 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.427794 24031 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.465984 22764 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.437168 24053 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.471956 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.409098 24196 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:11:43.474485 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:43.475509 23466 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:43.464524 24170 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:11:43.464071 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:43.454519 22758 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:43.477834 23886 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.478529 22765 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:43.459342 23985 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.478796 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:43.473443 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:43.479485 22761 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.473630 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:43.473964 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.482940 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.439481 22911 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:43.477226 24170 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:43.444300 24131 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.446244 24183 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.435937 24118 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.487315 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.442541 24162 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:43.490747 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 208487us
W20260709 13:11:43.506462 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.505430 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:43.447821 24188 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.549158 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.488451 24190 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.454778 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.495143 24105 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.490079 24031 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.511627 24162 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.515137 24053 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:11:43.549295 23886 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:43.100379 23997 ts_tablet_manager.cc:1469] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Trace:
0709 13:11:32.511909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.584356 (+ 72447us) service_pool.cc:224] Handling call
0709 13:11:33.122465 (+538109us) ts_tablet_manager.cc:934] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
0709 13:11:36.837032 (+3714567us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:11:37.747729 (+910697us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:37.747856 (+   127us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:11:40.312857 (+2565001us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:11:40.313208 (+   351us) tablet_replica.cc:206] Creating consensus instance
0709 13:11:40.313575 (+   367us) ts_tablet_manager.cc:989] Replica registered.
0709 13:11:40.313656 (+    81us) inbound_call.cc:177] Queueing success response
0709 13:11:41.120319 (+806663us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:11:42.319772 (+1199453us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:11:42.341768 (+ 21996us) tablet_replica.cc:261] Starting instrumentation
0709 13:11:42.366454 (+ 24686us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":336,"raft.queue_time_us":365741,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":15767,"tablet-copy.queue_time_us":121873,"tablet-download-pool.queue_time_us":129,"tablet-download-pool.run_cpu_time_us":1127,"tablet-download-pool.run_wall_time_us":430279,"thread_start_us":674751,"threads_started":3}
I20260709 13:11:43.591398 23886 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.592123 23886 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:43.598484 24205 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:11:43.593232 23886 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:43.571941 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 91767.6us
I20260709 13:11:43.603044 24188 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.603289 24066 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.502414 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.583444 24149 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.585803 24053 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:43.497255 24170 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.600715 24104 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.609572 24104 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.609627 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.603698 24066 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.496763 24164 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:43.571453 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 146707us
I20260709 13:11:43.608760 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.609887 24173 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.610864 24066 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.612318 24066 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.612222 24162 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.610867 22759 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.610081 24104 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.614655 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.615104 24170 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.615407 22759 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:43.616338 24209 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:11:43.618041 22763 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:43.619395 22759 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.608430 24188 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:43.607149 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 105721us
I20260709 13:11:43.621013 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:43.621696 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.622457 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.627084 22911 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.609598 24165 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:11:43.628961 24151 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.629392 24151 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.630389 24151 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.631601 22833 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:43.631919 24184 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.632414 24151 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.632534 24184 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.634080 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:43.628093 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.637068 22761 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:43.644652 24190 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.577730 24105 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.646209 24105 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.647778 24105 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.648209 24103 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.648676 24103 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:43.649147 24103 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.643289 24053 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:43.652551 24053 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:43.652971 24164 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.653038 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.641279 24162 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.663801 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.664405 24186 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:43.664800 24186 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:43.670528 24155 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.676113 24186 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.677412 24186 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.678067 22661 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:43.695155 24118 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:43.695690 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:43.702154 22879 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba1db22a7176432c86e7264f87984ba3. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:43.702689 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 230924us
I20260709 13:11:43.702263 22661 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:43.703737 23432 leader_election.cc:336] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.690s (SENT)
I20260709 13:11:43.703935 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:43.704278 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.704190 22661 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:43.707701 24092 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:43.708441 22838 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:43.709236 24092 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.709645 24092 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.709995 24092 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.711561 24179 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.718257 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.661015 22834 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:43.720024 22661 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:43.724085 24094 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.726065 24094 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.679893 24104 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.681273 24149 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:43.653609 24053 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:43.748061 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 241854us
I20260709 13:11:43.725539 22661 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:43.700726 24103 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.743858 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.756256 22661 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:43.768714 24155 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.769416 24218 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.769474 24155 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.763407 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.745362 23461 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:43.745162 23985 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:43.788798 24179 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.789281 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:43.773383 22661 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:43.789160 24134 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:43.807129 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.770105 24218 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:43.816442 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 351142us
I20260709 13:11:43.802587 24087 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:43.800179 22911 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.816980 22661 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:43.823418 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:43.812474 24134 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:43.805133 24118 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.834519 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.802971 24213 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:43.840824 23985 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.841348 24213 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.844141 24213 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.845453 22851 tablet_copy_service.cc:342] P 776baf137f8745dd866379ca15ab8cee: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-2f666ea4542d4951919f36de0b43b972 received from {username='slave'} at 127.0.0.1:46858
I20260709 13:11:43.845961 22851 tablet_copy_service.cc:434] P 776baf137f8745dd866379ca15ab8cee: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-2f666ea4542d4951919f36de0b43b972 on tablet 2f666ea4542d4951919f36de0b43b972 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:43.847329 24087 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:43.847447 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:43.813894 24132 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:43.850212 24132 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.855180 24094 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:43.801549 24208 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:11:43.840608 24196 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.855327 24218 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:43.822096 24183 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.838999 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:43.814169 24179 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.836279 24105 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:43.867165 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.838274 24134 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.848754 24087 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.874977 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:43.875813 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:43.876307 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 365055us
I20260709 13:11:43.876479 22765 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.875799 22759 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:43.881444 24031 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:43.861680 24196 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:43.883828 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44977.9us
I20260709 13:11:43.864176 24201 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:43.817355 24149 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.883929 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.890244 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.871017 24131 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:43.896754 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:43.909884 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.894155 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:43.911494 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:43.921564 22909 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:43.909431 22914 raft_consensus.cc:2376] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2. Re-sending same reply.
I20260709 13:11:43.862095 24118 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:43.889742 24196 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.948134 22839 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.882192 24031 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.939879 22837 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:43.940824 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.886341 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:43.930640 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.885627 24164 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:43.947748 22908 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:43.885213 24190 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.894639 24218 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.893833 22834 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:43.875801 24105 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:43.969408 22839 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:43.972066 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57519.9us
I20260709 13:11:43.936846 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:43.959765 22760 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.963090 22761 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:43.990193 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:43.946805 24163 raft_consensus.cc:2764] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:11:43.947784 24086 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:43.993204 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62682.2us
I20260709 13:11:43.938591 24104 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.009733 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.967314 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:44.014308 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:44.016553 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:44.020612 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.983237 24162 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.025120 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:43.989156 24155 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.041486 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:43.985970 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:44.025525 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:44.052160 22837 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:44.016263 22763 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:44.034406 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:43.987560 24218 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 2 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:44.024246 24208 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:44.064286 24221 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:11:44.064371 24208 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.054762 24213 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:44.030400 24170 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.016762 24086 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.071218 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:44.045745 24189 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:44.048935 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:44.036136 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:44.017400 24103 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.002977 24222 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:44.059527 22832 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:44.045928 24212 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:11:44.078624 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:44.001125 24165 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:11:44.064666 24105 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:44.034895 24162 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.037776 23886 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:44.031476 24188 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:43.976588 24190 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.058363 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:44.077157 22766 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:43.981321 24164 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.072790 24213 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:44.071689 24211 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:11:44.092142 24003 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.026s	sys 0.035s
W20260709 13:11:44.127125 24003 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.027s	sys 0.035s
I20260709 13:11:44.127475 24224 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.026381 24131 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.089876 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:44.137997 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:44.093251 24170 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:44.142220 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.143795 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.154524 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:44.162783 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:44.131512 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:44.168974 24162 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:44.169936 24031 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:44.170553 24031 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.121730 24179 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:44.155169 24170 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:44.206655 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.164820 24201 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.171170 24031 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.214212 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:44.215772 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:44.231307 22908 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:44.221450 22759 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:44.190941 24189 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.181545 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 351) took 1070 ms. Trace:
I20260709 13:11:44.158785 24184 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:44.246393 22759 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:44.253225 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:44.165588 24218 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.163865 24197 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:11:44.206043 24170 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:44.269166 22908 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:44.279075 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:44.281456 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:44.215546 24118 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:44.288024 24094 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:44.292583 22839 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:44.261863 24162 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:44.270452 24227 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.293273 24186 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.005s
I20260709 13:11:44.294467 24163 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:44.310922 24190 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:44.302908 22758 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.005s
I20260709 13:11:44.217626 24179 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:11:44.284721 24053 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:44.306310 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:44.328799 24230 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:11:44.299762 24094 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.330094 24094 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.299489 24224 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.262972 24031 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.306448 24136 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.303629 24162 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.319511 24208 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:44.333750 24163 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.337862 22913 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:44.321417 24226 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.331669 22758 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:44.402575 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:44.267181 23461 rpcz_store.cc:276] 0709 13:11:43.094117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.101661 (+  7544us) service_pool.cc:224] Handling call
0709 13:11:44.158787 (+1057126us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400514854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:44.164588 (+  5801us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744012160}
I20260709 13:11:44.300814 24213 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:44.416847 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 404182us
W20260709 13:11:44.412333 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 370978us
I20260709 13:11:44.333240 24227 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.394668 24163 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.418205 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:44.425856 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:44.411761 24164 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:44.355957 24136 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.433355 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.437297 22841 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:44.447464 22839 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:44.454860 22730 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:44.360598 23997 ts_tablet_manager.cc:933] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:44.357053 24162 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.457017 24151 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:44.552119 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:44.397933 24224 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.509341 24132 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.560804 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.564704 22839 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:44.561959 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:44.568231 23886 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:44.552209 24003 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.561056 24031 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:44.565802 24231 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 13:11:44.584695 23886 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.583810 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:44.584267 24219 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:44.591058 24219 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:44.591632 24224 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.592551 24224 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:44.572062 24190 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.580217 22907 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:11:44.581156 24196 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.004s
W20260709 13:11:44.580709 22912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.027s	sys 0.051s
I20260709 13:11:44.576733 24243 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:44.572813 24092 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.859s	user 0.014s	sys 0.001s
W20260709 13:11:44.579869 24232 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 13:11:44.421478 24226 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:44.613782 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:44.663053 22840 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.016s	sys 0.017s
W20260709 13:11:44.663553 22840 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.016s	sys 0.017s
I20260709 13:11:44.664429 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:44.666920 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:44.611923 22912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.027s	sys 0.051s
I20260709 13:11:44.626029 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.627017 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.627929 24092 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.606154 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:44.596846 24105 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:44.593477 24224 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:44.637434 24243 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:44.637996 24225 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.001s	sys 0.027s
I20260709 13:11:44.639026 24163 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.604473 24196 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.594002 24190 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.582614 24003 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.676517 24165 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.660876 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.673375 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.653030 24170 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.678671 24239 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:44.681602 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.604470 24151 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:44.606598 24136 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:44.607949 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:44.659643 24219 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.657442 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.679800 24105 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:44.711575 22839 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:44.707825 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.686125 24087 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:44.710788 24225 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.002s	sys 0.028s
W20260709 13:11:44.709821 24188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.017s	sys 0.022s
W20260709 13:11:44.717629 24222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.010s	sys 0.012s
I20260709 13:11:44.707137 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:44.685261 24184 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:44.721840 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.014s	sys 0.028s
I20260709 13:11:44.703634 24211 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:44.747164 24233 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:11:44.748242 24227 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.717938 24191 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:44.750231 24196 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:44.751081 24224 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.714481 24190 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:44.732445 24134 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.843s	user 0.014s	sys 0.000s
I20260709 13:11:44.755651 22839 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:44.746300 24092 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.738492 24087 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.727238 24151 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.718622 24165 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.739017 22835 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:44.732228 24219 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:44.760624 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.014s	sys 0.028s
W20260709 13:11:44.756999 24222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.011s	sys 0.013s
I20260709 13:11:44.705235 24243 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:44.756556 24188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.017s	sys 0.023s
I20260709 13:11:44.830078 24196 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:44.822271 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:44.797883 22838 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.007s	sys 0.025s
I20260709 13:11:44.845778 22841 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:44.823482 24183 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.952s	user 0.015s	sys 0.000s
I20260709 13:11:44.765676 22757 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:44.768435 24245 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:44.779191 24134 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.855808 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.820698 24087 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.824810 24186 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:44.773926 24105 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.857653 22762 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:44.849357 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:44.839197 24222 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:44.889007 23886 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:44.846148 22838 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.008s	sys 0.026s
W20260709 13:11:44.892350 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175054us
I20260709 13:11:44.845708 24188 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:44.838376 22763 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:44.893787 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:44.890625 22839 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:44.905114 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:44.856904 24183 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.742687 24239 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.914458 24186 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.895916 24227 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.891435 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:44.853619 24196 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:44.893237 24226 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.902339 24254 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:44.923383 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 200959us
I20260709 13:11:44.926990 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:44.903682 24092 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.888512 24252 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:44.925205 23465 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:44.911156 23997 tablet_copy_client.cc:323] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:11:44.921706 24053 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:44.992697 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:44.921250 24003 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:44.912649 24066 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:11:44.889552 24245 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.005549 24103 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:44.920881 24191 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:45.005125 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:44.986310 24226 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.011945 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.013372 24186 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.014367 22840 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:44.948642 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:44.986987 24170 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.013238 24066 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.026008 24165 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:45.026985 24256 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 13:11:45.021258 24219 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.011005 24053 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:45.027712 23464 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:45.029965 24263 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:45.031010 24241 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 13:11:45.033684 24249 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:11:45.034688 24243 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.035156 24196 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.035668 24134 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.022385 24246 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.081547 24163 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.004614 24252 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.945549 24105 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:44.944193 24227 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.043558 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:45.022778 24188 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.053805 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:45.054749 24094 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.724s	user 0.014s	sys 0.001s
I20260709 13:11:44.968645 24254 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.051358 22757 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:45.035705 24103 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.063673 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 907) took 1066 ms. Trace:
I20260709 13:11:45.107478 22759 rpcz_store.cc:276] 0709 13:11:43.997616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.273884 (+276268us) service_pool.cc:224] Handling call
0709 13:11:44.274584 (+   700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.275033 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.275072 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:44.348398 (+ 73326us) log.cc:844] Serialized 492 byte log entry
0709 13:11:44.843458 (+495060us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:44.843509 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.843603 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.063401 (+219798us) raft_consensus.cc:1685] finished
0709 13:11:45.063433 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.063603 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":426361,"threads_started":1,"wal-append.queue_time_us":568556}
I20260709 13:11:45.064507 22906 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:45.067029 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 71) took 1186 ms. Trace:
I20260709 13:11:45.067313 24053 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.990655 24092 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:45.068362 24269 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:45.036952 24226 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.043093 24189 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.042534 24191 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:44.989665 24155 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.141914 24155 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.037744 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:45.144063 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 909) took 1101 ms. Trace:
I20260709 13:11:45.144570 22915 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
W20260709 13:11:45.085338 24253 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:11:45.029480 24162 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.086871 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 900) took 1175 ms. Trace:
W20260709 13:11:45.081755 24257 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:11:45.154346 22766 rpcz_store.cc:276] 0709 13:11:43.911023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.174732 (+263709us) service_pool.cc:224] Handling call
0709 13:11:44.180820 (+  6088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.181158 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.181188 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:44.181736 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:11:44.680362 (+498626us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:44.680423 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.680526 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.086387 (+405861us) raft_consensus.cc:1685] finished
0709 13:11:45.086416 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.086664 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":368808,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":7215,"thread_start_us":498166,"threads_started":1,"wal-append.queue_time_us":652871}
W20260709 13:11:45.093881 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 196704us
I20260709 13:11:45.083531 24190 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:44.859750 24211 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.054283 24066 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:45.102139 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48477.9us
I20260709 13:11:45.120355 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 74) took 1222 ms. Trace:
I20260709 13:11:45.111930 22907 rpcz_store.cc:276] 0709 13:11:43.880486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.880623 (+   137us) service_pool.cc:224] Handling call
0709 13:11:43.880948 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.598635 (+717687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.066683 (+468048us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:45.066927 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561392512}
I20260709 13:11:45.168551 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:45.105310 24243 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.103565 24094 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.170053 24066 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.159192 24162 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.171196 24094 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.125116 24227 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.161001 22912 rpcz_store.cc:276] 0709 13:11:43.897564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.897795 (+   231us) service_pool.cc:224] Handling call
0709 13:11:43.934101 (+ 36306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.934602 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.934646 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:43.967869 (+ 33223us) log.cc:844] Serialized 26 byte log entry
0709 13:11:44.679021 (+711152us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:44.888706 (+209685us) log.cc:844] Serialized 14 byte log entry
0709 13:11:44.889567 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.889662 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.120045 (+230383us) raft_consensus.cc:1685] finished
0709 13:11:45.120069 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.120230 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9699,"prepare.queue_time_us":315200,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":8497,"raft.queue_time_us":399,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":35336,"thread_start_us":858897,"threads_started":2,"wal-append.queue_time_us":775464}
I20260709 13:11:45.129194 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.172962 22759 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:45.138520 24252 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:45.139662 24235 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.795s	user 0.001s	sys 0.003s
I20260709 13:11:45.017797 24222 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:45.080389 24263 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.157260 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:45.157575 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet ba1db22a7176432c86e7264f87984ba3 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:11:45.158272 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.159152 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:45.156304 24164 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.160095 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:45.135319 24226 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.159943 24239 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.091827 24246 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.173394 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:45.173687 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:45.191026 22839 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:45.112421 24103 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.173921 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.136121 24189 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:45.127228 24268 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:45.179127 24092 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.179988 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:45.140491 24269 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:45.153399 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 950) took 1070 ms. Trace:
I20260709 13:11:45.188657 22766 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:45.167804 24260 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.176970 24031 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:45.189960 24164 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.192309 24134 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.192065 24252 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.194145 22759 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:45.196229 24261 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:11:45.196151 24186 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:45.203258 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:45.203727 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:45.192134 24271 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:11:45.204453 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:45.188772 24233 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:45.197579 24272 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:11:45.193745 23464 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:45.214958 24263 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.200160 22838 rpcz_store.cc:276] 0709 13:11:44.070381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.084824 (+ 14443us) service_pool.cc:224] Handling call
0709 13:11:44.130289 (+ 45465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.130904 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:44.912201 (+781297us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.102203 (+190002us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.104355 (+  2152us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:45.110338 (+  5983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.130048 (+ 19710us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.135348 (+  5300us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":472713,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":1838,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":2168,"thread_start_us":938884,"threads_started":2,"wal-append.queue_time_us":214629}
I20260709 13:11:45.220921 24184 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:45.221195 24245 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.222383 22907 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:45.200188 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 69) took 1324 ms. Trace:
I20260709 13:11:45.225932 22915 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.232393 24251 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:11:45.190559 22757 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:45.234202 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:45.236339 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:45.237093 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:45.240237 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:45.216248 24003 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:45.241596 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.193305 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet ba1db22a7176432c86e7264f87984ba3 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-ba1db22a7176432c86e7264f87984ba3
I20260709 13:11:45.215787 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:45.215591 24087 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.222888 24260 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.241887 24272 consensus_queue.cc:1243] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:45.233676 24252 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.304955 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:45.287588 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107652us
W20260709 13:11:45.372889 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68085.9us
I20260709 13:11:45.204084 22834 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:45.351115 23886 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.375161 23886 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:45.350124 24151 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.512s	user 0.014s	sys 0.000s
I20260709 13:11:45.373723 22912 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.243497 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:45.376217 22909 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:45.379029 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 135602us
I20260709 13:11:45.346817 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:45.144701 22758 rpcz_store.cc:276] 0709 13:11:44.043115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.455649 (+412534us) service_pool.cc:224] Handling call
0709 13:11:44.554418 (+ 98769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.584084 (+ 29666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.586324 (+  2240us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:44.589607 (+  3283us) log.cc:844] Serialized 26 byte log entry
0709 13:11:44.913257 (+323650us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:45.142017 (+228760us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.143546 (+  1529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.143691 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.143757 (+    66us) raft_consensus.cc:1685] finished
0709 13:11:45.143782 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.143979 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":261653,"raft.queue_time_us":43251,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":287,"thread_start_us":439121,"threads_started":2,"wal-append.queue_time_us":484830}
I20260709 13:11:45.380005 24173 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.274919 24094 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.390769 23886 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.391285 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.391582 23886 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:45.393183 23886 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:45.233875 24191 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.377803 24137 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.374555 24184 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:45.396976 22766 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:45.233675 24186 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.360355 24276 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.383899 24151 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.231792 24134 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.402175 24087 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.402400 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:45.402650 24162 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:11:45.299365 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1766) took 1411 ms (client timeout 1852 ms). Trace:
I20260709 13:11:45.374974 24246 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.378619 24164 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.449831 23886 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.389601 24173 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.471591 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.478616 24173 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.404701 24137 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.473405 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:45.409104 24103 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.238731 24031 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.443286 24191 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:45.383499 24003 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.481886 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:45.396958 24094 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.476526 22757 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.306689 24269 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.483675 24087 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:11:45.239164 24233 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:45.473217 22913 rpcz_store.cc:269] 0709 13:11:43.888026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.888441 (+   415us) service_pool.cc:224] Handling call
0709 13:11:45.299065 (+1410624us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54001d0e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:45.299214 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710906752}
I20260709 13:11:45.482069 24151 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.494531 24252 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.469127 24276 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.479660 24162 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.556231 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.376873 22911 rpcz_store.cc:276] 0709 13:11:43.873296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.877544 (+  4248us) service_pool.cc:224] Handling call
0709 13:11:43.877904 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.878306 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.878338 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:43.878845 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:11:44.017315 (+138470us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:44.025283 (+  7968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.047660 (+ 22377us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.139035 (+1091375us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:11:45.193237 (+ 54202us) raft_consensus.cc:1685] finished
0709 13:11:45.194559 (+  1322us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.197655 (+  3096us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1207,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"spinlock_wait_cycles":2561792,"thread_start_us":103289,"threads_started":1,"wal-append.queue_time_us":400023}
I20260709 13:11:45.557025 22766 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:45.558122 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:45.547354 22912 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:45.559315 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.549072 22909 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:45.560124 22766 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:45.483935 24137 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:45.545811 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1716) took 2094 ms (client timeout 1926 ms). Trace:
I20260709 13:11:45.492818 24191 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.568643 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 906) took 1587 ms. Trace:
I20260709 13:11:45.558859 24269 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.549492 22761 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:45.553735 24260 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.562852 24003 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.569602 22762 rpcz_store.cc:276] 0709 13:11:43.972519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.270467 (+297948us) service_pool.cc:224] Handling call
0709 13:11:44.281673 (+ 11206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.889013 (+607340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.559235 (+670222us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400393354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:45.559412 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324042624}
I20260709 13:11:45.559581 24162 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.531590 24086 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.498179 22758 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.579516 22914 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:45.571880 24134 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.524015 24227 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.529335 24164 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.510972 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:45.569537 22764 rpcz_store.cc:269] 0709 13:11:43.451325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.652723 (+201398us) service_pool.cc:224] Handling call
0709 13:11:45.545521 (+1892798us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400311354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:45.545701 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2498107776}
I20260709 13:11:45.533041 24031 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.582297 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:45.561668 24239 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.510627 24103 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:45.572844 24191 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.533170 24254 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.597980 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.580930 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:45.494999 24184 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.599560 24260 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.762588 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.763370 24103 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.653126 24219 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.764087 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.789134 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.671979 24104 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.652861 22760 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:45.594224 24162 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.762990 24239 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.763860 24184 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.653173 24186 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:45.735004 24105 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.002s
I20260709 13:11:45.602841 24227 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:45.768689 24226 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.001s
I20260709 13:11:45.808081 24104 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.704000 24151 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.597868 24086 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.809540 24219 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.762976 24260 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.814424 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.789921 22762 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:45.844808 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.707676 23985 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:45.847529 23985 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:45.601007 24134 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:45.611754 24164 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.743662 24094 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.835047 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.768828 24276 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.715553 24277 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.795437 24233 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.843335 24151 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.851958 24189 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.860544 24104 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.860734 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 910) took 1817 ms. Trace:
I20260709 13:11:45.868178 22764 rpcz_store.cc:276] 0709 13:11:44.043529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.859875 (+1816346us) service_pool.cc:224] Handling call
0709 13:11:45.860225 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.860417 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.860447 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.860490 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:45.860521 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.860577 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.860678 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.861321 24094 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:45.863590 24164 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.832540 24162 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.846077 24086 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.879289 22759 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:45.879570 24239 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.879959 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 912) took 1835 ms. Trace:
I20260709 13:11:45.880317 22759 rpcz_store.cc:276] 0709 13:11:44.044173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.878606 (+1834433us) service_pool.cc:224] Handling call
0709 13:11:45.878941 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.879709 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.879903 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.881796 24219 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:45.884312 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 86) took 1365 ms (client timeout 1537 ms). Trace:
I20260709 13:11:45.840018 24226 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.885950 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 913) took 1841 ms. Trace:
W20260709 13:11:45.884752 22763 rpcz_store.cc:269] 0709 13:11:44.518548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.607591 (+ 89043us) service_pool.cc:224] Handling call
0709 13:11:45.884132 (+1276541us) spinlock_profiling.cc:243] Waited 97.9 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:45.884250 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225182080}
I20260709 13:11:45.887879 24260 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.888273 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1727) took 1321 ms. Trace:
I20260709 13:11:45.888741 22763 rpcz_store.cc:276] 0709 13:11:44.567042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.887374 (+1320332us) service_pool.cc:224] Handling call
0709 13:11:45.887701 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.887901 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.887937 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.887987 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:45.888026 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.888091 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.888215 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.870667 24276 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.885141 24227 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.864275 24277 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.886379 22759 rpcz_store.cc:276] 0709 13:11:44.044508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.881075 (+1836567us) service_pool.cc:224] Handling call
0709 13:11:45.881388 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.883785 (+  2397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.883850 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.883901 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:45.883935 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.885782 (+  1847us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.885897 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.886713 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:45.900362 22766 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:45.902769 24086 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:45.902871 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1731) took 1041 ms. Trace:
I20260709 13:11:45.876528 24094 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.903573 22763 rpcz_store.cc:276] 0709 13:11:44.861225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.901454 (+1040229us) service_pool.cc:224] Handling call
0709 13:11:45.902089 (+   635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.902370 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.902416 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.902491 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:45.902544 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.902631 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.902789 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:45.904994 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 341) took 1893 ms (client timeout 1606 ms). Trace:
W20260709 13:11:45.905462 23465 rpcz_store.cc:269] 0709 13:11:44.011496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.016200 (+  4704us) service_pool.cc:224] Handling call
0709 13:11:45.904818 (+1888618us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b540063f954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:45.904923 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2088139520}
I20260709 13:11:45.900774 24276 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:45.906510 22763 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:45.866505 24134 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.848531 23985 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:45.907917 24189 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.905460 24094 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.947935 24277 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.948920 24263 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:45.949518 24263 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.896468 24173 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:45.966452 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.022s	sys 0.027s
I20260709 13:11:45.952046 24218 raft_consensus.cc:1064] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:45.952181 24260 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:45.970046 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.022s	sys 0.028s
I20260709 13:11:45.979933 24276 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:45.979951 24137 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:45.983141 24137 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.958741 24103 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.963886 24263 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.975762 24226 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:45.988157 24246 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.984046 24218 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:46.019404 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.013s	sys 0.038s
I20260709 13:11:45.997243 22758 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:46.025396 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.021s	sys 0.028s
W20260709 13:11:46.025848 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.021s	sys 0.028s
I20260709 13:11:45.989035 24260 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:46.020394 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.013s	sys 0.038s
I20260709 13:11:46.016731 24162 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:46.019784 24151 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:46.010993 22763 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:46.054255 22878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.007s	sys 0.040s
I20260709 13:11:46.019969 24155 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:46.019448 24226 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:46.054750 22878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.007s	sys 0.040s
W20260709 13:11:46.028864 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.020s	sys 0.020s
I20260709 13:11:46.049353 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1733) took 1186 ms. Trace:
I20260709 13:11:46.104115 22660 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 130) took 4179 ms. Trace:
I20260709 13:11:46.018131 24246 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:46.082296 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.021s	sys 0.020s
W20260709 13:11:46.090469 24224 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.010s	sys 0.024s
I20260709 13:11:46.054607 24181 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:46.071060 24136 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.007s	sys 0.039s
W20260709 13:11:46.134222 24224 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.010s	sys 0.024s
I20260709 13:11:46.140136 24086 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:46.121637 22757 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.001s
I20260709 13:11:46.122380 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 920) took 1203 ms. Trace:
W20260709 13:11:46.052168 22731 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.020s	sys 0.025s
W20260709 13:11:46.156261 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.021s	sys 0.067s
W20260709 13:11:46.169857 22731 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.020s	sys 0.026s
W20260709 13:11:46.169957 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.022s	sys 0.068s
W20260709 13:11:46.163061 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.015s	sys 0.021s
W20260709 13:11:46.171612 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.015s	sys 0.021s
I20260709 13:11:46.174057 23461 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.175146 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:46.175963 23465 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:46.176620 24233 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:46.177670 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.139060 24242 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:46.137387 24155 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:46.138104 24183 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:46.094554 22758 rpcz_store.cc:276] 0709 13:11:44.861897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.481477 (+619580us) service_pool.cc:224] Handling call
0709 13:11:46.047794 (+566317us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:46.153270 24136 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.008s	sys 0.040s
I20260709 13:11:46.140992 24284 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:46.091509 24226 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:46.104647 22660 rpcz_store.cc:276] 0709 13:11:41.925156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.925427 (+   271us) service_pool.cc:224] Handling call
0709 13:11:46.103988 (+4178561us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:43.025000 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:43.060052 (+ 35052us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:43.060110 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:11:43.060131 (+    21us) tablet.cc:662] Decoding operations
0709 13:11:43.060865 (+   734us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:43.060952 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:43.060987 (+    35us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:11:43.086170 (+ 25183us) tablet.cc:701] Row locks acquired
0709 13:11:43.086212 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:11:43.086431 (+   219us) write_op.cc:270] Start()
0709 13:11:43.086606 (+   175us) write_op.cc:276] Timestamp: P: 1783602703086360 usec, L: 0
0709 13:11:43.086626 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:11:43.185862 (+ 99236us) log.cc:844] Serialized 1662 byte log entry
0709 13:11:44.046919 (+861057us) op_driver.cc:464] REPLICATION: finished
0709 13:11:44.773321 (+726402us) write_op.cc:301] APPLY: starting
0709 13:11:44.775207 (+  1886us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:44.838802 (+ 63595us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:44.858891 (+ 20089us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:45.029623 (+170732us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:45.029841 (+   218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:11:45.029889 (+    48us) write_op.cc:312] APPLY: finished
0709 13:11:45.030139 (+   250us) log.cc:844] Serialized 41 byte log entry
0709 13:11:45.030324 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:45.030567 (+   243us) write_op.cc:454] Released schema lock
0709 13:11:45.030629 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:45.031335 (+   706us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:45.031366 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":713920,"apply.run_cpu_time_us":4374,"apply.run_wall_time_us":262701,"num_ops":4,"prepare.queue_time_us":378293,"prepare.run_cpu_time_us":104296,"prepare.run_wall_time_us":431591,"raft.queue_time_us":460002,"raft.run_cpu_time_us":66967,"raft.run_wall_time_us":878172,"replication_time_us":960121,"thread_start_us":1588334,"threads_started":4,"wal-append.queue_time_us":312005,"wal-append.run_cpu_time_us":25905,"wal-append.run_wall_time_us":2590404}]]}
I20260709 13:11:46.178202 24173 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:46.235827 24284 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:46.164042 22757 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:46.177110 24086 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:46.143710 24181 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:46.741859 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.000s	sys 0.022s
W20260709 13:11:46.759428 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.000s	sys 0.023s
I20260709 13:11:46.348043 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 949) took 2242 ms. Trace:
W20260709 13:11:46.553623 22764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.000s	sys 0.017s
W20260709 13:11:46.342741 22841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1011) took 1676 ms (client timeout 2140 ms). Trace:
W20260709 13:11:46.427215 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.011s	sys 0.031s
W20260709 13:11:46.301649 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.002s	sys 0.059s
W20260709 13:11:46.764374 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.011s	sys 0.031s
W20260709 13:11:46.764513 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.477s	user 0.017s	sys 0.081s
I20260709 13:11:46.342201 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 85) took 1873 ms. Trace:
W20260709 13:11:46.766008 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.54767e+06us
W20260709 13:11:46.427286 22804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.000s	sys 0.042s
W20260709 13:11:46.766815 22804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.000s	sys 0.051s
W20260709 13:11:46.762806 22841 rpcz_store.cc:269] 0709 13:11:44.666100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.681199 (+ 15099us) service_pool.cc:224] Handling call
0709 13:11:46.342523 (+1661324us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001bce54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:46.342637 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292579200}
I20260709 13:11:46.765184 22766 rpcz_store.cc:276] 0709 13:11:44.469199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.813647 (+1344448us) service_pool.cc:224] Handling call
0709 13:11:46.341908 (+528261us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b540033fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:46.342116 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129154304}
W20260709 13:11:46.760967 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.012s	sys 0.009s
W20260709 13:11:46.773131 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.012s	sys 0.009s
W20260709 13:11:46.762472 22764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.000s	sys 0.025s
W20260709 13:11:46.774580 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:46.775111 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.776383 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.788810 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.789765 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:46.789817 22915 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:46.789827 22914 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:46.794138 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.794731 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.794868 22911 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:46.795634 22909 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:46.796715 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.797256 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.797425 22915 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:46.797976 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.800258 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:46.647464 24188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.027s	sys 0.024s
W20260709 13:11:46.578207 22910 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.174s	user 0.012s	sys 0.027s
W20260709 13:11:46.652421 24053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.093s	user 0.030s	sys 0.059s
W20260709 13:11:46.551549 24213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.009s	sys 0.033s
I20260709 13:11:46.761874 22837 rpcz_store.cc:276] 0709 13:11:44.061104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.069422 (+  8318us) service_pool.cc:224] Handling call
0709 13:11:44.083900 (+ 14478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.157289 (+ 73389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.157321 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:44.172603 (+ 15282us) log.cc:844] Serialized 492 byte log entry
0709 13:11:44.727441 (+554838us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:44.727495 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.200297 (+1472802us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54001bc954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:46.200373 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:46.200440 (+    67us) raft_consensus.cc:1685] finished
0709 13:11:46.200460 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.303336 (+102876us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167535488,"thread_start_us":466090,"threads_started":1,"wal-append.queue_time_us":702270,"wal-append.run_cpu_time_us":3733,"wal-append.run_wall_time_us":1241931}
W20260709 13:11:46.409166 24268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.018s	sys 0.044s
I20260709 13:11:46.344842 24246 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:46.223718 24155 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:46.550916 24136 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:46.521597 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:46.216470 24233 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:46.225680 24183 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:46.229957 24242 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:47.468920 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.421s	user 0.060s	sys 0.097s
I20260709 13:11:46.808262 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:47.617066 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.569s	user 0.065s	sys 0.107s
I20260709 13:11:47.617403 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:47.617751 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 1.56962e+06us
I20260709 13:11:46.505204 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 898) took 2604 ms. Trace:
W20260709 13:11:47.552286 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.148s	user 0.020s	sys 0.032s
I20260709 13:11:46.413017 24286 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:46.370884 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 902) took 2429 ms (client timeout 1681 ms). Trace:
I20260709 13:11:46.814719 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.772871 24226 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:46.858072 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:47.639606 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.008s	sys 0.004s
I20260709 13:11:46.163769 22763 rpcz_store.cc:276] 0709 13:11:44.918788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.943943 (+1025155us) service_pool.cc:224] Handling call
0709 13:11:45.972653 (+ 28710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:46.069200 (+ 96547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.115990 (+ 46790us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:46.549353 24285 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:47.629590 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.006s	sys 0.006s
W20260709 13:11:47.649387 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.006s	sys 0.006s
W20260709 13:11:47.650017 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 876252us
W20260709 13:11:47.642215 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.238s	user 0.024s	sys 0.032s
I20260709 13:11:47.617635 23465 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
W20260709 13:11:46.808935 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.596257 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.009s	sys 0.003s
W20260709 13:11:47.655180 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.010s	sys 0.003s
W20260709 13:11:47.643841 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.003s	sys 0.010s
I20260709 13:11:47.042810 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:47.656939 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.003s	sys 0.010s
W20260709 13:11:47.544104 22732 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.018s	sys 0.036s
W20260709 13:11:47.657531 22732 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.020s	sys 0.037s
W20260709 13:11:47.640782 24053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.082s	user 0.054s	sys 0.091s
I20260709 13:11:47.645128 22766 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:47.661978 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1743) took 1709 ms (client timeout 1776 ms). Trace:
W20260709 13:11:47.662396 22766 rpcz_store.cc:269] 0709 13:11:45.952351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.772322 (+819971us) service_pool.cc:224] Handling call
0709 13:11:47.661905 (+889583us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:47.618649 24268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.765s	user 0.043s	sys 0.085s
W20260709 13:11:47.387447 22807 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.020s	sys 0.020s
W20260709 13:11:47.658034 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.36569e+06us
W20260709 13:11:47.700042 22807 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.024s	sys 0.022s
W20260709 13:11:47.700508 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 936832us
I20260709 13:11:47.643489 22758 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:46.892069 24189 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.945s	user 0.006s	sys 0.009s
W20260709 13:11:47.650902 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.000s	sys 0.010s
W20260709 13:11:47.703629 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.000s	sys 0.010s
W20260709 13:11:47.703799 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.004s	sys 0.045s
W20260709 13:11:47.704262 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.004s	sys 0.045s
I20260709 13:11:47.211557 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:46.810529 22910 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.403s	user 0.016s	sys 0.030s
W20260709 13:11:46.800731 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1741) took 1597 ms (client timeout 1612 ms). Trace:
W20260709 13:11:46.807053 24188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.027s	sys 0.024s
I20260709 13:11:47.662075 24066 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:47.622859 24213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.913s	user 0.031s	sys 0.045s
W20260709 13:11:47.659747 22731 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.005s	sys 0.037s
W20260709 13:11:47.734061 22731 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.005s	sys 0.037s
W20260709 13:11:47.660056 22764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.009s	sys 0.005s
W20260709 13:11:47.734767 22764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.009s	sys 0.005s
I20260709 13:11:47.735412 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 911) took 3691 ms. Trace:
I20260709 13:11:46.873409 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:47.641409 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:47.198707 24289 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.845523 24173 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.894977 24134 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:47.700587 22880 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.020s	sys 0.030s
W20260709 13:11:47.518536 22759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.016s	sys 0.028s
I20260709 13:11:47.701774 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:46.990345 24260 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:47.702184 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.758s (SENT) [suppressed 11 similar messages]
W20260709 13:11:47.744374 22880 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.021s	sys 0.030s
W20260709 13:11:47.744479 22759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.020s	sys 0.028s
W20260709 13:11:47.744711 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 980651us
I20260709 13:11:47.022805 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:46.931110 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:47.704576 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 913275us
I20260709 13:11:47.814191 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1798) took 1014 ms. Trace:
I20260709 13:11:47.814575 22911 rpcz_store.cc:276] 0709 13:11:46.799275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.802666 (+  3391us) service_pool.cc:224] Handling call
0709 13:11:47.814120 (+1011454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:47.792620 24186 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:47.062566 24298 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:47.815940 24186 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:47.816933 24292 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.860s	user 0.000s	sys 0.001s
I20260709 13:11:47.816409 24298 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:47.799029 24134 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:47.643955 24295 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.817920 24134 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.818511 24295 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:47.820092 24134 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:47.800129 24260 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.047708 24300 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:47.660449 24226 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.660316 24181 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:47.712263 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80657.9us
W20260709 13:11:47.705811 22804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.016s	sys 0.006s
I20260709 13:11:47.201467 24290 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.506s
I20260709 13:11:47.735679 22764 rpcz_store.cc:276] 0709 13:11:44.043860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.869596 (+1825736us) service_pool.cc:224] Handling call
0709 13:11:45.875970 (+  6374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.876240 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:45.894651 (+ 18411us) log.cc:844] Serialized 14 byte log entry
0709 13:11:47.735105 (+1840454us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:47.735159 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:47.735192 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.735247 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.735366 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32587,"prepare.queue_time_us":18159,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":868556,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5563,"thread_start_us":1807335,"threads_started":2,"wal-append.queue_time_us":1534989}
W20260709 13:11:47.738317 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.652659 24286 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.647814 24136 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.737986 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:47.660883 24285 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.724321 24086 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:47.640901 22761 rpcz_store.cc:276] 0709 13:11:43.901102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.166970 (+265868us) service_pool.cc:224] Handling call
0709 13:11:45.549243 (+1382273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.579089 (+ 29846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.412895 (+833806us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400311350. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:46.505040 (+ 92145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3178092416}
I20260709 13:11:47.663779 24291 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:47.742193 22765 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:47.800747 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 914) took 2895 ms. Trace:
I20260709 13:11:47.728755 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:47.736140 24297 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.798640 24183 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:47.820717 24295 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:47.646004 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.009s	sys 0.004s
W20260709 13:11:47.641615 22760 rpcz_store.cc:269] 0709 13:11:43.941024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.069171 (+128147us) service_pool.cc:224] Handling call
0709 13:11:46.370538 (+2301367us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54000816d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:46.370753 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3408849152}
I20260709 13:11:47.838055 24136 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:47.838610 22907 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:47.840018 24191 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 2.074s	user 0.007s	sys 0.010s
I20260709 13:11:47.840420 24191 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:47.841360 24285 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:47.841580 24291 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:47.843572 24286 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:47.843930 24291 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:47.844841 22915 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:11:47.829427 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.829615 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 88) took 1802 ms (client timeout 1753 ms). Trace:
I20260709 13:11:47.830430 22759 rpcz_store.cc:276] 0709 13:11:44.905567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.367117 (+1461550us) service_pool.cc:224] Handling call
0709 13:11:46.367707 (+   590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:46.368005 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:46.375120 (+  7115us) log.cc:844] Serialized 14 byte log entry
0709 13:11:47.800418 (+1425298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:47.800491 (+    73us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:47.800538 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.800618 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.800702 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6864,"prepare.queue_time_us":5754,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"thread_start_us":984244,"threads_started":1,"wal-append.queue_time_us":1367688}
I20260709 13:11:47.829985 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:47.830062 22804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.017s	sys 0.006s
I20260709 13:11:47.731820 24053 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.726321 24189 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:47.794263 24299 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:11:47.862524 24189 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.863276 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1788) took 1074 ms. Trace:
I20260709 13:11:47.798395 24155 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:47.821324 24260 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:47.863749 22913 rpcz_store.cc:276] 0709 13:11:46.788721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.799795 (+ 11074us) service_pool.cc:224] Handling call
0709 13:11:47.862414 (+1062619us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:47.862647 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.862847 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.862882 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:47.862929 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:47.862977 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.863052 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.863208 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41364736}
I20260709 13:11:47.865113 24189 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:47.794669 24296 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.723s	user 0.000s	sys 0.001s
I20260709 13:11:47.839335 24290 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:47.830152 24173 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:47.843492 24242 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:47.822134 24319 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.735313 24268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51126 (request call id 5) took 5545 ms. Trace:
W20260709 13:11:47.924245 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 94364.6us
I20260709 13:11:47.925115 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:47.738701 24066 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:47.841645 24294 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.644s	user 0.000s	sys 0.001s
I20260709 13:11:47.845151 24314 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:47.861949 24305 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.242s	user 0.000s	sys 0.002s
W20260709 13:11:47.708284 22757 rpcz_store.cc:269] 0709 13:11:45.180386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.390913 (+210527us) service_pool.cc:224] Handling call
0709 13:11:46.777792 (+1386879us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:47.742502 23458 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:47.826818 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125309us
W20260709 13:11:47.847652 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.846851 22766 rpcz_store.cc:269] 0709 13:11:46.027128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.699378 (+1672250us) service_pool.cc:224] Handling call
0709 13:11:47.829535 (+130157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:47.857594 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:47.858001 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 917) took 2940 ms. Trace:
W20260709 13:11:47.938637 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1744) took 1986 ms (client timeout 1926 ms). Trace:
I20260709 13:11:47.938484 22762 rpcz_store.cc:276] 0709 13:11:44.917909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.829915 (+2912006us) service_pool.cc:224] Handling call
0709 13:11:47.857405 (+ 27490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.857645 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.857674 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:47.857722 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:47.857751 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.857812 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.857941 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:47.801083 22732 connection.cc:256] RPC call timeout handler was delayed by -1.212096628999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 13:11:47.800734 24317 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.834245 24131 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:47.896970 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81475.7us
I20260709 13:11:47.857159 24053 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:47.800287 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.813381 24184 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.982s	user 0.006s	sys 0.011s
I20260709 13:11:47.923602 24173 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:47.820725 24134 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 13:11:47.927937 22915 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:47.819873 24298 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:47.855962 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:47.931661 24314 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.821379 24300 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.932792 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 77) took 3761 ms. Trace:
I20260709 13:11:47.792085 24289 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.824875 24320 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:11:47.856932 24233 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:47.857686 22763 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:47.824210 24293 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:11:47.881166 24290 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:47.939316 22764 rpcz_store.cc:269] 0709 13:11:45.952649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.830647 (+1877998us) service_pool.cc:224] Handling call
0709 13:11:47.938537 (+107890us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:47.799985 24188 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.878981 24310 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:47.933188 24304 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:47.950356 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:47.922399 24250 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.885546 24284 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:47.846537 24226 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.930742 24285 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:47.927738 24315 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:47.973918 23431 connection.cc:256] RPC call timeout handler was delayed by -1.1711904619999132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 13:11:47.941557 24312 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.951270 24260 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:47.983763 22762 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:47.986092 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1735) took 2893 ms. Trace:
I20260709 13:11:47.936344 24242 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.923210 24268 rpcz_store.cc:276] 0709 13:11:42.166531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.166909 (+   378us) service_pool.cc:224] Handling call
0709 13:11:42.226944 (+ 60035us) log.cc:844] Serialized 492 byte log entry
0709 13:11:47.707889 (+5480945us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":185178,"raft.run_cpu_time_us":1885,"raft.run_wall_time_us":1883,"thread_start_us":763952,"threads_started":2,"wal-append.queue_time_us":647395,"wal-append.run_cpu_time_us":2924,"wal-append.run_wall_time_us":1179139}
I20260709 13:11:48.049857 24318 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:48.014467 24300 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:47.969424 24131 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:47.860401 24297 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:48.141968 24282 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 2.014s	user 0.017s	sys 0.014s
I20260709 13:11:48.144452 24314 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.015560 22910 rpcz_store.cc:276] 0709 13:11:44.171577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.173422 (+  1845us) service_pool.cc:224] Handling call
0709 13:11:44.180777 (+  7355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.181184 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.181212 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:45.402958 (+1221746us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b2c00051d28. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:11:45.403529 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:11:47.735209 (+2331680us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:47.739759 (+  4550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.792171 (+ 52412us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:47.932425 (+140254us) raft_consensus.cc:1685] finished
0709 13:11:47.932456 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.932688 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":302,"spinlock_wait_cycles":4939008,"thread_start_us":2305229,"threads_started":1,"wal-append.queue_time_us":1542737}
I20260709 13:11:48.051935 24233 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
W20260709 13:11:48.047145 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 108884us
W20260709 13:11:48.103169 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 89) took 2075 ms (client timeout 2098 ms). Trace:
I20260709 13:11:48.146410 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:47.984480 24189 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.147826 24191 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:48.148594 24189 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.134477 24242 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.056257 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:47.988454 24286 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.023165 24307 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:48.153716 24307 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:48.127351 24315 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.154981 24307 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.061319 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1791) took 1269 ms. Trace:
I20260709 13:11:48.066428 24304 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.007505 24132 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.157840 24304 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.158257 24132 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:47.998267 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.023810 24087 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.967654 24317 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.162075 24087 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.131501 24295 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:48.168123 24295 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.168579 24295 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:48.090404 22730 connection.cc:256] RPC call timeout handler was delayed by -1.5630342879999262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 13:11:48.095395 23462 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:48.048664 24298 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
W20260709 13:11:48.095901 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 145645us
I20260709 13:11:48.100482 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.115474 22766 rpcz_store.cc:276] 0709 13:11:45.093245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.938290 (+2845045us) service_pool.cc:224] Handling call
0709 13:11:47.985545 (+ 47255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.985759 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.985786 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:47.985828 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:47.985858 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.985909 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:47.986033 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:48.115729 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1738) took 3018 ms. Trace:
W20260709 13:11:48.051211 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1746) took 2096 ms (client timeout 1851 ms). Trace:
I20260709 13:11:48.027144 24155 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:11:48.010500 24327 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:48.060889 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:48.136008 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.139444 24066 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.132227 24312 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:48.140637 24281 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.019s	user 0.009s	sys 0.005s
I20260709 13:11:48.141644 24131 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:48.153242 24323 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:11:48.065253 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:47.930073 24319 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.152467 24189 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.100075 24285 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.067598 24188 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.095935 24250 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.157663 22912 rpcz_store.cc:276] 0709 13:11:46.792215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.792338 (+   123us) service_pool.cc:224] Handling call
0709 13:11:46.792686 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:46.843287 (+ 50601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:46.843322 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:46.843370 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:46.843399 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.060945 (+1217546us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b54002107d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:48.061022 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.061231 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115800448}
I20260709 13:11:48.389356 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.439851 22909 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:48.211624 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.222561 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 168858us
W20260709 13:11:48.443084 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1801) took 1569 ms (client timeout 2067 ms). Trace:
I20260709 13:11:48.379825 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 361) took 2200 ms. Trace:
W20260709 13:11:48.442387 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53189.8us
I20260709 13:11:48.439925 22762 rpcz_store.cc:276] 0709 13:11:45.097126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.941197 (+2844071us) service_pool.cc:224] Handling call
0709 13:11:47.983379 (+ 42182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:48.115394 (+132015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.115661 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:48.190186 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43867.6us
W20260709 13:11:48.444154 22909 rpcz_store.cc:269] 0709 13:11:46.873454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.873672 (+   218us) service_pool.cc:224] Handling call
0709 13:11:48.065002 (+1191330us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:48.443020 (+378018us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41253120}
I20260709 13:11:48.376951 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:48.446173 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.445453 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:48.444278 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 343887us
I20260709 13:11:48.134140 24213 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:48.448380 24330 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.448760 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.448904 24086 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:48.449360 24213 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.375305 23985 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.083766 22757 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:48.451752 24213 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.164316 24317 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.247721 24326 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.376861 24328 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:11:48.134080 24173 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:48.443342 24285 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.454424 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.479734 22762 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.173379 24286 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.483263 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.144476 24297 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:48.309307 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.444201 23466 rpcz_store.cc:276] 0709 13:11:46.172155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.172330 (+   175us) service_pool.cc:224] Handling call
0709 13:11:46.172730 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:46.805937 (+633207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:46.805970 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:46.806020 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:46.806052 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.371875 (+1565823us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54006112d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:48.371946 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.372183 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455762304}
I20260709 13:11:48.464957 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:11:48.489703 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44313.4us
I20260709 13:11:48.490769 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.492051 22807 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.134713 24329 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.476387 22879 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.478971 22764 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:48.479468 24066 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:48.247092 24191 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.174404 24310 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.490906 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.434257 24332 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.182416 24260 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:48.456073 22758 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.743s	user 0.007s	sys 0.004s
I20260709 13:11:48.484549 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.480270 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:48.402118 24315 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.480577 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.166851 24289 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.423203 24087 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.246515 24300 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:11:48.464713 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 87852.6us
I20260709 13:11:48.478803 24131 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.454731 24226 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.494356 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.457660 24327 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.480913 24173 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.497841 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.498337 22879 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:48.499189 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.004s	sys 0.018s
W20260709 13:11:48.146817 22761 rpcz_store.cc:269] 0709 13:11:46.027438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.844484 (+1817046us) service_pool.cc:224] Handling call
0709 13:11:48.098405 (+253921us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:48.439859 22763 rpcz_store.cc:269] 0709 13:11:45.953926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.699259 (+1745333us) service_pool.cc:224] Handling call
0709 13:11:48.049906 (+350647us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:48.558559 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.004s	sys 0.018s
I20260709 13:11:48.487269 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.459493 24326 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.495527 23457 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:48.524407 24286 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:48.253654 24331 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:48.581749 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.495360 24297 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.581171 24312 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.522307 24310 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.583837 24155 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:48.584395 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.585059 22912 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:48.536513 24332 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.505735 24066 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:48.600135 24066 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.558961 24213 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.601783 24213 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.602142 24213 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.530256 24328 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:48.603624 24328 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:48.604041 24304 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.547202 24131 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.534088 24260 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.497812 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:48.554757 23466 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:48.491441 24333 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:48.501089 24317 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.503023 24184 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:48.497031 24337 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.522089 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.521725 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.173202 24318 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:48.560858 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62652.2us
I20260709 13:11:48.522434 24336 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:48.562480 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:48.567432 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 907149us
I20260709 13:11:48.523545 24290 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.531431 22758 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:48.521816 22925 tablet_copy_source_session.cc:215] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:48.528463 24319 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.580562 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:48.488049 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 366) took 1684 ms (client timeout 2031 ms). Trace:
I20260709 13:11:48.572815 24173 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:48.508706 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.587489 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:48.570760 24087 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.587651 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.587926 24310 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.570322 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.542032 22913 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:48.594623 24326 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.505612 24191 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.598917 24327 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.540711 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:48.545463 24250 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.609925 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:48.554526 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60270.9us
I20260709 13:11:48.702344 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.612383 22912 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:48.703176 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 212546us
I20260709 13:11:48.579269 24181 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.704514 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 926) took 3573 ms. Trace:
I20260709 13:11:48.617931 22762 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.705044 22757 rpcz_store.cc:276] 0709 13:11:45.130660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.949788 (+2819128us) service_pool.cc:224] Handling call
0709 13:11:48.026516 (+ 76728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:48.054015 (+ 27499us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:48.602745 (+548730us) log.cc:844] Serialized 14 byte log entry
0709 13:11:48.659639 (+ 56894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:48.660816 (+  1177us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:48.667029 (+  6213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.694004 (+ 26975us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.699370 (+  5366us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5784,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":3198,"thread_start_us":44349,"threads_started":1}
I20260709 13:11:48.706347 22925 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:51142 (request call id 80) took 3549 ms. Trace:
I20260709 13:11:48.706848 24332 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.707108 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.707850 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:48.660436 24336 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:48.708315 24295 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.003s
I20260709 13:11:48.708117 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.708794 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.604761 24328 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.708739 24295 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.694043 24268 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.709905 24328 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.710235 24053 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.709970 24268 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.711161 24318 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.712780 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.574613 24188 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.663403 22763 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.597083 22910 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:48.637282 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.623945 24184 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.769167 24184 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.771914 23459 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:11:48.772665 23459 pending_rounds.cc:85] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 4
I20260709 13:11:48.606581 24321 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:48.773695 24321 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:48.774739 24321 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.776089 24321 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.623539 23464 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:11:48.619299 23458 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.008s
I20260709 13:11:48.500890 24329 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.614671 24131 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.616045 24331 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:48.653388 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.548210 24300 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.654260 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.709923 24336 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:48.785079 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 130918us
I20260709 13:11:48.658979 22766 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.625609 24317 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.653456 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:48.659524 22759 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:48.643360 24337 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.661690 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 896) took 4763 ms. Trace:
W20260709 13:11:48.789686 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 136324us
W20260709 13:11:48.659812 24303 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.000s	sys 0.019s
W20260709 13:11:48.700163 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 159548us
I20260709 13:11:48.838240 22661 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 239) took 5322 ms. Trace:
W20260709 13:11:48.838320 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52100.9us
I20260709 13:11:48.705875 24289 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:48.709079 22906 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:48.718475 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.630127 24304 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:48.734248 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:48.730527 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.636251 24211 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.695307 22760 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.706969 22925 rpcz_store.cc:276] 0709 13:11:45.157022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.157226 (+   204us) service_pool.cc:224] Handling call
0709 13:11:48.521349 (+3364123us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54001e2dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1afb3f7 00007f68a6a91036 00007f68a6a9aa3e 00007f689e35e27b 00007f68a6a9a9ba 00007f68a6a95acc 00007f68a6a90c7b 00007f68a6a858d1 00007f68a5b394b4 00007f68a5b39442
0709 13:11:48.699157 (+177808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6079824640}
I20260709 13:11:48.856721 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:48.746979 24053 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.860066 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.860296 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:48.756861 24213 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.707126 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.759514 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.865239 22762 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.661533 24316 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:11:48.866309 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:48.867043 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:48.661999 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1745) took 2708 ms (client timeout 1626 ms). Trace:
I20260709 13:11:48.715314 22761 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:48.840197 24289 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.766060 24295 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:48.661103 23461 rpcz_store.cc:269] 0709 13:11:46.803405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.805229 (+  1824us) service_pool.cc:224] Handling call
0709 13:11:48.135748 (+1330519us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:48.485606 (+349858us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32639360}
I20260709 13:11:48.657783 24285 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.779422 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1805) took 1417 ms. Trace:
I20260709 13:11:48.780310 22764 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.863144 24053 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.653213 24297 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:11:48.773185 23459 pending_rounds.cc:107] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 13:11:48.786664 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 133382us
I20260709 13:11:46.800426 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:48.777875 23458 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:48.789732 22765 rpcz_store.cc:276] 0709 13:11:43.896276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.899962 (+  3686us) service_pool.cc:224] Handling call
0709 13:11:44.140523 (+240561us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:44.140819 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.829295 (+3688476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.658932 (+829637us) spinlock_profiling.cc:243] Waited 3.6 s on lock 0x7b540036f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:48.659143 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8304983680}
I20260709 13:11:48.892269 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.778527 24307 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:48.852500 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 134178us
W20260709 13:11:48.853142 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 122764us
I20260709 13:11:48.897248 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:48.854584 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:48.855537 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 121389us
I20260709 13:11:48.857420 22841 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:48.899832 22766 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:48.900350 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.838012 24303 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.215s	user 0.001s	sys 0.043s
I20260709 13:11:48.860984 22837 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:48.837215 24317 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.746215 24326 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.839803 24181 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:48.923966 24181 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.924412 24181 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.924552 24242 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.921842 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.926569 22766 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:48.926865 24285 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:48.927947 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.867885 23465 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:48.928666 24300 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.929579 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.763113 24260 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.872401 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:48.930209 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:48.932026 24316 consensus_queue.cc:1243] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:11:48.854070 24328 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:11:48.851528 24268 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:48.940465 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.841208 24337 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.877717 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:48.890127 24053 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.857024 24134 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.857375 24310 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.858289 24186 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.837244 24320 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:11:48.840819 24086 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.652185 24333 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:48.863171 24318 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:48.881274 22915 rpcz_store.cc:276] 0709 13:11:47.362466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.641123 (+278657us) service_pool.cc:224] Handling call
0709 13:11:48.779188 (+1138065us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540010e0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:48.779339 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246516096}
I20260709 13:11:48.888015 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.976938 22762 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.886560 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.978402 22764 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:48.961964 24310 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.979432 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.962476 24285 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.980284 22764 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:48.857174 24211 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.893908 22765 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:48.983369 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.984223 22765 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.895514 24336 raft_consensus.cc:2764] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:11:48.896040 24331 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:48.903575 24298 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.993800 24298 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:48.994366 24298 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.994140 24318 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.996795 24318 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:48.894526 24191 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:48.906062 22757 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:48.997952 24333 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:48.998920 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.000367 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:48.906406 23458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 367) took 2098 ms (client timeout 1563 ms). Trace:
I20260709 13:11:48.874996 24289 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.908056 23463 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:48.838768 22661 rpcz_store.cc:276] 0709 13:11:43.473374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.474223 (+   849us) service_pool.cc:224] Handling call
0709 13:11:48.786389 (+5312166us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:44.624027 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:44.646616 (+ 22589us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:44.646659 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:11:44.646683 (+    24us) tablet.cc:662] Decoding operations
0709 13:11:44.647734 (+  1051us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:44.648007 (+   273us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:44.648042 (+    35us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:11:44.648732 (+   690us) tablet.cc:701] Row locks acquired
0709 13:11:44.648767 (+    35us) write_op.cc:260] PREPARE: finished
0709 13:11:44.648945 (+   178us) write_op.cc:270] Start()
0709 13:11:44.649063 (+   118us) write_op.cc:276] Timestamp: P: 1783602704648894 usec, L: 0
0709 13:11:44.649083 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:11:44.860141 (+211058us) log.cc:844] Serialized 3261 byte log entry
0709 13:11:44.928941 (+ 68800us) op_driver.cc:464] REPLICATION: finished
0709 13:11:45.031908 (+102967us) write_op.cc:301] APPLY: starting
0709 13:11:45.032027 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:45.032279 (+   252us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:45.032386 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:45.043715 (+ 11329us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:45.111425 (+ 67710us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:11:45.120292 (+  8867us) write_op.cc:312] APPLY: finished
0709 13:11:45.133976 (+ 13684us) log.cc:844] Serialized 65 byte log entry
0709 13:11:45.134179 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:45.154433 (+ 20254us) write_op.cc:454] Released schema lock
0709 13:11:45.156516 (+  2083us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:46.174007 (+1017491us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aece28 00007f68a5527581 00007f68a5526d58 00007f68a552b9f1 00007f68a552b9a9 00007f68a552b939 00007f68a552b901 00007f68a552ab2d 000000000038f354 000000000038f199
0709 13:11:46.174284 (+   277us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:46.174325 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":937838,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":88395,"apply.run_cpu_time_us":6401,"apply.run_wall_time_us":136100,"num_ops":8,"prepare.queue_time_us":604748,"prepare.run_cpu_time_us":12030,"prepare.run_wall_time_us":1590875,"replication_time_us":268460,"spinlock_wait_cycles":240310016,"thread_start_us":535538,"threads_started":2}]]}
I20260709 13:11:48.908967 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:48.912563 24132 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:48.918049 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:48.895728 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:48.909077 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:48.921934 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 923) took 3966 ms. Trace:
I20260709 13:11:48.922427 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:49.008783 23462 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:49.009284 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99080.3us
I20260709 13:11:49.008440 22759 rpcz_store.cc:276] 0709 13:11:44.949180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.858828 (+2909648us) service_pool.cc:224] Handling call
0709 13:11:48.659256 (+800428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.789465 (+130209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.915052 (+125587us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b540035dc50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:48.915265 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839935744}
I20260709 13:11:49.010515 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:48.657747 22907 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
W20260709 13:11:48.934172 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44204.7us
I20260709 13:11:48.879397 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.012789 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.013028 23460 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:49.013105 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:48.947675 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70046.7us
I20260709 13:11:49.010808 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.014986 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:49.015089 22759 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:48.859920 24173 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.941469 22766 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:48.951848 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:48.954752 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:48.958154 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:48.871671 22758 rpcz_store.cc:269] 0709 13:11:45.953575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.326037 (+372462us) service_pool.cc:224] Handling call
0709 13:11:48.661056 (+2335019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:48.905845 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.974534 24290 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.979602 24310 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.890918 23459 raft_consensus.cc:2889] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:48.986112 24211 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:49.003408 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94581.4us
I20260709 13:11:48.923257 22760 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:49.011866 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.012220 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.015481 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:49.008231 24285 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:49.016274 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107267us
I20260709 13:11:49.005987 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:48.975813 24086 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:49.027380 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.046918 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:49.048765 22806 connection.cc:256] RPC call timeout handler was delayed by -2.0447111699999141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:11:49.019868 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.019622 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1747) took 1021 ms. Trace:
I20260709 13:11:49.051005 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.051252 22766 rpcz_store.cc:276] 0709 13:11:47.998559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.940076 (+941517us) service_pool.cc:224] Handling call
0709 13:11:49.019533 (+ 79457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.030577 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:49.001776 23458 rpcz_store.cc:269] 0709 13:11:46.808154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.808322 (+   168us) service_pool.cc:224] Handling call
0709 13:11:48.906276 (+2097954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.051841 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.037596 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:11:49.049293 22806 connection.cc:256] RPC call timeout handler was delayed by -1.7766526750000367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 13:11:49.037906 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83331.3us
I20260709 13:11:49.015599 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1757) took 1012 ms. Trace:
I20260709 13:11:49.053045 22759 rpcz_store.cc:276] 0709 13:11:48.002883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.010479 (+1007596us) service_pool.cc:224] Handling call
0709 13:11:49.015540 (+  5061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.020573 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.007354 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.033478 24173 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.051904 22909 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.019361 24338 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:49.055025 22762 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:49.019246 24300 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.050501 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.029188 24218 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.062134 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:49.063416 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48057us
I20260709 13:11:49.065043 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.070070 22909 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.071312 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.071414 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.072270 22833 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.072355 22841 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.038066 24303 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:49.073726 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.029359 22832 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:49.075664 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.076323 22841 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.076493 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.077332 22833 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.043747 24191 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:49.079170 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.079027 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.048066 24285 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.081492 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.092176 22909 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.082368 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.081005 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.039891 22906 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:49.053973 24086 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.045657 24317 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.101423 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.102181 22906 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.102646 24181 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.067651 24173 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.103472 22906 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:49.104878 24181 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.063433 24331 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:49.080133 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:49.085856 22908 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.086529 22912 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.053694 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 931) took 1004 ms. Trace:
I20260709 13:11:49.040074 23461 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.093124 22841 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.074215 24196 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.098373 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 180) took 1113 ms. Trace:
I20260709 13:11:49.098826 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.099854 23985 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:49.064702 24286 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:49.107358 22836 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:49.041184 22761 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.060954 24329 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.068542 22910 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:49.067615 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:49.077775 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.229852 22840 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:49.158071 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.268777 22762 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3. Re-sending same reply.
I20260709 13:11:49.289155 23466 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.163081 23997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1db22a7176432c86e7264f87984ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:49.074505 24218 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.294023 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.114203 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:49.075703 24303 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:49.192867 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.074501 24300 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.300714 24303 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.300989 24300 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.297094 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.301801 22764 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.302024 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.105938 24285 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.303117 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:49.299578 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 102894us
W20260709 13:11:49.297895 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 183814us
I20260709 13:11:49.165202 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.105684 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.165952 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.188651 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:49.191427 24353 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:11:49.153333 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:49.306115 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 253060us
I20260709 13:11:49.313478 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:49.133276 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:49.313580 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 228083us
I20260709 13:11:49.103482 24317 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.122200 22906 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:49.190160 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.319416 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.202309 24213 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.316495 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:49.319993 24338 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.213593 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.294020 24191 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:49.297235 24218 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.323658 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.306041 24339 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:49.330037 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 946) took 1191 ms. Trace:
W20260709 13:11:49.325243 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 192117us
I20260709 13:11:49.355317 22766 rpcz_store.cc:276] 0709 13:11:48.138920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.143714 (+1004794us) service_pool.cc:224] Handling call
0709 13:11:49.329970 (+186256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.354388 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:49.345474 24338 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.333897 24181 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:49.331496 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.331694 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.363408 22906 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:49.364266 22907 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:49.356105 24339 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:49.308372 24331 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:49.303203 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1754) took 1296 ms. Trace:
I20260709 13:11:49.365480 22761 rpcz_store.cc:276] 0709 13:11:48.000934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.882954 (+882020us) service_pool.cc:224] Handling call
0709 13:11:49.296911 (+413957us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.366600 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.366037 24331 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.367244 22761 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:49.368835 24196 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:49.266605 22760 rpcz_store.cc:276] 0709 13:11:48.041715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.587172 (+545457us) service_pool.cc:224] Handling call
0709 13:11:49.045546 (+458374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.293946 24286 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:49.344353 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.369956 22757 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.347451 24132 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.348088 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:49.389909 24132 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.390136 22834 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.390374 24132 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.348768 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.392421 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.390147 24086 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.360337 24338 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.349831 24191 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.349828 22913 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:49.323127 22835 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.357900 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:49.358103 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.330559 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.394344 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.359550 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.362270 22762 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.267882 23459 rpcz_store.cc:276] 0709 13:11:47.970479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.997741 (+ 27262us) service_pool.cc:224] Handling call
0709 13:11:48.770945 (+773204us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:48.771201 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.046224 (+275023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.083741 (+ 37517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38599936}
W20260709 13:11:49.363330 22763 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.580s	user 0.015s	sys 0.008s
I20260709 13:11:49.364849 24339 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.416626 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.365267 24213 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:49.368957 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.305369 22832 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.336357 24352 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.346719 22764 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:49.429185 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1760) took 1420 ms. Trace:
I20260709 13:11:49.429594 22764 rpcz_store.cc:276] 0709 13:11:48.008959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.981168 (+972209us) service_pool.cc:224] Handling call
0709 13:11:49.296962 (+315794us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:49.428892 (+131930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5084544}
I20260709 13:11:49.338033 24329 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:11:49.371515 24184 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.602s	user 0.020s	sys 0.000s
I20260709 13:11:49.378736 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.434616 24184 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.441258 22758 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:49.441644 22910 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:49.442090 24086 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:49.442463 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.378887 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.385803 24173 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.388698 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.361415 24181 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:49.349431 24131 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.006s
I20260709 13:11:49.357045 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.402624 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:49.450796 22766 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:49.450902 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48369.4us
I20260709 13:11:49.451872 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.377346 24286 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.452698 22766 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.430588 24337 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.408445 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.454413 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.454654 24337 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:49.410221 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 79708.7us
I20260709 13:11:49.385063 24289 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.417590 22840 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.420018 24356 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.399588 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.428220 24352 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:49.450773 24181 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.466570 24352 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.431924 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.432205 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.424744 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.422766 24357 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.416707 24333 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:11:49.463817 24351 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:11:49.443214 22915 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.442675 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.445971 22912 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:49.422011 22911 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:49.448132 24184 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:49.420275 24213 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.414101 22763 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:49.451938 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:49.410266 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.409667 24318 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:11:49.455981 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.444525 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:49.458545 24321 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.680s	user 0.015s	sys 0.005s
I20260709 13:11:49.469246 24132 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.491303 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.457103 24131 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:49.437012 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.455879 22906 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.578052 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.585168 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:49.587198 22766 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:49.588188 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 136325us
I20260709 13:11:49.457216 24086 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:49.579479 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.460831 24356 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.503867 22758 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:49.594653 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:49.567907 24053 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.008s
I20260709 13:11:49.592564 24155 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:49.595446 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.638283 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.523392 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:49.568804 24355 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
W20260709 13:11:49.571956 24290 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 13:11:49.635905 22840 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:49.632684 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53351.4us
I20260709 13:11:49.643834 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:49.514840 22761 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:49.639812 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.591291 24213 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.641424 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.650384 22837 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.650520 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.645237 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.645583 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.645716 24337 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.646688 22764 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.646581 24132 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.638331 24066 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:49.647547 24310 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.000s
I20260709 13:11:49.591822 24173 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.634451 24289 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.652252 22766 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:49.638055 24307 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:49.492334 24298 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.654832 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.638363 24086 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:49.656191 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.658005 22914 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:49.634766 24356 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:49.595105 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 948) took 1290 ms (client timeout 1522 ms). Trace:
I20260709 13:11:49.657312 24333 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.642297 24155 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.595005 22910 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.658607 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.660647 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.690550 22840 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.655468 24337 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:49.544880 24357 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.664505 24352 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.638932 24053 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.718786 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:49.719636 23457 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Advancing to term 2
I20260709 13:11:49.686965 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.720795 24053 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.654009 22757 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:49.722035 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.722289 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.722589 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.723106 24331 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:49.723237 24211 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:49.723815 24331 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:49.724154 24196 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:49.724944 24196 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:49.656226 24066 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:49.725081 24331 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.725771 24066 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.725970 24196 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.723824 24211 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:49.728003 24211 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.729568 24211 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.723498 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.685811 24333 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:49.694322 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.698439 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.697878 24086 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.737535 22913 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.700354 24337 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.707509 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.708276 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.707567 24357 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.744230 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.745039 22910 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.745893 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:49.746786 22838 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.747663 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.709398 22664 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 242) took 7048 ms. Trace:
I20260709 13:11:49.722429 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.702006 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.712926 24352 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.710393 24087 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:49.714962 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.753708 22839 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.759205 22765 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:49.759783 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1748) took 1760 ms (client timeout 2223 ms). Trace:
I20260709 13:11:49.718178 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.762563 22832 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:49.763686 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.764333 22832 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.679922 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 930) took 1638 ms. Trace:
I20260709 13:11:49.765651 22766 rpcz_store.cc:276] 0709 13:11:48.041367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.667801 (+1626434us) service_pool.cc:224] Handling call
0709 13:11:49.678957 (+ 11156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.679284 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.679324 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.679391 (+    67us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:49.679438 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.679513 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.679816 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.767067 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.767658 22766 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.768622 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.768085 24338 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.667501 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.770531 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.771199 22906 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.772162 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.773072 22906 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:49.732689 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.731035 22914 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.735714 24303 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.776005 24303 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:49.776639 24303 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:49.776815 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.734639 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.777738 22757 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.778239 22760 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:49.778879 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.779299 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.780272 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.780609 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.781121 22757 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.689783 24155 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.684209 24191 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:49.786015 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.686139 24307 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.789170 24307 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.709944 24317 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:49.799515 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:49.689563 24173 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.751386 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.721432 24053 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.759994 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.807178 22836 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.808199 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.808998 22836 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.809187 24132 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.809906 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.809782 24132 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.765350 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.810984 22832 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.812040 24132 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.799361 24317 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:49.812505 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.813849 24317 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.814491 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.719767 22841 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.655319 24300 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:49.760185 22765 rpcz_store.cc:269] 0709 13:11:47.998983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.714686 (+1715703us) service_pool.cc:224] Handling call
0709 13:11:49.759724 (+ 45038us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.660211 24310 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.760499 24087 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:49.745976 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.738533 24333 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.777002 24303 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.791808 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.750308 22664 rpcz_store.cc:276] 0709 13:11:42.660812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.662013 (+  1201us) service_pool.cc:224] Handling call
0709 13:11:49.709307 (+7047294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2880557,"reported_tablets":5}
I20260709 13:11:49.792984 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:49.769429 22766 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:49.736510 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.732239 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:49.875903 22837 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.881119 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:49.832489 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.882380 22908 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:49.883239 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.793888 22660 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 2, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.796252 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.689455 24286 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.797828 22761 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:49.686102 22763 rpcz_store.cc:269] 0709 13:11:48.305224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.577831 (+272607us) service_pool.cc:224] Handling call
0709 13:11:49.593438 (+1015607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.895797 24310 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.896454 24310 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:49.817364 24300 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.806932 24053 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:49.896931 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:49.897332 24053 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.897634 22913 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.898033 24286 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.806355 24173 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:49.898483 24339 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.899010 24173 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.890340 24191 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:49.900678 24339 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.807225 24285 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.901688 24338 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.818547 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.749121 24181 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:49.861714 24298 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.760100 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.942935 22839 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.818974 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.820047 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 936) took 1776 ms. Trace:
I20260709 13:11:49.885629 24196 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.815745 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:49.884420 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:49.876295 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83401.1us
I20260709 13:11:49.886574 22759 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.950227 22908 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:49.950461 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 952) took 1521 ms (client timeout 1766 ms). Trace:
W20260709 13:11:49.950845 22759 rpcz_store.cc:269] 0709 13:11:48.428788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.791120 (+1362332us) service_pool.cc:224] Handling call
0709 13:11:49.950397 (+159277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.951956 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.952675 22759 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:49.887342 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.885396 22660 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.953899 22837 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:49.889062 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 941) took 1837 ms. Trace:
I20260709 13:11:49.954807 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.955561 22837 raft_consensus.cc:2368] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:49.954928 22660 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.956375 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:49.956943 22837 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.957160 22660 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 2, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:49.887305 22732 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.728s (SENT)
I20260709 13:11:49.959183 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:49.806828 24289 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.960702 24289 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.961275 24289 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.962044 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.963333 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.963886 23463 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:49.964865 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.964910 24318 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:49.815426 24317 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:49.891000 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 158834us
I20260709 13:11:49.966885 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:11:49.952692 22834 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.010s
I20260709 13:11:49.892539 22835 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:49.967991 22834 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:49.893568 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1759) took 1889 ms. Trace:
I20260709 13:11:49.968587 22764 rpcz_store.cc:276] 0709 13:11:48.003843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.430537 (+1426694us) service_pool.cc:224] Handling call
0709 13:11:49.893355 (+462818us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:49.893488 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404980480}
I20260709 13:11:49.970232 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.893916 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 943) took 1798 ms. Trace:
I20260709 13:11:49.971194 22761 rpcz_store.cc:276] 0709 13:11:48.095253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.892899 (+1797646us) service_pool.cc:224] Handling call
0709 13:11:49.893273 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.893484 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.893545 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.893601 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:49.893639 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.893703 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.893846 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:49.800402 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.973660 23458 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:49.953488 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.974382 23997 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40710 (request call id 174) took 6059 ms. Trace:
I20260709 13:11:49.974591 22759 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.974805 23997 rpcz_store.cc:276] 0709 13:11:43.881126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.881347 (+   221us) service_pool.cc:224] Handling call
0709 13:11:44.329227 (+447880us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cd1a2 00007f68a790ebb9 00007f68a6a3fd88 00007f68a6b03a07 00007f68a6b35f4e 00007f68a6b1afdd 00007f68a6b1af89 00007f68a6b1af19 00007f68a6b1aee1
0709 13:11:44.517583 (+188356us) ts_tablet_manager.cc:934] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
0709 13:11:49.069760 (+4552177us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:11:49.452777 (+383017us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:49.452932 (+   155us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:11:49.864656 (+411724us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:11:49.865089 (+   433us) tablet_replica.cc:206] Creating consensus instance
0709 13:11:49.879347 (+ 14258us) ts_tablet_manager.cc:989] Replica registered.
0709 13:11:49.940016 (+ 60669us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":66678,"spinlock_wait_cycles":11828096,"tablet-copy.queue_time_us":145072,"thread_start_us":640244,"threads_started":2}
I20260709 13:11:49.976581 23997 tablet_copy_client.cc:806] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:49.977185 23997 tablet_copy_client.cc:670] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:49.903527 23466 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:49.880074 24303 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.979375 24181 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:49.882540 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.981904 23459 raft_consensus.cc:2376] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3. Re-sending same reply.
I20260709 13:11:49.983273 23457 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:49.984006 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 372) took 1124 ms. Trace:
I20260709 13:11:49.920253 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:49.984861 22912 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.986814 24087 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.988338 24087 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.925832 22913 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:49.818186 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.989903 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:49.781339 22760 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:49.990625 23462 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:49.991037 22913 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:11:49.991484 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.843593 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.814711 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.944047 22765 rpcz_store.cc:276] 0709 13:11:48.043297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.819367 (+1776070us) service_pool.cc:224] Handling call
0709 13:11:49.819670 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.819826 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.819843 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.819876 (+    33us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:49.819894 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.819929 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.820004 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:49.992285 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:49.992221 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:49.993089 22910 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.945101 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:49.942154 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.946744 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.013741 22839 raft_consensus.cc:2368] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:50.014616 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.015594 22839 raft_consensus.cc:2368] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:50.015666 22837 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:50.016690 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.969014 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.983424 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:49.935957 22879 connection.cc:256] RPC call timeout handler was delayed by -4.4599402129999817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 13:11:49.990130 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.933086 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.032284 22915 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.993140 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.033313 22765 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:50.034806 23460 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.825891 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:11:50.036460 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 210637us
I20260709 13:11:50.037705 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:49.992426 23464 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:49.991273 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:49.901861 24066 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:49.994928 24337 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.054818 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1062) took 1041 ms. Trace:
W20260709 13:11:49.996640 24218 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.003s
I20260709 13:11:50.055521 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 960) took 1094 ms. Trace:
I20260709 13:11:50.056151 24218 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.056095 22760 rpcz_store.cc:276] 0709 13:11:48.961477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.054569 (+1093092us) service_pool.cc:224] Handling call
0709 13:11:50.054933 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.055139 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.055175 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.055228 (+    53us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:50.055270 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.055337 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.055458 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.041930 24213 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.057857 22841 raft_consensus.cc:2368] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4.
I20260709 13:11:50.058449 24218 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.058341 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:50.058966 24213 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:49.999037 24131 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:50.059777 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:50.059913 24320 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:50.060431 23464 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:50.060535 24320 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:50.061148 24320 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.002386 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.003253 24331 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.934652 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:49.918286 24357 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.010301 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:49.955225 22766 rpcz_store.cc:276] 0709 13:11:48.051326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.888352 (+1837026us) service_pool.cc:224] Handling call
0709 13:11:49.888712 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.888845 (+   133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.888860 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.888891 (+    31us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:49.888909 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.888943 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.889016 (+    73us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:49.960438 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73185.7us
I20260709 13:11:50.061400 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:50.084255 22833 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 1.
I20260709 13:11:50.085706 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.086133 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:11:50.086949 22732 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.758s (SENT)
I20260709 13:11:50.087612 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:50.087956 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:50.088371 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86087us
I20260709 13:11:50.088792 23464 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 1.
I20260709 13:11:50.019078 22912 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:49.960381 24155 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.099074 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.100020 22912 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:50.012481 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.026216 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.026400 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.017377 22839 raft_consensus.cc:2368] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
W20260709 13:11:50.108687 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.029548 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:50.030392 22879 connection.cc:256] RPC call timeout handler was delayed by -2.9646989990000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
I20260709 13:11:50.109262 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:50.109529 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8531989990000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3128s for a response
I20260709 13:11:50.038365 22909 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.109935 22912 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:50.043232 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:49.985349 24285 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
W20260709 13:11:50.047246 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 262602us
W20260709 13:11:50.139905 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.105585 23460 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:50.047444 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.047636 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:50.037997 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:50.127321 24318 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.054157 22840 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.000552 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.089295 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:50.067605 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:50.089664 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:50.089839 24331 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.129709 24155 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:50.219501 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 130280us
I20260709 13:11:50.220633 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:50.222115 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.222726 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 957) took 1570 ms. Trace:
I20260709 13:11:50.074425 24289 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.223209 22765 rpcz_store.cc:276] 0709 13:11:48.652355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.033966 (+1381611us) service_pool.cc:224] Handling call
0709 13:11:50.034332 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.034699 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.034730 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:50.035269 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.133259 (+ 97990us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:50.133301 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.133378 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.222495 (+ 89117us) raft_consensus.cc:1685] finished
0709 13:11:50.222520 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.222661 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6932,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7874,"thread_start_us":97521,"threads_started":1,"wal-append.queue_time_us":185297}
I20260709 13:11:50.091662 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1762) took 1046 ms. Trace:
I20260709 13:11:50.226348 24289 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:50.064685 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.226802 24317 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.227229 22907 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:50.227717 24317 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.228551 24317 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.063122 22760 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.229766 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1761) took 1184 ms. Trace:
I20260709 13:11:50.055282 22911 rpcz_store.cc:276] 0709 13:11:49.016685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.019465 (+  2780us) service_pool.cc:224] Handling call
0709 13:11:49.191341 (+171876us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:49.191574 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.592624 (+401050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.048903 (+456279us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54003c87d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:50.054746 (+  5843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493468416}
I20260709 13:11:50.230530 24218 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.231287 24218 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.231761 24218 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.232000 22911 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.095782 24053 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.050678 22915 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:50.102288 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89898.5us
I20260709 13:11:50.235272 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.018040 22759 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:50.087132 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.124456 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:50.236004 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 354948us
W20260709 13:11:50.237088 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 112624us
I20260709 13:11:50.126374 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.238200 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.130184 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.111521 22837 raft_consensus.cc:2368] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3.
I20260709 13:11:50.240197 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.141706 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:50.241163 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99700.6us
I20260709 13:11:50.056272 24337 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:50.148206 22910 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:50.148252 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:50.149781 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 102218us
I20260709 13:11:50.053953 24066 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.108844 24213 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:50.147413 24191 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.090687 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.195489 22834 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:11:50.192828 24356 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.001s
I20260709 13:11:50.196331 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.254873 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.315375 22913 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.219440 24330 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.319932 22836 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.153812 24339 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.213410 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.321957 22914 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:50.214169 23466 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.217041 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 87) took 4190 ms. Trace:
I20260709 13:11:50.219344 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:50.091158 24357 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.230186 22760 rpcz_store.cc:276] 0709 13:11:49.045311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.062579 (+1017268us) service_pool.cc:224] Handling call
0709 13:11:50.062923 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.229396 (+166473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.229679 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.331481 24357 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.223806 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:50.334199 24337 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:50.335367 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111665us
I20260709 13:11:50.335994 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.250877 24134 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.116420 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.225715 22766 rpcz_store.cc:276] 0709 13:11:49.045658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.090678 (+1045020us) service_pool.cc:224] Handling call
0709 13:11:50.090985 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.091301 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.091334 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.091382 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:50.091413 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.091476 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.091606 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":346,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":121}
W20260709 13:11:50.342600 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 226313us
W20260709 13:11:50.145985 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.234483 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.346019 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:50.246294 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:50.347245 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 101124us
I20260709 13:11:50.348812 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.248837 22909 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:50.241745 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.317206 24132 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.250669 22912 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:11:50.326689 24361 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:11:50.015074 23457 rpcz_store.cc:276] 0709 13:11:48.859709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.862992 (+  3283us) service_pool.cc:224] Handling call
0709 13:11:49.719241 (+856249us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:49.719478 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.983727 (+264249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.983948 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6234368}
I20260709 13:11:50.254972 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:11:50.313037 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 75069.1us
W20260709 13:11:50.364300 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 109409us
I20260709 13:11:50.364540 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.253720 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.238440 24173 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.240859 24318 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.318759 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.229019 22907 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.322371 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.322834 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.324630 22915 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.398710 22759 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.238652 24331 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:50.325474 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 106234us
I20260709 13:11:50.244491 24155 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:50.327756 22758 rpcz_store.cc:276] 0709 13:11:46.026779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.659821 (+3633042us) service_pool.cc:224] Handling call
0709 13:11:49.661408 (+  1587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.662214 (+   806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.662248 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.662935 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.889788 (+226853us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:50.146901 (+257113us) log.cc:844] Serialized 14 byte log entry
0709 13:11:50.193389 (+ 46488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.216743 (+ 23354us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.216807 (+    64us) raft_consensus.cc:1685] finished
0709 13:11:50.216820 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.216975 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7411,"prepare.queue_time_us":149574,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":2628,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":83560,"thread_start_us":367531,"threads_started":2,"wal-append.queue_time_us":207009}
I20260709 13:11:50.346189 22765 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.242487 22762 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.249717 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:50.409982 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 160338us
I20260709 13:11:50.351095 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:50.413821 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62856.9us
I20260709 13:11:50.414845 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.351940 22833 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.334810 24337 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:11:50.419003 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 54566us
I20260709 13:11:50.419826 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1066) took 1033 ms. Trace:
I20260709 13:11:50.314368 24213 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.159173 22908 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.420428 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.320207 24330 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.202883 23461 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:50.378718 22840 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.378691 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:50.338183 24196 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:50.366559 22835 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.389210 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.503417 23462 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:50.340790 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1032) took 1196 ms. Trace:
I20260709 13:11:50.355228 24298 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:50.356238 24297 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.319343 24191 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.323530 24356 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.355635 24131 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:50.365744 24338 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.361522 24285 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:50.362612 24289 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.363248 24132 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.389227 24318 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.381176 24320 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.405849 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1067) took 1019 ms. Trace:
I20260709 13:11:50.588366 22841 rpcz_store.cc:276] 0709 13:11:49.386663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.322020 (+935357us) service_pool.cc:224] Handling call
0709 13:11:50.405748 (+ 83728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.378845 24066 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.379498 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 938) took 2327 ms. Trace:
W20260709 13:11:50.594638 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 250877us
I20260709 13:11:50.354257 22913 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:50.596403 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1823) took 1165 ms. Trace:
I20260709 13:11:50.418586 23430 leader_election.cc:400] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3.
W20260709 13:11:50.338374 24211 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.004s
I20260709 13:11:50.420288 22833 rpcz_store.cc:276] 0709 13:11:49.386393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.322877 (+936484us) service_pool.cc:224] Handling call
0709 13:11:50.419745 (+ 96868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.409190 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1093) took 1016 ms. Trace:
I20260709 13:11:50.458309 24330 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:50.570082 24333 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.002s
I20260709 13:11:50.586812 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 964) took 1492 ms. Trace:
I20260709 13:11:50.493494 22765 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:50.422713 24213 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:50.495510 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 253809us
I20260709 13:11:50.496459 22759 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
W20260709 13:11:50.508116 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 87776.9us
I20260709 13:11:50.499648 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 91) took 2121 ms. Trace:
I20260709 13:11:50.506929 24365 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:50.525094 24338 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:11:50.526780 24367 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:11:50.537632 24376 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:50.525964 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.508790 24297 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.524937 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.664534 22840 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:11:50.664801 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 286273us
I20260709 13:11:50.665031 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1068) took 1278 ms. Trace:
I20260709 13:11:50.545987 24053 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.574072 22832 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.571949 24132 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.671178 22759 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:50.575048 24191 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:50.674015 24191 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:50.505270 22834 rpcz_store.cc:276] 0709 13:11:49.307226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.026113 (+718887us) service_pool.cc:224] Handling call
0709 13:11:50.326956 (+300843us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.676455 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.668236 24297 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.678293 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.678416 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.679312 22834 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.679796 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1069) took 1292 ms. Trace:
I20260709 13:11:50.581945 24173 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.585032 24339 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:50.584553 24337 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.659026 24181 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.695973 24181 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.680313 22835 rpcz_store.cc:276] 0709 13:11:49.387267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.678038 (+1290771us) service_pool.cc:224] Handling call
0709 13:11:50.679714 (+  1676us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:50.597944 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:50.698329 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:50.698459 24364 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.664s	user 0.004s	sys 0.001s
I20260709 13:11:50.698925 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.699822 22835 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:50.597241 23430 leader_election.cc:403] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:50.719071 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1045) took 1338 ms. Trace:
W20260709 13:11:50.719779 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 317587us
I20260709 13:11:50.566932 24289 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.721757 24289 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.566200 24379 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:50.580610 24357 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.724470 24357 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.601024 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:50.599332 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 186641us
I20260709 13:11:50.726248 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.601987 22764 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:50.727443 24378 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:50.727679 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:50.725699 23466 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:11:50.638329 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1062) took 1253 ms. Trace:
I20260709 13:11:50.730161 24286 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3.
I20260709 13:11:50.730290 24388 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.636897 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 92) took 2261 ms. Trace:
I20260709 13:11:50.731645 24388 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:50.731874 22762 rpcz_store.cc:276] 0709 13:11:48.375163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.009895 (+1634732us) service_pool.cc:224] Handling call
0709 13:11:50.236370 (+226475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.420573 (+184203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.636275 (+215702us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54002233d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:50.636613 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519925120}
I20260709 13:11:50.732524 24388 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.601575 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.599993 24155 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:50.733705 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 963) took 1640 ms. Trace:
I20260709 13:11:50.648372 24320 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:50.609176 24213 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.734146 22766 rpcz_store.cc:276] 0709 13:11:49.093325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.342671 (+1249346us) service_pool.cc:224] Handling call
0709 13:11:50.343030 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.392021 (+ 48991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.392072 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:50.394738 (+  2666us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.593419 (+198681us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:50.593485 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.593642 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.733428 (+139786us) raft_consensus.cc:1685] finished
0709 13:11:50.733453 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.733640 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58030,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":50804,"thread_start_us":180265,"threads_started":2,"wal-append.queue_time_us":321317}
I20260709 13:11:50.604163 24333 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.735967 24289 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.737277 22838 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:11:50.603852 24303 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.002s
I20260709 13:11:50.730968 22836 rpcz_store.cc:276] 0709 13:11:49.385250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.809654 (+424404us) service_pool.cc:224] Handling call
0709 13:11:50.638000 (+828346us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54000a5854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:50.638237 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171133312}
I20260709 13:11:50.649446 24356 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:50.606738 24368 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.192s	user 0.002s	sys 0.000s
I20260709 13:11:50.738715 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.605958 22758 rpcz_store.cc:276] 0709 13:11:49.094130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.459145 (+1365015us) service_pool.cc:224] Handling call
0709 13:11:50.571523 (+112378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.586197 (+ 14674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.586249 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.586332 (+    83us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:50.586387 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.586489 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.586707 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.600960 24211 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:11:50.739470 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1063) took 1354 ms. Trace:
I20260709 13:11:50.739812 22836 rpcz_store.cc:276] 0709 13:11:49.385560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.738361 (+1352801us) service_pool.cc:224] Handling call
0709 13:11:50.739412 (+  1051us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.737980 24303 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.740638 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.719542 22835 rpcz_store.cc:276] 0709 13:11:49.380637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.698141 (+1317504us) service_pool.cc:224] Handling call
0709 13:11:50.718979 (+ 20838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.741338 22836 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.655783 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 965) took 1299 ms. Trace:
I20260709 13:11:50.595341 24066 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.758846 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1037) took 1385 ms. Trace:
I20260709 13:11:50.759203 24066 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.760545 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1048) took 1379 ms. Trace:
I20260709 13:11:50.597772 24134 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:50.661886 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 247137us
I20260709 13:11:50.761745 24134 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.604553 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.665460 22840 rpcz_store.cc:276] 0709 13:11:49.386920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.512220 (+1125300us) service_pool.cc:224] Handling call
0709 13:11:50.664959 (+152739us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.764783 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.765570 22840 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.765965 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1050) took 1383 ms. Trace:
I20260709 13:11:50.766270 22840 rpcz_store.cc:276] 0709 13:11:49.382121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.764454 (+1382333us) service_pool.cc:224] Handling call
0709 13:11:50.765907 (+  1453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.767156 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.768232 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.769143 22840 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.665828 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:50.658983 24338 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.770442 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.771204 22840 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.772650 22840 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.392904 22911 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:50.773208 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1028) took 1687 ms. Trace:
I20260709 13:11:50.759262 22836 rpcz_store.cc:276] 0709 13:11:49.373821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.740400 (+1366579us) service_pool.cc:224] Handling call
0709 13:11:50.758774 (+ 18374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.774791 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1812) took 1378 ms. Trace:
I20260709 13:11:50.775194 22836 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:50.685910 24053 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:50.776418 24053 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.602201 22907 rpcz_store.cc:276] 0709 13:11:49.391143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.228185 (+837042us) service_pool.cc:224] Handling call
0709 13:11:50.228581 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.389052 (+160471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.403276 (+ 14224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.689860 23464 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:50.689437 24218 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.777720 22840 rpcz_store.cc:276] 0709 13:11:49.085371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.772196 (+1686825us) service_pool.cc:224] Handling call
0709 13:11:50.772493 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.772995 (+   502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.773159 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.779891 24218 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.690279 24173 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:50.691066 24383 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.780706 24383 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.781208 22839 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:50.756867 24213 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.782363 24053 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:50.782902 24383 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.775295 22911 rpcz_store.cc:276] 0709 13:11:49.396561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.243563 (+847002us) service_pool.cc:224] Handling call
0709 13:11:50.362432 (+118869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.774228 (+411796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.774536 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.783346 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 106) took 1453 ms. Trace:
I20260709 13:11:50.783897 22833 raft_consensus.cc:2368] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:50.784124 24213 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:50.691788 24087 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.703s	user 0.014s	sys 0.000s
I20260709 13:11:50.783691 22839 rpcz_store.cc:276] 0709 13:11:49.330223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.782570 (+1452347us) service_pool.cc:224] Handling call
0709 13:11:50.782840 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.783028 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.783057 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.783101 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:50.783130 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.783188 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.783298 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.605134 24317 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:50.694684 24374 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:11:50.786602 22840 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.787367 22833 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.695261 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 949) took 2346 ms. Trace:
I20260709 13:11:50.662029 24376 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.788388 24376 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.788182 22763 rpcz_store.cc:276] 0709 13:11:48.349404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.894620 (+1545216us) service_pool.cc:224] Handling call
0709 13:11:49.894991 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.895371 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.895408 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.896003 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.034686 (+138683us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:50.034727 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.034809 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.686240 (+651431us) raft_consensus.cc:1685] finished
0709 13:11:50.686262 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.695192 (+  8930us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7788,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":28573,"thread_start_us":133541,"threads_started":1,"wal-append.queue_time_us":242658}
I20260709 13:11:50.790040 24388 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.792312 24317 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.596971 22913 rpcz_store.cc:276] 0709 13:11:49.431184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.324258 (+893074us) service_pool.cc:224] Handling call
0709 13:11:50.353748 (+ 29490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.595949 (+242201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.596319 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.794628 24066 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.796116 22841 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.797364 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1034) took 1472 ms. Trace:
I20260709 13:11:50.797796 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1041) took 1418 ms. Trace:
I20260709 13:11:50.797757 22841 rpcz_store.cc:276] 0709 13:11:49.325049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.781898 (+1456849us) service_pool.cc:224] Handling call
0709 13:11:50.782226 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.796530 (+ 14304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.797106 (+   576us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54003205d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:50.797297 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31482752}
I20260709 13:11:50.798503 22833 rpcz_store.cc:276] 0709 13:11:49.379619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.786461 (+1406842us) service_pool.cc:224] Handling call
0709 13:11:50.786832 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.795064 (+  8232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.797716 (+  2652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.799643 22841 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.800310 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 108) took 1146 ms. Trace:
I20260709 13:11:50.715528 24181 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:50.717386 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1817) took 1317 ms. Trace:
I20260709 13:11:50.800988 24333 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:50.670074 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 116225us
I20260709 13:11:50.595073 22757 rpcz_store.cc:276] 0709 13:11:48.043876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.874897 (+1831021us) service_pool.cc:224] Handling call
0709 13:11:49.893483 (+ 18586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.893752 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:49.951344 (+ 57592us) log.cc:844] Serialized 14 byte log entry
0709 13:11:50.320048 (+368704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.326442 (+  6394us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:50.337258 (+ 10816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.350015 (+ 12757us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.371600 (+ 21585us) inbound_call.cc:177] Queueing success response
0709 13:11:50.606172 (+234572us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
Metrics: {"mutex_wait_us":51307,"prepare.queue_time_us":139209,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2700,"raft.queue_time_us":57402,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":10640,"spinlock_wait_cycles":15830912,"thread_start_us":353084,"threads_started":2,"wal-append.queue_time_us":442635}
I20260709 13:11:50.728693 24378 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.802137 24378 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.801229 22910 rpcz_store.cc:276] 0709 13:11:49.399920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.243810 (+843890us) service_pool.cc:224] Handling call
0709 13:11:50.244334 (+   524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.496171 (+251837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.496206 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:50.502625 (+  6419us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.595520 (+ 92895us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:50.595561 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.665655 (+ 70094us) spinlock_profiling.cc:243] Waited 93.7 ms on lock 0x7b54003c96d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:50.665725 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.717035 (+ 51310us) raft_consensus.cc:1685] finished
0709 13:11:50.717067 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.717317 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119451,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":10891,"spinlock_wait_cycles":215428352,"thread_start_us":228576,"threads_started":2,"wal-append.queue_time_us":188651}
I20260709 13:11:50.662853 24386 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:50.598528 24381 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:50.741894 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.804028 22835 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:50.804462 24381 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.804544 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1040) took 1425 ms. Trace:
I20260709 13:11:50.742381 24303 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.805174 24381 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.631472 22761 rpcz_store.cc:276] 0709 13:11:48.374813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.972801 (+1597988us) service_pool.cc:224] Handling call
0709 13:11:49.973304 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.004957 (+ 31653us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:50.071811 (+ 66854us) log.cc:844] Serialized 14 byte log entry
0709 13:11:50.459738 (+387927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.468603 (+  8865us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:50.470338 (+  1735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.476126 (+  5788us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.495965 (+ 19839us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65224,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":55721,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":61567,"spinlock_wait_cycles":26310912,"thread_start_us":318591,"threads_started":3,"wal-append.queue_time_us":177579}
I20260709 13:11:50.744964 24132 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.749220 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1049) took 1367 ms. Trace:
I20260709 13:11:50.746632 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:50.812455 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 146731us
W20260709 13:11:50.812813 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 66251.6us
I20260709 13:11:50.813548 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:50.805168 22835 rpcz_store.cc:276] 0709 13:11:49.379315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.741563 (+1362248us) service_pool.cc:224] Handling call
0709 13:11:50.804468 (+ 62905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.815913 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.817368 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:50.817960 24173 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:50.752300 24289 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.818615 24173 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:50.818930 24289 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:50.819367 24289 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.819799 24384 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:50.819958 22760 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.819689 24173 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.821110 24389 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:50.821491 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 962) took 1729 ms. Trace:
I20260709 13:11:50.822556 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:50.776953 24134 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.792351 24388 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:50.760938 22832 rpcz_store.cc:276] 0709 13:11:49.381383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.814184 (+432801us) service_pool.cc:224] Handling call
0709 13:11:50.760332 (+946148us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54001bc454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:50.760479 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221030784}
I20260709 13:11:50.758692 22765 rpcz_store.cc:276] 0709 13:11:49.356180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.608511 (+1252331us) service_pool.cc:224] Handling call
0709 13:11:50.653016 (+ 44505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.655243 (+  2227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.655283 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.655357 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:50.655408 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.655484 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.655704 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.775889 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1029) took 1690 ms. Trace:
I20260709 13:11:50.846338 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.787858 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1031) took 1690 ms. Trace:
I20260709 13:11:50.826500 24053 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:50.680315 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.807945 22841 rpcz_store.cc:276] 0709 13:11:49.653476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.798992 (+1145516us) service_pool.cc:224] Handling call
0709 13:11:50.799456 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.800048 (+   592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.800260 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.565991 24298 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:50.817860 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:50.797240 24066 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.749676 24331 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:50.840821 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 241679us
I20260709 13:11:50.830952 24132 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:50.819797 24289 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.847647 22832 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
W20260709 13:11:50.849089 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 219184us
I20260709 13:11:50.871308 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.872380 22757 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3.
I20260709 13:11:50.873464 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.812259 22838 rpcz_store.cc:276] 0709 13:11:49.381745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.747403 (+365658us) service_pool.cc:224] Handling call
0709 13:11:50.748979 (+1001576us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54000a5354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:50.749153 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274200448}
I20260709 13:11:50.878804 22838 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:50.862849 24213 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:50.879909 24213 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:50.880661 22838 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.882380 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1044) took 1502 ms. Trace:
I20260709 13:11:50.882885 22834 rpcz_store.cc:276] 0709 13:11:49.380376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.680009 (+1299633us) service_pool.cc:224] Handling call
0709 13:11:50.882300 (+202291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.884364 22834 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.859269 24298 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:50.887676 24298 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:50.756100 24330 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.849241 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.848196 22840 rpcz_store.cc:276] 0709 13:11:49.098001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.778783 (+1680782us) service_pool.cc:224] Handling call
0709 13:11:50.779084 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.786996 (+  7912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.787523 (+   527us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b540019ebd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:50.787795 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16646528}
W20260709 13:11:50.849874 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 197159us
I20260709 13:11:50.846144 22836 rpcz_store.cc:276] 0709 13:11:49.085651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.774659 (+1689008us) service_pool.cc:224] Handling call
0709 13:11:50.774975 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.775651 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.775837 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.800118 22833 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.831313 24181 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:11:50.852607 22763 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:50.854482 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:50.907310 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52955.9us
I20260709 13:11:50.855229 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:50.856776 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:50.800180 24297 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.859543 23466 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.842947 24318 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.864357 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:50.868705 22841 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "072f69118119423dac7eba3f66879eee"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:11:50.870501 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:50.914218 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.914605 22840 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:50.915891 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:50.916337 22840 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.853291 24053 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:50.900883 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:50.904649 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 972) took 1024 ms. Trace:
I20260709 13:11:50.917994 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.918272 22833 rpcz_store.cc:276] 0709 13:11:49.880570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.799527 (+918957us) service_pool.cc:224] Handling call
0709 13:11:50.799873 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.904365 (+104492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.904585 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:50.920051 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.911943 22764 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:50.920637 22915 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.921175 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.921782 22910 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.912638 22841 raft_consensus.cc:606] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:11:50.923126 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.917243 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:50.926833 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 71688.1us
I20260709 13:11:50.928103 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:50.930625 22910 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:11:50.919394 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.934651 22914 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:50.822010 22760 rpcz_store.cc:276] 0709 13:11:49.092297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.331769 (+1239472us) service_pool.cc:224] Handling call
0709 13:11:50.332226 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.820482 (+488256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.821214 (+   732us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b540035f054. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:50.821420 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092835584}
I20260709 13:11:50.939980 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:50.936472 22836 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.935508 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.899070 24389 consensus_queue.cc:1243] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:11:50.935971 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:50.871361 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:50.959079 22836 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:50.956667 22913 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.948940 22915 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:50.963814 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68300.4us
I20260709 13:11:50.938539 22833 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:11:50.966135 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.955665 22837 raft_consensus.cc:2368] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
W20260709 13:11:50.941434 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ad66b5c9842437d9320507b92b78fd5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:50.975275 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:50.944567 22840 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.981252 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.983032 23466 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:50.906828 24066 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:50.986438 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 43872.8us
I20260709 13:11:51.009084 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.997309 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:51.005474 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:51.002974 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.020999 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:51.021281 22908 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.024542 22915 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:51.025897 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:51.022121 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86243.4us
I20260709 13:11:51.029860 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.995635 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:51.032328 22840 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.934290 24087 raft_consensus.cc:2764] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:11:50.938557 24380 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:51.033921 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:51.033109 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.961879 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.916788 24132 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.124s
I20260709 13:11:50.942255 24211 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:51.042429 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.988245 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:51.038429 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:50.959645 22838 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.903811 24378 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.881373 24213 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:50.938537 24356 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.047639 23466 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:50.967756 24388 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:11:50.964112 24365 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:51.047904 24286 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:50.955874 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.049429 22914 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.034245 24389 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:11:50.956975 24392 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:50.939103 24391 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:50.960975 24134 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:51.038053 24382 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:11:51.024909 24393 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:11:51.050040 22911 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.962014 24379 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:51.051151 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.100798 22913 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.047753 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:50.912676 24131 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:11:50.973064 24303 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:51.054893 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:50.901878 24337 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:11:51.055941 22841 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:51.058202 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:51.058795 22908 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:51.091637 24380 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:51.032712 22834 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:51.033655 24377 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:50.956627 24383 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
I20260709 13:11:50.938432 24390 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:50.966325 24218 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 13:11:51.068729 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:50.943414 24386 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
I20260709 13:11:51.051903 24086 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:11:51.085290 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.973063 24338 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:50.946723 24384 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.059257 24173 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:51.043545 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 102205us
I20260709 13:11:51.151280 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:51.108567 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 990) took 1020 ms. Trace:
I20260709 13:11:50.956030 24053 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.051188 24285 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:11:51.171037 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:51.172197 22841 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:11:51.098817 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 70545.9us
I20260709 13:11:51.055128 24320 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:11:51.083374 24286 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.107295 22837 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:51.109391 22836 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:50.938740 24357 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.110s
I20260709 13:11:51.088099 24398 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:51.095520 24391 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:51.123299 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84963.4us
I20260709 13:11:51.088519 24211 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:11:51.095763 24392 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:51.098543 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 982) took 1042 ms. Trace:
I20260709 13:11:51.072221 24399 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:11:51.033965 24066 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.106868 24303 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:51.107460 24134 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:11:51.156180 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 126921us
I20260709 13:11:51.213308 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.170678 24331 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.111900 22835 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:51.162971 22832 rpcz_store.cc:276] 0709 13:11:50.085358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.935317 (+849959us) service_pool.cc:224] Handling call
0709 13:11:51.054237 (+118920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:51.073118 (+ 18881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:51.080569 (+  7451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:51.086994 (+  6425us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:51.089324 (+  2330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.098391 (+  9067us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.105372 (+  6981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:51.185094 22911 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:51.185063 22913 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:51.185923 24403 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:51.170926 24086 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:51.189832 22839 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:51.127930 24380 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:51.148841 24390 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:51.150987 24377 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:51.208026 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:51.215200 24383 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.176924 24333 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:51.120345 24388 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:51.190307 24286 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.128657 24376 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.190423 24396 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:11:51.200798 24398 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.199684 24132 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:51.234331 24414 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:51.203374 24391 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.245393 22841 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.175447 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 981) took 1117 ms. Trace:
I20260709 13:11:51.205294 24392 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.207937 24318 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.215631 24339 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.200424 24330 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.222144 24365 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.319556 24390 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.320698 24365 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.284391 24399 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:51.321779 24399 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:51.322930 24399 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.295634 24066 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.231678 24134 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.297303 24298 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.257647 24383 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.298717 24338 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:51.299715 24401 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:11:51.299172 24380 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.253585 24388 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:51.300858 24377 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:51.303364 24394 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260709 13:11:51.235844 24086 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:51.237767 24211 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:51.285017 24398 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.254074 24333 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.305137 24381 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.313858 24303 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.314478 24378 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.356518 24303 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.356388 24388 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.330736 24218 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:11:51.339630 24386 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.496449 24392 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.287973 22834 rpcz_store.cc:276] 0709 13:11:50.052780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.885323 (+832543us) service_pool.cc:224] Handling call
0709 13:11:50.934025 (+ 48702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:51.123911 (+189886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.169806 (+ 45895us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:51.204967 22838 rpcz_store.cc:276] 0709 13:11:50.053060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.916689 (+863629us) service_pool.cc:224] Handling call
0709 13:11:50.959417 (+ 42728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:51.067402 (+107985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.095920 (+ 28518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:51.153462 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:11:51.303814 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.334650 24134 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:51.325428 24289 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.507375 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 107) took 2160 ms. Trace:
I20260709 13:11:51.507762 24392 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:51.324940 24339 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.362687 24376 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.237417 24384 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.359550 24388 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:51.513321 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 299879us
I20260709 13:11:51.514691 22907 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:51.347343 24399 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.318864 22914 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:51.429003 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 211446us
I20260709 13:11:51.508159 24289 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.356669 24297 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.452865 24381 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:51.454093 24356 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.456983 24218 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:51.545395 24356 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:51.548538 24218 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.550638 24218 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.554078 24376 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:51.556768 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 403197us
I20260709 13:11:51.496496 24386 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:11:51.466353 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.474671 24317 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:51.475829 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 991) took 1374 ms. Trace:
I20260709 13:11:51.560930 24317 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.561188 22833 rpcz_store.cc:276] 0709 13:11:50.100690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.937689 (+836999us) service_pool.cc:224] Handling call
0709 13:11:50.938113 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.938381 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:51.067821 (+129440us) log.cc:844] Serialized 14 byte log entry
0709 13:11:51.474502 (+406681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:51.474713 (+   211us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:51.474910 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.475082 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.475438 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5311,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":253,"thread_start_us":400152,"threads_started":1,"wal-append.queue_time_us":372988}
I20260709 13:11:51.476122 24320 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:51.563611 24320 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:51.479581 24331 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:11:51.565781 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.480396 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:51.484401 24405 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.348150 24211 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:51.490195 24382 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:51.494077 24383 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.566174 24331 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.513121 24339 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:51.508253 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.508095 22839 rpcz_store.cc:276] 0709 13:11:49.347202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.786114 (+1438912us) service_pool.cc:224] Handling call
0709 13:11:50.786447 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:51.230300 (+443853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.506455 (+276155us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54000c67d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:51.507143 (+   688us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905270528}
W20260709 13:11:51.568500 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 88654.8us
I20260709 13:11:51.459667 24298 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.575207 24405 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.565903 24320 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.591537 24382 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:51.593950 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.593835 24379 raft_consensus.cc:2764] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:11:51.593178 24211 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.594136 24155 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:51.595135 24405 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:51.553427 24218 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.595248 24211 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.476910 24333 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.602578 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:51.583117 24386 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.575505 24317 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.603726 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:51.621654 22914 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
I20260709 13:11:51.622936 24286 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.624194 24317 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.625409 22879 leader_election.cc:400] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.625561 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.628484 22840 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:51.637488 22879 leader_election.cc:403] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:11:51.629745 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:51.631796 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 105392us
I20260709 13:11:51.632932 24396 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:51.634018 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:51.641111 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.642338 22914 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.641171 22839 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:51.637988 24320 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:51.660301 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101695us
I20260709 13:11:51.577699 24331 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.664064 24331 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.665668 22838 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:51.671572 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:51.673857 24333 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.665766 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.676470 24411 raft_consensus.cc:2764] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:11:51.678040 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:51.678655 24413 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:11:51.674021 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48973.2us
I20260709 13:11:51.671653 23462 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:51.684108 24381 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.685304 22879 leader_election.cc:400] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.686173 22879 leader_election.cc:403] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:51.679617 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:51.691012 22914 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
W20260709 13:11:51.694180 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 397317us
I20260709 13:11:51.706688 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.656629 24391 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.712646 24053 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.718400 24181 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:51.725936 24181 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:51.727025 22879 leader_election.cc:400] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.727624 22879 leader_election.cc:403] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:51.735173 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.742064 24134 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.718969 24155 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.708103 24382 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:51.764339 22879 leader_election.cc:400] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.764853 24403 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:51.767678 22879 leader_election.cc:403] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:51.770227 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.744827 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:51.733700 24433 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:51.782194 22912 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:51.789803 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.789975 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:51.790367 24438 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.077s	user 0.003s	sys 0.000s
I20260709 13:11:51.791337 22907 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.793700 24285 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:51.794287 24211 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.794489 24285 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.795035 24211 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:51.789641 22914 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:51.780586 24433 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.798960 24433 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:51.782364 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 103030us
W20260709 13:11:51.800973 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 84390.8us
I20260709 13:11:51.805796 24356 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.796936 24211 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.806766 24356 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:11:51.809968 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.806069 24391 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:51.806020 23431 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3.
I20260709 13:11:51.813925 24391 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:51.727427 24181 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:51.815349 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51264.8us
I20260709 13:11:51.778898 24398 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.808444 24356 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:51.834388 22879 leader_election.cc:400] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4.
I20260709 13:11:51.795076 24285 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.815568 24181 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:51.863124 22841 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.014s
I20260709 13:11:51.863106 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.865662 22914 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:51.866816 22841 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:51.871255 24339 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:51.871479 22833 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:51.871780 24339 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:51.882331 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.869196 24286 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:51.884227 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.877593 24339 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:51.902412 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:51.910427 24303 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.010s
I20260709 13:11:51.840914 24384 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.183s
W20260709 13:11:51.909719 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.922185 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.902732 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:51.924126 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:51.924957 22914 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 1.
W20260709 13:11:51.905301 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47870.5us
I20260709 13:11:51.912612 22910 raft_consensus.cc:2376] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2. Re-sending same reply.
I20260709 13:11:51.903172 22911 raft_consensus.cc:2376] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2. Re-sending same reply.
I20260709 13:11:51.904963 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.927680 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:51.929661 22908 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2. Re-sending same reply.
I20260709 13:11:51.904651 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:51.905679 24331 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:51.933706 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 99512.1us
W20260709 13:11:51.934595 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.933388 24339 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.943252 22909 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:51.944103 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1847) took 1010 ms. Trace:
I20260709 13:11:51.944218 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.944766 22912 rpcz_store.cc:276] 0709 13:11:50.933201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.032981 (+ 99780us) service_pool.cc:224] Handling call
0709 13:11:51.039784 (+  6803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:51.783893 (+744109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.943796 (+159903us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54002111d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:51.944026 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603126656}
I20260709 13:11:51.945683 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.946290 24379 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:51.947052 24379 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:51.948916 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:51.949102 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.949438 24213 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.949962 24386 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.950062 24213 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.950703 24386 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:11:51.944191 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.953420 24396 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:51.956792 22841 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.935364 24211 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:11:51.941083 24331 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:51.960976 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:51.970482 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:51.972858 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.967204 24218 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:51.986334 24399 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.951704 24213 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:51.998595 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43901.8us
I20260709 13:11:51.980515 24218 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:52.004743 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:52.009038 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:52.016177 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77214.3us
I20260709 13:11:51.998057 24398 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:52.024839 22841 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:52.026458 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:51.948974 24379 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:52.029394 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:52.034183 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:52.035292 22915 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.037779 22764 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.043459 24286 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:52.043958 23462 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:52.043215 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:52.043255 24411 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:52.046341 24411 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:52.047401 24411 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:52.051791 22907 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
W20260709 13:11:52.052471 24446 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 13:11:52.052819 22906 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
W20260709 13:11:52.054350 24445 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:11:52.055919 24331 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.056366 22761 raft_consensus.cc:2376] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4. Re-sending same reply.
I20260709 13:11:52.059831 22912 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:52.062564 22909 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:51.816162 24382 consensus_queue.cc:1175] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "776baf137f8745dd866379ca15ab8cee" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:52.032735 24386 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.077333 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:52.078742 22912 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:52.079905 24286 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:52.082090 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:52.084913 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.085963 24218 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:52.095096 24218 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:52.087864 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63748.2us
W20260709 13:11:52.075202 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.097240 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.098925 22915 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:52.107244 24211 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:52.107997 24211 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:52.108498 24211 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.090945 24404 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:52.110105 24404 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:52.112083 24379 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.111634 24404 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.113325 24404 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.114859 24286 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:52.118479 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.118878 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:52.121693 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:52.051849 24285 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:52.123263 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:52.123885 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.127822 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:52.131657 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.167513 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.183065 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.139789 22914 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:52.187242 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 105398us
I20260709 13:11:52.128790 22907 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:52.202199 22660 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 262) took 5389 ms. Trace:
I20260709 13:11:52.207798 23466 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Advancing to term 3
I20260709 13:11:52.210204 22909 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:52.186954 22664 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.213029 22664 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.226804 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:52.225090 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:52.197923 24286 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:52.236884 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 147080us
I20260709 13:11:52.252046 23466 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:11:52.252499 23466 pending_rounds.cc:85] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 2
I20260709 13:11:52.233736 22761 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:52.252735 23466 pending_rounds.cc:107] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:11:52.260327 23466 raft_consensus.cc:2889] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:52.202818 22660 rpcz_store.cc:276] 0709 13:11:46.808671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.637450 (+828779us) service_pool.cc:224] Handling call
0709 13:11:52.197720 (+4560270us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:50.196093 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:50.196722 (+   629us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:50.196762 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:11:50.196784 (+    22us) tablet.cc:662] Decoding operations
0709 13:11:50.197772 (+   988us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:50.197857 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:50.197889 (+    32us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:11:50.198292 (+   403us) tablet.cc:701] Row locks acquired
0709 13:11:50.198310 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:11:50.198457 (+   147us) write_op.cc:270] Start()
0709 13:11:50.198586 (+   129us) write_op.cc:276] Timestamp: P: 1783602710198416 usec, L: 0
0709 13:11:50.198608 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:11:50.199332 (+   724us) log.cc:844] Serialized 1677 byte log entry
0709 13:11:50.693774 (+494442us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:11:51.065817 (+372043us) op_driver.cc:464] REPLICATION: finished
0709 13:11:51.302609 (+236792us) write_op.cc:301] APPLY: starting
0709 13:11:51.302756 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:51.303032 (+   276us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:51.303056 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:51.303914 (+   858us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:51.304231 (+   317us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:11:51.304312 (+    81us) write_op.cc:312] APPLY: finished
0709 13:11:51.304619 (+   307us) log.cc:844] Serialized 41 byte log entry
0709 13:11:51.325008 (+ 20389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:51.329201 (+  4193us) write_op.cc:454] Released schema lock
0709 13:11:51.331163 (+  1962us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:51.355795 (+ 24632us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:51.357920 (+  2125us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":970955,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":229821,"apply.run_cpu_time_us":3457,"apply.run_wall_time_us":44662,"num_ops":4,"prepare.queue_time_us":166471,"prepare.run_cpu_time_us":24322,"prepare.run_wall_time_us":308053,"raft.queue_time_us":237192,"raft.run_cpu_time_us":17271,"raft.run_wall_time_us":158015,"replication_time_us":852095,"spinlock_wait_cycles":10486016,"thread_start_us":546502,"threads_started":4,"wal-append.queue_time_us":339674}]]}
W20260709 13:11:52.274904 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 94541.5us
I20260709 13:11:52.257850 24405 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.291255 24087 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.293550 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:52.294478 22763 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.295969 23997 tablet_copy_client.cc:538] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:11:52.299052 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77373.8us
I20260709 13:11:52.294178 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:52.301730 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:52.260680 24331 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:11:52.277854 24213 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:11:52.264035 24356 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 13:11:52.316915 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.265331 24433 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.326146 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:52.328997 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:52.330433 22914 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:52.301469 24405 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.337342 24399 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:52.340973 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:52.327988 24298 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:52.338018 24405 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:52.349167 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:52.332162 24285 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:52.359027 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:52.606235 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 277101us
I20260709 13:11:52.700269 24320 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.700554 22763 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:52.363065 24397 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:52.700878 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.701217 24181 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.334584 24411 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:11:52.701812 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.735289 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.702047 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 352922us
W20260709 13:11:52.702270 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 366925us
I20260709 13:11:52.702759 22765 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:52.703212 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:52.603559 24399 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:52.356923 24286 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:11:52.740226 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 414140us
I20260709 13:11:52.703022 22761 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:52.702971 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:52.733738 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:52.707329 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:52.705651 24382 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.744689 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:11:52.721246 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.748490 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.735330 22806 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.559s (SENT) [suppressed 57 similar messages]
I20260709 13:11:52.732931 22835 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.763023 22907 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:52.763901 22908 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:52.761291 24211 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.652s	user 0.021s	sys 0.038s
I20260709 13:11:52.765295 24211 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:52.767581 24379 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.006s
I20260709 13:11:52.750404 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:52.772591 24424 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:52.773453 24424 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:52.774904 24424 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.755169 22760 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:52.739583 24399 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:52.756214 24396 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:52.757584 22909 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:52.755553 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:52.727317 24298 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:11:52.759894 24218 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.664s	user 0.014s	sys 0.004s
I20260709 13:11:52.762291 22915 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:52.732188 24320 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:52.757922 24086 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:52.719810 24404 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:11:52.769380 22912 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:52.822923 22841 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:52.797890 24405 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:52.805160 22914 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:52.807163 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:52.791364 24399 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.798933 24396 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.797014 24398 raft_consensus.cc:2764] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:11:52.835873 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.769392 24211 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:52.846457 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:52.781615 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 434438us
W20260709 13:11:52.863358 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 513803us
I20260709 13:11:52.793048 24181 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:52.868257 22839 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 2: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:52.862114 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:52.843032 24396 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:52.850762 24440 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:52.848642 24086 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:52.821326 24382 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:52.855209 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:52.887887 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.895792 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:52.897235 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.898048 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.875516 24398 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:52.901883 24398 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:52.902467 24398 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:52.893953 24440 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:52.904757 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:52.892802 24433 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:52.910301 24459 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.872927 24211 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.892678 22840 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:52.892097 24386 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:11:52.944443 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75715.2us
W20260709 13:11:52.939193 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 183737us
W20260709 13:11:52.943323 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 171002us
I20260709 13:11:52.876436 24213 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:52.947101 24433 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:52.950310 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73000.4us
W20260709 13:11:52.946929 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.969070 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 100448us
I20260709 13:11:52.941553 24218 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:52.955957 22760 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:11:52.944356 24404 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 13:11:53.000669 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50516.6us
I20260709 13:11:52.976336 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:11:52.969769 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 225140us
I20260709 13:11:52.944131 24331 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:11:52.970705 24087 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.678s	user 0.013s	sys 0.003s
I20260709 13:11:52.948246 24211 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.008087 24181 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.009912 24211 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.013648 24405 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.015498 24398 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:53.016170 22906 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:53.003810 22834 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:53.018054 24320 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:52.956792 24433 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.021968 24086 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.024226 24433 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.026309 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:53.027725 22757 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:52.998514 24386 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.030889 24386 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.016005 22914 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:52.893464 24298 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:52.951148 24459 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.036311 24459 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:52.997216 24286 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:53.037238 24440 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.039198 24286 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.042881 24087 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:53.045625 24213 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:53.048218 24213 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.049070 22838 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:53.049494 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89793.5us
I20260709 13:11:53.052953 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:53.051190 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.053457 24472 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:53.053972 24472 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.054420 24472 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.055135 24433 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:53.055727 22763 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:53.055574 24433 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.056309 24433 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.057914 24397 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:53.060382 24285 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:53.060962 24285 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:53.061492 24285 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:53.063947 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57282.7us
I20260709 13:11:53.054352 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.067021 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.065079 24396 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.065459 24433 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.066749 24382 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.068969 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:52.973136 24463 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
I20260709 13:11:53.069229 24211 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:11:53.083580 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:53.084014 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.134907 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.105314 24386 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
W20260709 13:11:53.087597 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.054445 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49217.4us
I20260709 13:11:53.121352 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:53.134197 24053 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:53.068902 24470 raft_consensus.cc:2764] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:11:53.086530 24213 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:11:53.146646 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:11:53.142352 24398 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.183244 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:53.185253 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44493.2us
I20260709 13:11:53.187342 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.152211 24218 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.152580 24317 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.151371 24053 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:53.214336 24459 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:53.210310 24218 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.215720 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:53.215782 24479 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:53.216804 24218 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.217247 24479 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:53.217669 24479 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:53.218736 24479 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.220247 24479 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.221900 23997 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
W20260709 13:11:53.227370 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.228369 24477 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:53.230398 24466 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:11:53.232056 24382 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:53.232908 24382 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:53.233690 24173 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.234302 24173 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.234473 24382 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.236651 24173 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.237323 24474 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.204689 24473 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:11:53.239426 24473 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:53.239894 24473 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:53.208899 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 131385us
W20260709 13:11:53.207751 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.213954 24317 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.238127 24474 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.174032 24464 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:11:53.210197 24412 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:53.260304 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:53.221946 24440 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:53.295857 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52821.6us
I20260709 13:11:53.303053 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.304060 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:53.253855 24386 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:53.307574 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 120259us
I20260709 13:11:53.205390 24468 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:53.309980 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:53.286511 24398 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:53.301136 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6140259379999407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 13:11:53.259035 24470 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:53.324306 24478 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:53.323410 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5200611150000896s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 13:11:53.324651 24467 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:53.325608 24475 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:11:53.321960 24468 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.328994 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:11:53.330008 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.332415 22841 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:11:53.333163 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.337699 24488 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:53.348044 23431 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.669s (SENT)
I20260709 13:11:53.348762 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.307269 24464 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.338450 24087 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:53.241102 24473 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.298308 24379 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:53.342370 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:53.364156 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.359138 24474 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.366583 24379 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:53.367036 24379 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:53.367141 24459 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.368355 22759 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:53.367929 24459 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.314116 24412 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:53.369832 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.371708 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.370343 24412 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.372970 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.320725 24333 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:53.307197 24479 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:11:53.382546 22759 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.298961 24399 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:53.351929 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 93346.5us
I20260709 13:11:53.343251 24478 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:53.357259 22760 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:53.387253 24181 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:53.387315 23464 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:53.358489 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.387394 24379 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.389245 22762 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:53.311143 24440 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:53.361179 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.389717 22840 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:53.367264 24317 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.340394 24396 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.343031 24467 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.392619 24440 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.387523 24478 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.394115 24473 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:53.394886 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46679.2us
I20260709 13:11:53.395915 24399 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.396621 24399 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.397382 24399 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.397830 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.398312 24459 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.399340 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.400430 22765 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:53.400887 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.401283 24487 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:53.389783 24471 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:11:53.402377 24487 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.404575 24464 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.380002 24333 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:53.379920 24213 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.406329 24213 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.406735 24333 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.399737 24399 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.407841 24333 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.370152 22761 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:53.408668 24213 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.319003 24386 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.410877 24386 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.345530 24477 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.411325 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:53.412058 22840 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:53.411952 24477 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.359325 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.374881 24433 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:53.382682 22835 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.620s	user 0.014s	sys 0.000s
I20260709 13:11:53.414405 22835 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:53.415067 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 128) took 1138 ms. Trace:
I20260709 13:11:53.415426 22835 rpcz_store.cc:276] 0709 13:11:52.277125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.284817 (+  7692us) service_pool.cc:224] Handling call
0709 13:11:53.414975 (+1130158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:53.392762 24467 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.416501 24285 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.343830 24397 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:11:53.418397 22757 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:53.427637 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.428325 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:53.428754 22757 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.429116 23457 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:53.429697 22838 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:53.430289 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.430624 24477 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.430950 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:53.431236 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52196.9us
I20260709 13:11:53.431684 22839 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:53.431268 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:53.435446 24397 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:53.437009 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:53.437829 23464 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:53.438918 24472 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.439813 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.351135 24382 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:53.377712 23466 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:53.407737 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:53.344439 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.452114 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:53.452828 22908 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:53.454888 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47244.8us
W20260709 13:11:53.421823 22732 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.522s (SENT)
I20260709 13:11:53.437912 24433 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.458784 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:53.336796 24398 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:53.442075 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.440830 24399 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.462272 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:53.467039 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:53.442919 24487 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.462546 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:53.469609 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47848.6us
I20260709 13:11:53.468649 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.471921 24477 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:53.474136 24087 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:53.469059 24286 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:53.477556 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.479619 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.450738 24379 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.442834 24382 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:53.484432 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:53.486091 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:53.487426 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:53.458356 24471 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.508978 24471 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.511765 24471 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.462070 24440 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:53.512516 24440 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.513130 24440 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.513732 24459 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:53.514175 24459 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.514513 24459 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.514703 24472 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.516256 24213 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:53.516682 24213 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.517017 24213 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.518651 24386 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:53.498065 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.491055 24331 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:53.490859 24181 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:53.497104 24286 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.488354 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:53.527299 22763 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:53.534186 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.488162 24479 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:11:53.501915 24433 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:53.536176 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:53.544816 22839 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:53.521284 24382 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.542855 22757 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:53.531034 24495 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:11:53.551535 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.559368 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.530666 24331 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:53.566363 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.693s (SENT)
I20260709 13:11:53.536778 24286 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.551187 22760 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.576076 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:53.577791 22762 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.537868 24398 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:53.604539 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72753.2us
I20260709 13:11:53.607518 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:53.608662 22910 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.572826 24331 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.611719 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.612741 24468 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:53.614202 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.617760 22759 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.615389 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:53.618631 22909 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:11:53.618887 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.583897 24320 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:53.620906 22912 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:53.622617 24493 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 13:11:53.624336 24497 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:11:53.626073 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.628360 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.633350 22908 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:53.644490 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.645185 22759 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.644177 24440 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.620342 24471 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.650100 22910 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:53.653123 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:11:53.655068 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 651962us
W20260709 13:11:53.656517 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 80526.3us
I20260709 13:11:53.636312 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.660434 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:53.665167 22839 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:53.656270 24181 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:11:53.666558 24499 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:11:53.657146 22760 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.619105 24467 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.663921 22758 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:53.651665 24468 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:53.685681 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 192203us
I20260709 13:11:53.676389 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.685931 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.696375 22760 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.697530 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:53.698184 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.697715 22757 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.699815 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.700117 24472 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.701483 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.657377 24471 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.702325 22757 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:53.702710 24459 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.703183 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.704011 24471 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.657678 24478 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:53.704384 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:53.705001 24459 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.699841 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:53.751966 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:11:53.705641 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:53.752149 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82420.4us
W20260709 13:11:53.755708 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46123.1us
W20260709 13:11:53.753072 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.758832 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:53.761245 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:53.762197 22833 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:53.686534 24468 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:53.752835 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.667490 24181 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.696375 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:53.785610 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98236.1us
W20260709 13:11:53.791445 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 91720us
W20260709 13:11:53.793514 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.796128 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.800401 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.801631 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.802253 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.803201 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.769793 24473 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.785068 24386 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.788189 24487 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.774678 24468 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.759928 24471 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:11:53.789287 24507 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:53.795572 24464 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.817399 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:53.817750 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.829066 22879 leader_election.cc:400] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:53.829675 22879 leader_election.cc:403] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:53.830341 24424 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:53.830758 24424 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:53.831130 24424 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.833768 24473 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:53.837152 24473 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:53.838773 24479 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:53.839195 24479 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:53.841560 24477 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:11:53.843660 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.844650 22879 leader_election.cc:400] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:53.845193 22879 leader_election.cc:403] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:53.813762 22758 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:53.878005 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60327.1us
I20260709 13:11:53.863319 24507 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:11:53.891304 22732 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items
I20260709 13:11:53.874825 24467 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.883471 24472 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.883122 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:53.905499 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 148659us
I20260709 13:11:53.887430 24386 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:53.845403 24382 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:53.922895 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.926868 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 981) took 1651 ms. Trace:
W20260709 13:11:53.932327 24484 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.538s	user 0.000s	sys 0.004s
I20260709 13:11:53.927575 22764 rpcz_store.cc:276] 0709 13:11:52.275918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.282639 (+  6721us) service_pool.cc:224] Handling call
0709 13:11:52.305989 (+ 23350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:52.346359 (+ 40370us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:52.839150 (+492791us) log.cc:844] Serialized 14 byte log entry
0709 13:11:53.926163 (+1087013us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b18002f8d20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287
0709 13:11:53.926317 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:53.926391 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:53.926466 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.926563 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.926751 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":540888,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":127869,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3240,"spinlock_wait_cycles":8649088,"thread_start_us":492307,"threads_started":3,"wal-append.queue_time_us":294022}
W20260709 13:11:53.937357 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1084) took 1141 ms (client timeout 1519 ms). Trace:
I20260709 13:11:53.891691 24487 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 13:11:53.909575 24502 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
I20260709 13:11:53.898502 24440 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:53.937889 22838 rpcz_store.cc:269] 0709 13:11:52.796078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.800674 (+  4596us) service_pool.cc:224] Handling call
0709 13:11:53.937165 (+1136491us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54000393d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:53.937297 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354768128}
I20260709 13:11:53.901304 24510 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:53.903756 24478 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.877018 24464 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:53.946408 24504 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:11:53.938809 24467 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3.
I20260709 13:11:53.963979 24487 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:53.943630 24505 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:53.972743 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 128395us
I20260709 13:11:53.981882 24505 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.983052 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:53.984725 24505 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.988682 24464 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:53.956123 24286 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:53.992547 24286 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:53.956321 24472 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:11:53.938736 24470 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:11:53.956060 24510 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:54.016010 22759 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:53.899641 22732 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1786)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1827)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1788)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1781)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
W20260709 13:11:54.019655 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.970561 24487 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:54.013646 24510 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.022784 24181 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.028174 24522 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:54.027778 24478 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.029623 24507 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.031438 24479 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.032677 24382 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:54.035131 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.055816 22911 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:54.049299 22909 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:54.049811 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.018271 24398 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:54.037650 24379 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.007s
W20260709 13:11:54.058137 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 138126us
I20260709 13:11:54.039917 24522 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.067836 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.068969 22912 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:54.039716 24467 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.070312 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:54.073051 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 90204.1us
I20260709 13:11:54.045666 24181 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.072007 24522 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.074359 24479 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:54.077736 24399 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.004s
I20260709 13:11:54.094403 24470 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:54.094930 24470 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:54.096503 24485 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.699s	user 0.002s	sys 0.002s
I20260709 13:11:54.074975 24520 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:11:54.054370 24478 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:54.102626 24213 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.008s
I20260709 13:11:54.103061 24412 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:54.103663 24412 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:54.103421 24213 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.108021 22760 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:11:54.110054 24213 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:54.112234 24285 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:54.114892 24087 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:54.115628 24087 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:54.116760 24087 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.073766 24487 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.120041 24487 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.120577 24087 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.086117 24521 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:11:54.080327 24467 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:54.125952 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.126442 24521 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:54.126896 24521 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:54.127600 24524 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:54.128016 24524 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:54.127920 24521 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.129318 24524 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.130393 22758 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:54.130877 24524 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.136870 22915 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:54.137817 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:54.138502 24331 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
I20260709 13:11:54.138684 22912 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.128758 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 986) took 1378 ms. Trace:
W20260709 13:11:54.141209 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55580.5us
I20260709 13:11:54.120514 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.140403 22765 rpcz_store.cc:276] 0709 13:11:52.750784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.791922 (+ 41138us) service_pool.cc:224] Handling call
0709 13:11:52.825368 (+ 33446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.400867 (+575499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.128382 (+727515us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54001fa254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:54.128604 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322024704}
I20260709 13:11:54.144156 22910 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:54.145646 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.139053 24331 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.147770 24181 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:54.148229 24331 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:54.148625 22839 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:54.153112 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:54.096386 24470 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:54.098376 24399 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:54.155926 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.158322 22915 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:54.159643 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89440.8us
I20260709 13:11:54.159569 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.157588 24331 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.159411 24521 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.160071 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.194149 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:54.195779 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.193192 22909 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:54.160799 24467 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:54.214963 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.093212 22731 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1781)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
W20260709 13:11:54.217970 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43792.8us
I20260709 13:11:54.220323 22909 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:54.221737 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.223073 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.223672 22909 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.214249 24470 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.231577 22915 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:54.219632 24399 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.182804 24424 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.234733 22731 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1005)
I20260709 13:11:54.143239 24412 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:54.245885 22913 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:54.247071 24424 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:54.248122 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.250267 24412 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:54.253763 24505 consensus_queue.cc:1175] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "776baf137f8745dd866379ca15ab8cee" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 2 } }
I20260709 13:11:54.255019 22757 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:54.254921 24468 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:54.255805 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.243356 22731 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1005)
I20260709 13:11:54.260466 23997 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:54.264427 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.265256 22908 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:54.260041 22731 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
I20260709 13:11:54.274611 24478 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:54.276427 22912 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:11:54.244717 24530 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
W20260709 13:11:54.276969 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.556s (SENT)
I20260709 13:11:54.277683 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:54.271399 22731 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
W20260709 13:11:54.312845 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.315032 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.317977 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.349256 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 175081us
I20260709 13:11:54.352137 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:54.329275 24469 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.261s	user 0.008s	sys 0.008s
I20260709 13:11:54.355743 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.357213 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.323336 24398 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:54.374058 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.358642 24469 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.291s	user 0.010s	sys 0.012s
I20260709 13:11:54.379845 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.385648 24526 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:11:54.387171 24510 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.403941 22761 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:54.409309 24510 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:54.395824 24399 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:54.382966 23997 tablet_bootstrap.cc:492] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:11:54.372025 24433 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:54.410879 24053 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.412423 23997 ts_tablet_manager.cc:1403] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 1.207s	user 0.016s	sys 0.030s
I20260709 13:11:54.413770 24086 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:54.344136 24531 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260709 13:11:54.414353 24086 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:54.418419 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.419903 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.420782 22759 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:54.420832 24399 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:54.421386 24399 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:54.421454 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1801) took 1033 ms. Trace:
I20260709 13:11:54.425077 22763 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:54.423065 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:54.424700 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.425323 22759 rpcz_store.cc:276] 0709 13:11:53.388245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.419527 (+1031282us) service_pool.cc:224] Handling call
0709 13:11:54.421391 (+  1864us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:54.430516 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:54.432102 22910 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:54.419512 23997 raft_consensus.cc:359] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:54.442171 22914 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:54.466195 22841 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:54.468994 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.470147 22914 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:54.375274 24398 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.423259 24468 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:54.493857 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.496387 22759 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3. Re-sending same reply.
I20260709 13:11:54.440563 23997 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:11:54.429383 24528 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:54.494176 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:54.498790 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.465854 24510 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:54.503225 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:54.511711 23462 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:54.511857 22841 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:54.504048 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.509613 22914 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:11:54.515523 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109831us
I20260709 13:11:54.489270 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:54.415481 24086 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.491490 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:54.491637 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:54.521588 22759 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:54.524319 22914 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:54.503945 24356 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:54.526381 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:54.527275 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 250349us
I20260709 13:11:54.528061 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1784) took 1171 ms. Trace:
I20260709 13:11:54.528450 22759 rpcz_store.cc:276] 0709 13:11:53.356373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.497287 (+1140914us) service_pool.cc:224] Handling call
0709 13:11:54.527985 (+ 30698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:54.510926 24510 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.543612 22914 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:11:54.544466 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.548188 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:54.548636 22841 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
W20260709 13:11:54.540053 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 173348us
I20260709 13:11:54.465379 24399 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.558990 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.592043 22759 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:54.527218 23457 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.525977 23464 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 1.
I20260709 13:11:54.540750 24510 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.539601 22838 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.606251 22910 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:11:54.611845 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63739.4us
I20260709 13:11:54.540311 23997 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:54.620252 24548 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:54.621429 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.622061 23997 ts_tablet_manager.cc:1434] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.209s	user 0.011s	sys 0.000s
I20260709 13:11:54.623641 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.646871 22915 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:54.623924 24522 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:11:54.595685 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:54.654009 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:54.654601 22834 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.610530 24331 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:54.625886 24537 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:11:54.631538 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 394) took 1020 ms. Trace:
I20260709 13:11:54.632781 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:54.610040 24086 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:54.650724 22914 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 5. Ops: [3.3-3.3]
I20260709 13:11:54.642858 24548 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.684669 22759 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:54.686040 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1818) took 1274 ms. Trace:
I20260709 13:11:54.686578 22759 rpcz_store.cc:276] 0709 13:11:53.414849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.606936 (+1192087us) service_pool.cc:224] Handling call
0709 13:11:54.685845 (+ 78909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:54.612135 24469 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
W20260709 13:11:54.653697 24492 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.242s	user 0.000s	sys 0.008s
I20260709 13:11:54.620136 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:54.660930 22909 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:11:54.661705 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.697600 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.661602 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 119811us
W20260709 13:11:54.665872 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 137256us
I20260709 13:11:54.660301 24399 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:54.663529 23466 rpcz_store.cc:276] 0709 13:11:53.611507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.618573 (+  7066us) service_pool.cc:224] Handling call
0709 13:11:53.626551 (+  7978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.626948 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.627000 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:53.627540 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.909217 (+281677us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:53.909273 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.910447 (+  1174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.627141 (+716694us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b2400396098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea42fb0 00007f68a16b9488 00007f68a16b9be8 00007f68a1691398 00007f68a168c76a 00007f68a168bbb6
0709 13:11:54.631195 (+  4054us) raft_consensus.cc:1685] finished
0709 13:11:54.631221 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.631460 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123685,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":18997,"spinlock_wait_cycles":12375296,"thread_start_us":281221,"threads_started":1,"wal-append.queue_time_us":380025}
I20260709 13:11:54.691331 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.649577 22833 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
I20260709 13:11:54.719889 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.713881 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:54.725850 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109354us
I20260709 13:11:54.726599 24531 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:54.732332 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:54.736138 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.689232 24548 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.719185 24469 consensus_queue.cc:1243] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:11:54.740211 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.744096 22914 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:54.691604 24468 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.658708 24522 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:54.751760 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.714375 24544 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.756268 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:54.762470 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.762766 22910 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:54.720414 24398 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:54.752645 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.768477 22765 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:54.764194 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.769279 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1828) took 1061 ms. Trace:
I20260709 13:11:54.771437 22765 rpcz_store.cc:276] 0709 13:11:53.707556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.739748 (+1032192us) service_pool.cc:224] Handling call
0709 13:11:54.769182 (+ 29434us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:54.782258 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.745155 22837 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:54.754732 22759 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:11:54.785439 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 193896us
I20260709 13:11:54.786433 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1821) took 1093 ms. Trace:
I20260709 13:11:54.786825 22759 rpcz_store.cc:276] 0709 13:11:53.693263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.687671 (+994408us) service_pool.cc:224] Handling call
0709 13:11:54.786362 (+ 98691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:54.809710 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:54.759138 24531 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:54.786278 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:54.846534 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.869632 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.767635 22907 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
W20260709 13:11:54.890933 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77150.1us
W20260709 13:11:54.892784 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48691us
I20260709 13:11:54.777994 24386 raft_consensus.cc:1064] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:11:54.871742 22765 raft_consensus.cc:2376] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3. Re-sending same reply.
I20260709 13:11:54.908283 24549 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:54.908181 24543 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:54.910923 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.919859 22759 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.911444 22914 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
W20260709 13:11:54.914304 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.630s (SENT)
I20260709 13:11:54.920646 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:54.915187 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1831) took 1146 ms. Trace:
I20260709 13:11:54.765026 24440 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:11:54.921890 22765 rpcz_store.cc:276] 0709 13:11:53.768465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.772255 (+1003790us) service_pool.cc:224] Handling call
0709 13:11:54.914894 (+142639us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:54.917930 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 181082us
I20260709 13:11:54.810078 24478 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:54.916145 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:54.920698 22910 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:54.890852 24546 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:54.891826 24521 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:54.930799 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.913640 22906 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:54.777741 24468 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:54.936241 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.769003 24544 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:54.922427 24547 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
I20260709 13:11:54.906947 24412 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:54.942993 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:54.957022 22910 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.946110 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.958029 22765 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:54.946895 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.868598 24331 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:11:54.949900 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 994) took 1410 ms. Trace:
I20260709 13:11:54.959263 22766 rpcz_store.cc:276] 0709 13:11:53.349580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.349722 (+   142us) service_pool.cc:224] Handling call
0709 13:11:53.358473 (+  8751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.358753 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.358780 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:54.150357 (+791577us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:11:54.150822 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:11:54.426022 (+275200us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:54.426062 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.426136 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.739173 (+313037us) raft_consensus.cc:1685] finished
0709 13:11:54.739197 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.760054 (+ 20857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6232704,"thread_start_us":274786,"threads_started":1,"wal-append.queue_time_us":337461}
I20260709 13:11:54.961308 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:54.962147 22766 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:54.962563 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1833) took 1151 ms. Trace:
I20260709 13:11:54.962862 22766 rpcz_store.cc:276] 0709 13:11:53.811569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.960992 (+1149423us) service_pool.cc:224] Handling call
0709 13:11:54.962509 (+  1517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:54.963841 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:54.964819 22766 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:54.965257 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 998) took 1502 ms. Trace:
I20260709 13:11:54.966423 22766 rpcz_store.cc:276] 0709 13:11:53.463371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.963528 (+1500157us) service_pool.cc:224] Handling call
0709 13:11:54.965201 (+  1673us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:54.967576 22730 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:58100 dropped due to backpressure: service queue is full with 50 items [suppressed 5 similar messages]
I20260709 13:11:54.930971 24546 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.951980 24053 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.914170 24553 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:11:54.998301 24531 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:55.002426 24551 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:11:54.971617 24412 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:55.005010 22834 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:55.006345 24562 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:55.006961 24470 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.002s
I20260709 13:11:54.933801 24478 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:55.010074 24528 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.012751 22759 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:54.924029 24549 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.020318 24562 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.024539 24549 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:55.025058 24562 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:55.027253 24478 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:55.019007 22914 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:54.976732 24546 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:55.020849 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 318570us
I20260709 13:11:54.947638 24521 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:54.994284 24333 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:55.011574 24476 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.031559 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:55.022182 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1001) took 1195 ms. Trace:
I20260709 13:11:55.037843 22759 rpcz_store.cc:276] 0709 13:11:53.826989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.867894 (+1040905us) service_pool.cc:224] Handling call
0709 13:11:55.022091 (+154197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.051231 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:55.052103 22914 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:55.060417 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.068035 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 159628us
I20260709 13:11:55.068858 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:55.069990 22911 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:11:55.076385 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.967854 22730 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:58100 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1011)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 115)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1010)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 112)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 114)
I20260709 13:11:55.081935 22914 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:55.050840 24333 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:55.049772 24478 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:55.099396 23457 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:11:55.049750 24528 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:54.976024 22910 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
W20260709 13:11:55.109906 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.219992 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.273690 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 347232us
I20260709 13:11:55.274899 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:55.277987 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:55.278542 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.279263 22914 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:55.075450 24412 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:55.090830 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 176560us
W20260709 13:11:55.288287 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.094376 24470 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:55.299559 22806 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.548s (SENT)
I20260709 13:11:55.300783 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:55.301383 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:55.299685 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 230421us
I20260709 13:11:55.302349 22764 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:55.100319 24544 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:55.305348 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:55.304855 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:55.288188 24541 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:55.329897 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1826) took 1635 ms (client timeout 1535 ms). Trace:
I20260709 13:11:55.330966 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:55.332412 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1835) took 1497 ms (client timeout 1935 ms). Trace:
W20260709 13:11:55.334355 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1787) took 1976 ms (client timeout 1935 ms). Trace:
I20260709 13:11:55.334539 22910 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:55.335002 24528 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:55.334877 22757 rpcz_store.cc:269] 0709 13:11:53.357434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.704128 (+346694us) service_pool.cc:224] Handling call
0709 13:11:55.334161 (+1630033us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54000028d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.334276 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539877632}
I20260709 13:11:55.303798 24412 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:11:55.337875 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1888) took 1184 ms. Trace:
I20260709 13:11:55.338305 22906 rpcz_store.cc:276] 0709 13:11:54.153307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.154288 (+   981us) service_pool.cc:224] Handling call
0709 13:11:55.337633 (+1183345us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54000fb254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.337785 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1301053824}
W20260709 13:11:55.339948 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1797) took 1957 ms (client timeout 1982 ms). Trace:
W20260709 13:11:55.340327 22763 rpcz_store.cc:269] 0709 13:11:53.382384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.120189 (+737805us) service_pool.cc:224] Handling call
0709 13:11:55.339737 (+1219548us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54000816d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.339871 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649605504}
I20260709 13:11:55.341765 24578 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.342132 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.342787 22766 raft_consensus.cc:2376] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3. Re-sending same reply.
I20260709 13:11:55.342867 22907 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:11:55.343312 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1834) took 1519 ms (client timeout 1931 ms). Trace:
W20260709 13:11:55.343693 22766 rpcz_store.cc:269] 0709 13:11:53.824353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.341790 (+1517437us) service_pool.cc:224] Handling call
0709 13:11:55.343252 (+  1462us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.345163 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.330482 22760 rpcz_store.cc:269] 0709 13:11:53.694888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.697178 (+  2290us) service_pool.cc:224] Handling call
0709 13:11:55.329636 (+1632458us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400311854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.329781 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941153024}
I20260709 13:11:55.355739 22764 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:55.356823 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1114) took 1116 ms. Trace:
W20260709 13:11:55.361377 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1811) took 1968 ms (client timeout 1556 ms). Trace:
I20260709 13:11:55.361467 22915 rpcz_store.cc:276] 0709 13:11:54.240443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.247822 (+  7379us) service_pool.cc:224] Handling call
0709 13:11:55.356413 (+1108591us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54001e2dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.356715 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915268992}
W20260709 13:11:55.362106 22761 rpcz_store.cc:269] 0709 13:11:53.393259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.409113 (+ 15854us) service_pool.cc:224] Handling call
0709 13:11:53.846508 (+437395us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:55.361089 (+1514581us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54002210d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.361269 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645450368}
W20260709 13:11:55.363232 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.364483 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1786) took 2007 ms (client timeout 1999 ms). Trace:
W20260709 13:11:55.364948 22764 rpcz_store.cc:269] 0709 13:11:53.357097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.286925 (+1929828us) service_pool.cc:224] Handling call
0709 13:11:55.364380 (+ 77455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.372920 24468 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:55.381392 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 127) took 1363 ms. Trace:
I20260709 13:11:55.381810 24528 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.382043 22913 rpcz_store.cc:276] 0709 13:11:54.018164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.018285 (+   121us) service_pool.cc:224] Handling call
0709 13:11:54.024732 (+  6447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.246323 (+221591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.380993 (+1134670us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:55.381269 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508095744}
I20260709 13:11:55.382349 24528 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.094079 24510 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.360116 24522 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:55.333011 22759 rpcz_store.cc:269] 0709 13:11:53.835211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.330357 (+1495146us) service_pool.cc:224] Handling call
0709 13:11:55.332325 (+  1968us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.364770 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.402388 22758 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:11:55.274905 24478 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:55.411826 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1781) took 2056 ms (client timeout 2078 ms). Trace:
W20260709 13:11:55.415191 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 383729us
I20260709 13:11:55.426406 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:55.217626 24333 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:55.414019 22758 rpcz_store.cc:269] 0709 13:11:53.355494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.364372 (+2008878us) service_pool.cc:224] Handling call
0709 13:11:55.411727 (+ 47355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.326930 22730 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1008)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1011)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 121)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 122)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 123)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1010)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 112)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 120)
I20260709 13:11:55.445603 22765 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:55.446708 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1832) took 1654 ms (client timeout 1897 ms). Trace:
W20260709 13:11:55.447273 22765 rpcz_store.cc:269] 0709 13:11:53.792425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.923699 (+1131274us) service_pool.cc:224] Handling call
0709 13:11:55.446609 (+522910us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.431519 24404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.237s	user 0.025s	sys 0.029s
W20260709 13:11:55.450009 24404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.025s	sys 0.029s
I20260709 13:11:55.450945 24433 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:55.451424 24433 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:11:55.451941 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1827) took 1756 ms (client timeout 1661 ms). Trace:
W20260709 13:11:55.452344 22762 rpcz_store.cc:269] 0709 13:11:53.707104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.451722 (+1744618us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:55.451846 (+   124us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:11:55.454092 22806 leader_election.cc:336] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.661s (SENT)
I20260709 13:11:55.384971 24544 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.410008 24541 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:11:55.484572 24568 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:11:55.431923 24440 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.432893 24576 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:55.319730 24470 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:55.457059 24522 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.484669 22906 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:55.504981 24544 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:55.452512 24433 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:55.566823 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 137570us
I20260709 13:11:55.567963 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:11:55.553195 24577 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:11:55.553858 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 142) took 1804 ms (client timeout 1984 ms). Trace:
I20260709 13:11:55.556808 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:55.557192 24548 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.789s	user 0.008s	sys 0.000s
W20260709 13:11:55.558221 24399 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.837s	user 0.013s	sys 0.001s
W20260709 13:11:55.559501 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105471us
W20260709 13:11:55.561013 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 256224us
I20260709 13:11:55.515208 24521 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:55.569278 22839 rpcz_store.cc:269] 0709 13:11:53.749673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.758400 (+  8727us) service_pool.cc:224] Handling call
0709 13:11:55.553643 (+1795243us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54000398d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:55.553799 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893836032}
I20260709 13:11:55.569877 22764 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:11:55.580938 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1836) took 1745 ms (client timeout 1982 ms). Trace:
I20260709 13:11:55.564589 24505 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:55.582657 22764 rpcz_store.cc:269] 0709 13:11:53.835570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.556434 (+1720864us) service_pool.cc:224] Handling call
0709 13:11:55.580814 (+ 24380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.581337 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:55.580665 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:55.576745 22806 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.691s (SENT)
I20260709 13:11:55.588233 22908 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:55.588155 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:55.593407 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:55.602469 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.603663 22764 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:55.562031 24440 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.574584 24576 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:11:55.608717 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1840) took 1771 ms (client timeout 2158 ms). Trace:
W20260709 13:11:55.617465 22764 rpcz_store.cc:269] 0709 13:11:53.836808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.587367 (+1750559us) service_pool.cc:224] Handling call
0709 13:11:55.608647 (+ 21280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.565657 24522 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:55.601773 24459 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:55.586755 24574 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:55.614488 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:55.619895 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.622614 22764 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:55.623016 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-ba1db22a7176432c86e7264f87984ba3 received from {username='slave'} at 127.0.0.1:51142
I20260709 13:11:55.623668 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-ba1db22a7176432c86e7264f87984ba3 on tablet ba1db22a7176432c86e7264f87984ba3 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:55.624364 24574 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:55.625373 24574 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.626466 24386 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.627446 24433 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:55.623178 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1846) took 1482 ms. Trace:
I20260709 13:11:55.629864 22764 rpcz_store.cc:276] 0709 13:11:54.141200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.618279 (+1477079us) service_pool.cc:224] Handling call
0709 13:11:55.623082 (+  4803us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.631002 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.630788 24576 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.631916 22764 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:55.632472 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1003) took 1540 ms. Trace:
I20260709 13:11:55.633237 22764 rpcz_store.cc:276] 0709 13:11:54.092384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.630661 (+1538277us) service_pool.cc:224] Handling call
0709 13:11:55.632355 (+  1694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.574976 24548 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:55.622609 22908 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:55.601601 24508 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:55.610688 24521 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:55.638386 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61708us
W20260709 13:11:55.650352 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.731s (SENT)
I20260709 13:11:55.661100 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:55.663578 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:55.672000 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.612298 24528 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:55.674034 22913 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:55.685549 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:55.677170 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.686558 22840 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:11:55.686954 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.680073 22839 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:55.681103 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.688411 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:11:55.693903 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.623777 24459 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:55.703859 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:55.704123 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.704983 22833 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 2: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:55.705082 22758 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:55.703958 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.708127 22762 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:11:55.708875 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58564.8us
W20260709 13:11:55.709703 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.777s (SENT)
I20260709 13:11:55.710704 22837 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:55.711380 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:55.712136 22764 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:11:55.713653 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1795) took 2339 ms (client timeout 2455 ms). Trace:
W20260709 13:11:55.714178 22758 rpcz_store.cc:269] 0709 13:11:53.374192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.684652 (+2310460us) service_pool.cc:224] Handling call
0709 13:11:55.713567 (+ 28915us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.709205 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:55.703423 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:55.716390 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:55.716925 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1779) took 2362 ms (client timeout 2950 ms). Trace:
I20260709 13:11:55.717198 22913 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:11:55.717336 22764 rpcz_store.cc:269] 0709 13:11:53.354858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.689180 (+2334322us) service_pool.cc:224] Handling call
0709 13:11:55.716865 (+ 27685us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.682915 24476 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.739601 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.740878 22763 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:55.741772 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1845) took 1656 ms (client timeout 1691 ms). Trace:
W20260709 13:11:55.742344 22763 rpcz_store.cc:269] 0709 13:11:54.084889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.699850 (+1614961us) service_pool.cc:224] Handling call
0709 13:11:55.741412 (+ 41562us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:55.743284 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1839) took 1906 ms (client timeout 2078 ms). Trace:
I20260709 13:11:55.743635 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:55.743810 22762 rpcz_store.cc:269] 0709 13:11:53.836512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.678872 (+1842360us) service_pool.cc:224] Handling call
0709 13:11:55.743204 (+ 64332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.744961 22763 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:55.747269 22839 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:55.682727 24508 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:11:55.705655 24086 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.012s	user 0.009s	sys 0.005s
I20260709 13:11:55.759363 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.760540 22758 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:55.761062 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1806) took 2370 ms (client timeout 2952 ms). Trace:
I20260709 13:11:55.690999 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:55.761952 22758 rpcz_store.cc:269] 0709 13:11:53.390982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.715283 (+2324301us) service_pool.cc:224] Handling call
0709 13:11:55.760983 (+ 45700us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.699674 24579 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:55.764483 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1804) took 2374 ms (client timeout 3152 ms). Trace:
W20260709 13:11:55.768877 22763 rpcz_store.cc:269] 0709 13:11:53.389685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.743279 (+2353594us) service_pool.cc:224] Handling call
0709 13:11:55.764406 (+ 21127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.764451 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:55.770304 22764 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:55.770921 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1008) took 1511 ms. Trace:
I20260709 13:11:55.771246 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.771955 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.771226 22764 rpcz_store.cc:276] 0709 13:11:54.259435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.718074 (+1458639us) service_pool.cc:224] Handling call
0709 13:11:55.770853 (+ 52779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.680680 24521 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:55.776013 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:55.787815 22762 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:55.785326 22909 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
W20260709 13:11:55.787175 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77513.4us
W20260709 13:11:55.790653 22806 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.838s (SENT)
I20260709 13:11:55.791170 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:55.717955 24580 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.703387 24412 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:55.795395 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:55.796476 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:55.800640 22758 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:55.801345 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1011) took 1062 ms. Trace:
I20260709 13:11:55.801759 22758 rpcz_store.cc:276] 0709 13:11:54.739017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.767467 (+1028450us) service_pool.cc:224] Handling call
0709 13:11:55.801266 (+ 33799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.804330 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:55.805059 22764 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.747190 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:55.787211 22906 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:55.803397 24580 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:55.813210 22806 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.870s (SENT)
I20260709 13:11:55.813907 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:55.621404 24541 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.768487 24579 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.827948 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:55.828692 22758 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:55.829105 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 115) took 1014 ms. Trace:
I20260709 13:11:55.830384 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:55.831130 22762 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:55.741855 24581 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:11:55.797812 24086 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:55.837571 22837 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:55.803962 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:55.850231 22833 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:55.810164 24478 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:55.835901 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:55.817883 24476 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.826563 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:55.851413 22911 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:55.809135 24412 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:55.907627 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.372318 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:55.846721 22834 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:57.016937 23461 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:55.845371 24470 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:56.818881 24333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.037s	sys 0.084s
I20260709 13:11:56.441869 24590 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.046438 24522 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.047114 24522 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.046937 24590 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.047612 24522 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:56.900446 24433 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.273s	user 0.013s	sys 0.001s
I20260709 13:11:56.041249 22764 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:57.018033 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 999) took 1370 ms. Trace:
I20260709 13:11:57.016674 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1144) took 1390 ms. Trace:
W20260709 13:11:56.822414 23430 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.006s	sys 0.014s
I20260709 13:11:57.018033 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1005) took 1209 ms. Trace:
W20260709 13:11:57.052345 23430 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.010s	sys 0.018s
W20260709 13:11:56.762338 22838 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 2.114s	user 0.018s	sys 0.000s
I20260709 13:11:56.026774 24505 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:56.159351 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.057106 22910 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.230s	user 0.016s	sys 0.000s
W20260709 13:11:57.057631 22910 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.245s	user 0.017s	sys 0.000s
W20260709 13:11:57.057791 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.058585 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1898) took 1623 ms. Trace:
W20260709 13:11:56.795584 23457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.000s	sys 0.008s
I20260709 13:11:57.049556 22834 rpcz_store.cc:276] 0709 13:11:55.647595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.682797 (+ 35202us) service_pool.cc:224] Handling call
0709 13:11:55.786636 (+103839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.017436 (+1230800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.017748 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.846166 24510 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.038218 24521 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:55.907162 24583 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:11:57.074223 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1841) took 3237 ms (client timeout 1988 ms). Trace:
I20260709 13:11:57.020545 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1901) took 1452 ms. Trace:
W20260709 13:11:57.074759 22759 rpcz_store.cc:269] 0709 13:11:53.837110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.073999 (+3236889us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:57.074141 (+   142us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:11:56.393765 24546 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.348s	user 0.012s	sys 0.005s
I20260709 13:11:55.899346 24086 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.021394 22907 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:55.856766 24544 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:57.005033 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:11:56.111065 24571 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.353s	user 0.000s	sys 0.000s
I20260709 13:11:56.736485 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:56.284952 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:56.048569 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.174047 24572 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:55.947203 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.820307 22763 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 2.
W20260709 13:11:55.865365 24536 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.349s	user 0.024s	sys 0.023s
I20260709 13:11:55.852536 24584 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.829442 22758 rpcz_store.cc:276] 0709 13:11:54.815072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.802624 (+987552us) service_pool.cc:224] Handling call
0709 13:11:55.829048 (+ 26424us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.056679 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 193) took 1340 ms (client timeout 1535 ms). Trace:
I20260709 13:11:55.907315 24508 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:55.952301 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.106050 22762 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:55.826282 24579 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.906245 24459 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:55.871322 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 123125us
W20260709 13:11:55.933817 24589 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:11:56.738015 22832 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:11:57.034572 24478 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:11:56.766877 22912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.000s	sys 0.038s
I20260709 13:11:56.822716 24607 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.036195 22839 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.934911 22841 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:57.128779 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.046161 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1838) took 3210 ms (client timeout 2020 ms). Trace:
W20260709 13:11:57.049577 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1823) took 3355 ms (client timeout 3048 ms). Trace:
W20260709 13:11:56.013247 24561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.007s	sys 0.025s
I20260709 13:11:57.052419 22833 rpcz_store.cc:276] 0709 13:11:55.808561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.808772 (+   211us) service_pool.cc:224] Handling call
0709 13:11:55.850041 (+ 41269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.017579 (+1167538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.017753 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:55.833271 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:57.052871 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 834949us
I20260709 13:11:56.225859 24548 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:57.043098 24333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.037s	sys 0.085s
I20260709 13:11:57.012220 24440 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:11:57.052201 22906 rpcz_store.cc:276] 0709 13:11:55.625802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.808013 (+182211us) service_pool.cc:224] Handling call
0709 13:11:57.016573 (+1208560us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.061249 24598 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:11:57.052346 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.006s	sys 0.010s
I20260709 13:11:56.900708 23458 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:57.059094 22910 rpcz_store.cc:276] 0709 13:11:55.435291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.435701 (+   410us) service_pool.cc:224] Handling call
0709 13:11:55.476075 (+ 40374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.476408 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:55.651008 (+174600us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.058028 (+1407020us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.058113 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:57.058187 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.058288 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.058497 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8865,"prepare.queue_time_us":153083,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"thread_start_us":1529518,"threads_started":3,"wal-append.queue_time_us":904231}
I20260709 13:11:57.074901 22911 rpcz_store.cc:276] 0709 13:11:55.567935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.850914 (+282979us) service_pool.cc:224] Handling call
0709 13:11:55.851230 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.020170 (+1168940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.020455 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.059725 23457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.002s	sys 0.010s
I20260709 13:11:57.053931 24522 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.092653 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:57.095757 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 117) took 2167 ms (client timeout 1693 ms). Trace:
W20260709 13:11:56.352676 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1778) took 2998 ms (client timeout 2471 ms). Trace:
W20260709 13:11:57.263315 22763 rpcz_store.cc:269] 0709 13:11:54.928720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.771656 (+842936us) service_pool.cc:224] Handling call
0709 13:11:57.095648 (+1323992us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.263497 22766 rpcz_store.cc:269] 0709 13:11:53.354576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.352362 (+2997786us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:56.352570 (+   208us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:11:57.056910 22838 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
W20260709 13:11:57.099713 24536 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.589s	user 0.042s	sys 0.057s
W20260709 13:11:57.126125 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1820) took 3433 ms (client timeout 2172 ms). Trace:
W20260709 13:11:57.118046 22914 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.367s	user 0.004s	sys 0.024s
I20260709 13:11:57.188577 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.191700 22912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.378s	user 0.000s	sys 0.041s
W20260709 13:11:57.228847 24561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.918s	user 0.009s	sys 0.048s
I20260709 13:11:57.281461 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.130610 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1014) took 1543 ms (client timeout 1622 ms). Trace:
I20260709 13:11:55.855984 23462 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.099305 24433 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:57.103754 23461 rpcz_store.cc:269] 0709 13:11:55.713032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.818591 (+105559us) service_pool.cc:224] Handling call
0709 13:11:57.051522 (+1232931us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.226554 22764 rpcz_store.cc:269] 0709 13:11:53.693917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.773009 (+2079092us) service_pool.cc:224] Handling call
0709 13:11:57.049470 (+1276461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.229449 22836 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:11:57.290278 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.128943 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.113s	user 0.006s	sys 0.021s
W20260709 13:11:57.290995 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.275s	user 0.027s	sys 0.058s
I20260709 13:11:57.233532 22758 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:57.118135 24597 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.356s	user 0.000s	sys 0.002s
W20260709 13:11:57.253054 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.006s	sys 0.010s
W20260709 13:11:57.295130 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.296945 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.297345 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.126715 22832 pending_rounds.cc:85] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 1
I20260709 13:11:57.242272 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.290591 22836 pending_rounds.cc:85] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 2
W20260709 13:11:57.216073 24567 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 2.094s	user 0.005s	sys 0.013s
I20260709 13:11:57.118266 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 133) took 1536 ms. Trace:
I20260709 13:11:57.264794 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:57.310247 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1015) took 1720 ms (client timeout 1898 ms). Trace:
I20260709 13:11:57.129151 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:57.299295 22832 pending_rounds.cc:107] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:11:57.315131 22832 raft_consensus.cc:2889] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:57.315675 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.316215 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1007) took 1386 ms. Trace:
I20260709 13:11:57.316437 22834 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.316634 22832 rpcz_store.cc:276] 0709 13:11:55.929771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.213762 (+283991us) service_pool.cc:224] Handling call
0709 13:11:56.586832 (+373070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.315933 (+729101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.316153 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.212007 24470 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:57.103096 24599 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
I20260709 13:11:57.280930 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1900) took 1762 ms. Trace:
I20260709 13:11:57.324208 22912 rpcz_store.cc:276] 0709 13:11:55.518823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.518993 (+   170us) service_pool.cc:224] Handling call
0709 13:11:55.561390 (+ 42397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.561725 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:55.685780 (+124055us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.280480 (+1594700us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.280553 (+    73us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:57.280613 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.280688 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.280847 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67973,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1286966,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10457,"spinlock_wait_cycles":25559936,"thread_start_us":1707856,"threads_started":3,"wal-append.queue_time_us":430413}
I20260709 13:11:57.282058 22759 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
W20260709 13:11:57.326622 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 122) took 2033 ms (client timeout 2084 ms). Trace:
W20260709 13:11:57.327008 22759 rpcz_store.cc:269] 0709 13:11:55.292970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.109360 (+1816390us) service_pool.cc:224] Handling call
0709 13:11:57.326552 (+217192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.099332 23459 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:57.284549 22914 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.534s	user 0.007s	sys 0.025s
I20260709 13:11:57.290186 24608 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.282476 22766 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:57.337144 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 338307us
I20260709 13:11:57.251013 24521 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.087937 24546 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:57.293985 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1016) took 1696 ms (client timeout 1662 ms). Trace:
W20260709 13:11:57.342533 22758 rpcz_store.cc:269] 0709 13:11:55.597620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.101382 (+1503762us) service_pool.cc:224] Handling call
0709 13:11:57.293871 (+192489us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.294776 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.280390 22765 rpcz_store.cc:269] 0709 13:11:53.692902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.125849 (+3432947us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:57.126030 (+   181us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:11:57.297673 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.346479 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.897s (SENT)
I20260709 13:11:57.346946 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:57.299907 22836 pending_rounds.cc:107] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
W20260709 13:11:57.226634 22760 rpcz_store.cc:269] 0709 13:11:53.836180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.045898 (+3209718us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:57.046074 (+   176us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:11:57.253232 24610 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.260201 24505 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:57.310657 22763 rpcz_store.cc:269] 0709 13:11:55.589449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.264428 (+1674979us) service_pool.cc:224] Handling call
0709 13:11:57.310181 (+ 45753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.127048 24476 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.183s
I20260709 13:11:57.308620 22907 rpcz_store.cc:276] 0709 13:11:55.578361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.162840 (+584479us) service_pool.cc:224] Handling call
0709 13:11:57.020963 (+858123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.085905 (+ 64942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.114769 (+ 28864us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.285387 24548 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.319608 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:57.286296 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:11:57.297058 22835 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:57.323148 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:57.296046 24478 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:57.283977 22762 rpcz_store.cc:269] 0709 13:11:55.587091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.851179 (+264088us) service_pool.cc:224] Handling call
0709 13:11:57.130511 (+1279332us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.420435 23431 connection.cc:256] RPC call timeout handler was delayed by -1.3773561470000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 13:11:57.337616 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1013) took 1751 ms (client timeout 1988 ms). Trace:
W20260709 13:11:57.421190 23431 connection.cc:256] RPC call timeout handler was delayed by -1.15145614700009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260709 13:11:57.102459 24572 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.235917 24607 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:57.346055 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.369840 24544 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:57.425252 22732 connection.cc:256] RPC call timeout handler was delayed by -1.9907684439999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:11:57.425354 22765 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:57.426077 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8539684439999746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:11:57.307737 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 144) took 2856 ms (client timeout 2310 ms). Trace:
W20260709 13:11:57.426481 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8359684440000592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:11:57.350948 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.425864 22833 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:57.259721 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1837) took 3423 ms (client timeout 1999 ms). Trace:
I20260709 13:11:57.368772 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.330086 24470 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.295292 24433 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:57.377828 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.384066 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 289231us
I20260709 13:11:57.370218 22836 raft_consensus.cc:2917] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:11:57.379571 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.391050 22840 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:11:57.416595 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70153.6us
I20260709 13:11:57.414103 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.421042 22766 rpcz_store.cc:269] 0709 13:11:55.586695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.268605 (+1681910us) service_pool.cc:224] Handling call
0709 13:11:57.337522 (+ 68917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.195780 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1006) took 1350 ms. Trace:
I20260709 13:11:57.390944 24086 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.349668 24584 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.309813 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 195) took 1521 ms. Trace:
I20260709 13:11:57.390457 22911 raft_consensus.cc:606] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:11:57.523590 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:57.428372 22759 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:57.372866 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1899) took 1875 ms. Trace:
I20260709 13:11:57.374085 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 402) took 1699 ms. Trace:
W20260709 13:11:57.390111 24571 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 13:11:57.538206 24603 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:11:57.429162 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7810684439999704s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 13:11:57.539044 22732 connection.cc:256] RPC call timeout handler was delayed by -1.743268444000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:11:57.539399 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7315684440000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 13:11:57.539705 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6982684440000639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 13:11:57.540030 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6757684440000276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
W20260709 13:11:57.540328 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6127684440000394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
W20260709 13:11:57.540608 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5974684440000146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 13:11:57.541121 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5656763700001193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
W20260709 13:11:57.427624 22880 connection.cc:256] RPC call timeout handler was delayed by -1.2939940129999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260709 13:11:57.375344 24588 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:57.447772 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:57.375852 24614 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.520118 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:57.561590 24614 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:57.562924 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:57.563459 22730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1021) took 1925 ms. Trace:
I20260709 13:11:57.428361 24476 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:57.421816 23431 connection.cc:256] RPC call timeout handler was delayed by -1.0981839900000523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 13:11:57.564428 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:57.564991 23431 connection.cc:256] RPC call timeout handler was delayed by -1.0839561470000945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.231s for a response
W20260709 13:11:57.565388 23431 connection.cc:256] RPC call timeout handler was delayed by -1.0401814240000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
W20260709 13:11:57.565807 23431 connection.cc:256] RPC call timeout handler was delayed by -1.0354671970001164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2959s for a response
W20260709 13:11:57.567370 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.553s (SENT)
I20260709 13:11:57.568048 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:57.411326 22832 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:57.457430 22763 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:57.457924 22758 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:11:57.578612 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1856) took 1151 ms. Trace:
I20260709 13:11:57.579192 22758 rpcz_store.cc:276] 0709 13:11:56.427248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.370469 (+943221us) service_pool.cc:224] Handling call
0709 13:11:57.578496 (+208027us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.458508 22806 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.904s (SENT)
I20260709 13:11:57.459208 22836 raft_consensus.cc:2889] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:57.453686 22841 rpcz_store.cc:276] 0709 13:11:55.849725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.855741 (+  6016us) service_pool.cc:224] Handling call
0709 13:11:55.920593 (+ 64852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.195397 (+1274804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.195689 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.507793 22765 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:11:57.603814 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1852) took 1501 ms (client timeout 1995 ms). Trace:
W20260709 13:11:57.604440 22765 rpcz_store.cc:269] 0709 13:11:56.102412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.345689 (+1243277us) service_pool.cc:224] Handling call
0709 13:11:57.603722 (+258033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.458724 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:57.425673 24469 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:57.420360 24478 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:57.351116 24521 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.350500 24608 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.521171 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.428247 22838 rpcz_store.cc:269] 0709 13:11:54.438005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.439577 (+  1572us) service_pool.cc:224] Handling call
0709 13:11:57.285154 (+2845577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.450268 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1004) took 1669 ms. Trace:
W20260709 13:11:57.525750 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1851) took 1589 ms (client timeout 1956 ms). Trace:
I20260709 13:11:57.566638 24086 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:57.528450 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.432261 22757 rpcz_store.cc:269] 0709 13:11:53.835859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.236109 (+3400250us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:57.254151 (+ 18042us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:11:57.525736 22914 rpcz_store.cc:276] 0709 13:11:55.497394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.498022 (+   628us) service_pool.cc:224] Handling call
0709 13:11:55.561311 (+ 63289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.562571 (+  1260us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:55.750738 (+188167us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.372230 (+1621492us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.372438 (+   208us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:57.372543 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.372644 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.372792 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52629,"prepare.queue_time_us":67350,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":62866,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":9392,"spinlock_wait_cycles":54912,"thread_start_us":1743283,"threads_started":2}
I20260709 13:11:57.459362 24602 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.532193 22833 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.532290 22764 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:57.534411 24619 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:57.635612 24626 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.636039 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.635911 24619 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:57.636225 24626 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:57.637038 24580 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:57.637518 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.639017 23465 raft_consensus.cc:1217] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:57.639333 24619 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:57.639637 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 181189us
W20260709 13:11:57.541653 22880 connection.cc:256] RPC call timeout handler was delayed by -1.1555370129999574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260709 13:11:57.640188 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 197) took 1741 ms. Trace:
W20260709 13:11:57.640342 22806 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.931s (SENT)
I20260709 13:11:57.421988 24548 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:57.641477 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.935s (SENT)
I20260709 13:11:57.641964 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:57.643154 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.982s (SENT)
I20260709 13:11:57.643299 24548 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:57.542737 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5227684440000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
W20260709 13:11:57.643999 22732 connection.cc:256] RPC call timeout handler was delayed by -1.427368443999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 13:11:57.644403 24604 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.430s	user 0.002s	sys 0.010s
W20260709 13:11:57.644500 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3787684440000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
I20260709 13:11:57.567842 24588 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:57.644959 22732 connection.cc:256] RPC call timeout handler was delayed by -1.360768443999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 13:11:57.505189 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.544207 24574 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:57.544822 24612 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:11:57.378607 24610 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:57.545781 24522 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.545698 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:57.547129 24616 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.657676 24522 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.658006 24616 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.507311 24611 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:57.659432 24611 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.660398 24522 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.506969 24605 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:57.661901 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1905) took 2013 ms. Trace:
I20260709 13:11:57.525978 24584 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:57.570513 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.612289 24607 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:57.580504 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:57.564337 22730 rpcz_store.cc:276] 0709 13:11:55.637617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.563357 (+1925740us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:11:57.673132 22730 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 13:11:57.527606 24590 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:57.612255 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.612077 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.616125 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 157459us
I20260709 13:11:57.618613 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:57.678237 22913 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:57.620304 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.679045 22765 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:57.620481 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.531555 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1842) took 3688 ms (client timeout 2078 ms). Trace:
W20260709 13:11:57.681654 22761 rpcz_store.cc:269] 0709 13:11:53.837388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.510723 (+3673335us) service_pool.cc:211] Skipping call since client already timed out
0709 13:11:57.523946 (+ 13223us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:11:57.682984 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.683832 22761 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:57.684700 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.618520 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.685695 24582 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.653458 24086 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:57.686394 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:57.687001 22841 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:57.688272 24086 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.565290 22907 raft_consensus.cc:2376] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4. Re-sending same reply.
W20260709 13:11:57.626765 23431 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.084s (SENT)
I20260709 13:11:57.689421 24572 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:57.629230 22766 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:57.691130 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:57.661680 24605 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.691702 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 131) took 2114 ms. Trace:
I20260709 13:11:57.692214 22915 rpcz_store.cc:276] 0709 13:11:55.577644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.039275 (+461631us) service_pool.cc:224] Handling call
0709 13:11:56.108522 (+ 69247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:56.612957 (+504435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:56.612998 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:56.756669 (+143671us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.332420 (+575751us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:57.332470 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.332545 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:57.691461 (+358916us) raft_consensus.cc:1685] finished
0709 13:11:57.691484 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.691635 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44290,"raft.queue_time_us":723700,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":29195,"thread_start_us":332079,"threads_started":1,"wal-append.queue_time_us":891258}
I20260709 13:11:57.693709 24605 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.694070 24412 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.694710 22915 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:57.695343 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 136) took 2025 ms. Trace:
W20260709 13:11:57.681192 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 132) took 2103 ms (client timeout 2182 ms). Trace:
I20260709 13:11:57.695664 22915 rpcz_store.cc:276] 0709 13:11:55.670138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.693792 (+2023654us) service_pool.cc:224] Handling call
0709 13:11:57.694353 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.695087 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.695287 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.695782 22913 rpcz_store.cc:269] 0709 13:11:55.578047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.799528 (+221481us) service_pool.cc:224] Handling call
0709 13:11:57.618446 (+1818918us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:57.681110 (+ 62664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8776448}
I20260709 13:11:57.450505 24628 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:57.629935 22759 rpcz_store.cc:269] 0709 13:11:55.936169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.335114 (+1398945us) service_pool.cc:224] Handling call
0709 13:11:57.525680 (+190566us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.564435 24614 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:57.631822 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 108473us
I20260709 13:11:57.629769 22835 rpcz_store.cc:276] 0709 13:11:55.763855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.768141 (+  4286us) service_pool.cc:224] Handling call
0709 13:11:57.433194 (+1665053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.626058 24625 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.635267 24627 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.612169 24585 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:57.724184 24585 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:57.645304 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2002763700000969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
W20260709 13:11:57.725658 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1678763700000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
W20260709 13:11:57.726193 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0341037089999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
I20260709 13:11:57.725433 24585 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.726897 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:57.727269 24585 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:57.728363 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:57.725394 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1902) took 2142 ms. Trace:
W20260709 13:11:57.644398 22806 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.999s (SENT)
I20260709 13:11:57.650897 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.652474 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:57.624886 24541 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:57.646711 23465 rpcz_store.cc:276] 0709 13:11:55.898644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.161278 (+262634us) service_pool.cc:224] Handling call
0709 13:11:57.638501 (+1477223us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:57.638746 (+   245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.639798 (+  1052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:57.639834 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.639886 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:57.639927 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.639995 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.640120 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9176,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":283,"spinlock_wait_cycles":5436672}
I20260709 13:11:57.729239 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:57.613991 24469 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:57.655282 22908 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
W20260709 13:11:57.737392 22908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1906) took 2044 ms (client timeout 1992 ms). Trace:
I20260709 13:11:57.524945 23458 rpcz_store.cc:276] 0709 13:11:55.774883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.856973 (+ 82090us) service_pool.cc:224] Handling call
0709 13:11:56.263456 (+406483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.257708 (+994252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.296513 (+ 38805us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.662361 22912 rpcz_store.cc:276] 0709 13:11:55.648956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.326170 (+1677214us) service_pool.cc:224] Handling call
0709 13:11:57.350806 (+ 24636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.351072 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:57.417704 (+ 66632us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.547881 (+130177us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b18002f97a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:11:57.661471 (+113590us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.661560 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:57.661623 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.661698 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.661833 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53631,"prepare.queue_time_us":61893,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"spinlock_wait_cycles":10676992,"thread_start_us":59561,"threads_started":1}
I20260709 13:11:57.460065 24544 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.668311 24470 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:57.669041 24333 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:57.670145 22760 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:57.670270 24476 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.672629 22911 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:57.603135 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1849) took 1797 ms. Trace:
I20260709 13:11:57.680025 22762 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:57.688889 24610 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:57.778980 22763 rpcz_store.cc:276] 0709 13:11:55.805993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.379226 (+1573233us) service_pool.cc:224] Handling call
0709 13:11:57.603046 (+223820us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.612833 24613 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:11:57.691025 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1127) took 2971 ms (client timeout 2219 ms). Trace:
I20260709 13:11:57.673815 22730 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 129)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 123)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1854)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 128)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 132)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1855)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 130)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 140)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 135)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 136)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 125)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 137)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 133)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1010)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 112)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 119)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 120)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1847)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1853)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 131)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 134)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 139)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1022)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1023)
I20260709 13:11:57.691702 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1850) took 1846 ms. Trace:
I20260709 13:11:57.781555 22766 rpcz_store.cc:276] 0709 13:11:55.844993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.453437 (+1608444us) service_pool.cc:224] Handling call
0709 13:11:57.691640 (+238203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.756597 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 153) took 2049 ms. Trace:
I20260709 13:11:57.699745 24626 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:57.650704 24574 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:57.702303 22839 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:57.784044 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1017) took 2178 ms. Trace:
W20260709 13:11:57.784324 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 155) took 1949 ms (client timeout 1841 ms). Trace:
I20260709 13:11:57.703576 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.784588 22762 rpcz_store.cc:276] 0709 13:11:55.606200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.420826 (+1814626us) service_pool.cc:224] Handling call
0709 13:11:57.620255 (+199429us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:57.783772 (+163517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106197888}
I20260709 13:11:57.704329 24628 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:57.704941 24594 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.847s	user 0.005s	sys 0.008s
I20260709 13:11:57.705462 24546 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:57.706979 24521 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.706912 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.708765 24608 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.531391 23457 rpcz_store.cc:276] 0709 13:11:55.668571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.679379 (+ 10808us) service_pool.cc:224] Handling call
0709 13:11:55.684919 (+  5540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.703478 (+ 18559us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:55.805750 (+102272us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.303813 (+1498063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.309691 (+  5878us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:57.323803 (+ 14112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.335739 (+ 11936us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.368225 (+ 32486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21585,"prepare.queue_time_us":84050,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":42305,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8861,"thread_start_us":1433039,"threads_started":1,"wal-append.queue_time_us":1433264}
I20260709 13:11:57.803381 24544 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:57.709756 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.804652 22838 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.806216 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.656193 24433 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:57.634809 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.808053 22836 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.808983 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.807180 22838 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.809820 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.810014 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:57.810546 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.810829 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:57.811225 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.811241 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.812500 23458 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:57.812619 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.717376 22913 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:57.719844 23462 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:11:57.723768 24627 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:57.648269 24580 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:57.639338 22757 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:57.729600 22914 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
W20260709 13:11:57.647066 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.732388 24541 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:57.734727 24611 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.735131 24616 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.691859 22907 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:57.735934 24548 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.736519 23459 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:57.730191 22910 rpcz_store.cc:276] 0709 13:11:55.583190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.268587 (+1685397us) service_pool.cc:224] Handling call
0709 13:11:57.295892 (+ 27305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.304579 (+  8687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:57.304626 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:57.324697 (+ 20071us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.630857 (+306160us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:57.630933 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.631056 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:57.725006 (+ 93950us) raft_consensus.cc:1685] finished
0709 13:11:57.725047 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.725305 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30941,"raft.queue_time_us":48449,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":42692,"thread_start_us":251900,"threads_started":1,"wal-append.queue_time_us":378967}
I20260709 13:11:57.813861 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:57.852619 22838 raft_consensus.cc:2376] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4. Re-sending same reply.
I20260709 13:11:57.852941 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.738307 22841 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:11:57.737818 22908 rpcz_store.cc:269] 0709 13:11:55.692540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.770775 (+ 78235us) service_pool.cc:224] Handling call
0709 13:11:57.545389 (+1774614us) spinlock_profiling.cc:243] Waited 893 us on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:11:57.737277 (+191888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055040}
I20260709 13:11:57.855013 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.854130 22757 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:11:57.742048 22879 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.622s (SENT) [suppressed 51 similar messages]
I20260709 13:11:57.654716 24602 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:57.679924 24478 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:57.777491 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.777566 24584 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:57.771937 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 127560us
I20260709 13:11:57.753690 24333 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:57.689517 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:57.781188 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52911.3us
I20260709 13:11:57.813920 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:57.781315 22909 rpcz_store.cc:269] 0709 13:11:54.719289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.719520 (+   231us) service_pool.cc:224] Handling call
0709 13:11:57.690830 (+2971310us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000eae54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:57.690932 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415993600}
I20260709 13:11:57.785938 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.784842 22839 rpcz_store.cc:269] 0709 13:11:55.835244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.835576 (+   332us) service_pool.cc:224] Handling call
0709 13:11:57.784235 (+1948659us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:57.782619 22840 rpcz_store.cc:276] 0709 13:11:55.706923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.707040 (+   117us) service_pool.cc:224] Handling call
0709 13:11:55.747720 (+ 40680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.452393 (+1704673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.756191 (+303798us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54000a4e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:57.756508 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3767771392}
I20260709 13:11:57.753186 24470 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:57.794276 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:57.772464 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1848) took 2023 ms (client timeout 2455 ms). Trace:
I20260709 13:11:57.797379 24628 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:57.798111 24574 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:57.784370 24386 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:11:57.793797 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.685307 22761 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:57.856334 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:57.741811 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98496.8us
I20260709 13:11:57.958217 22763 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:11:57.844578 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.876987 22806 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.020s (SENT)
I20260709 13:11:57.877406 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 139) took 2205 ms. Trace:
I20260709 13:11:57.856683 23462 pending_rounds.cc:85] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 2
W20260709 13:11:57.878108 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.776153 24624 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.877625 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 138) took 2206 ms. Trace:
W20260709 13:11:57.950759 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 324016us
I20260709 13:11:57.978355 23462 pending_rounds.cc:107] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
W20260709 13:11:57.827207 24576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.543s	user 0.038s	sys 0.018s
I20260709 13:11:57.810081 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.989981 22836 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:57.992252 22836 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:11:57.992740 22836 pending_rounds.cc:85] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 1
I20260709 13:11:57.959113 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:57.979420 23462 raft_consensus.cc:2917] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:11:57.979722 22759 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:57.814448 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:57.979753 22839 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:58.002899 23462 raft_consensus.cc:2889] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:57.980114 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1855) took 1553 ms. Trace:
W20260709 13:11:57.980465 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.162s (SENT)
I20260709 13:11:58.004460 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:11:57.879611 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 137592us
I20260709 13:11:57.981048 22913 rpcz_store.cc:276] 0709 13:11:55.672388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.696669 (+2024281us) service_pool.cc:224] Handling call
0709 13:11:57.716992 (+ 20323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.877088 (+160096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.877336 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:57.982270 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.982337 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105400us
I20260709 13:11:58.003798 22763 rpcz_store.cc:276] 0709 13:11:56.426846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.780292 (+1353446us) service_pool.cc:224] Handling call
0709 13:11:57.980032 (+199740us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.006784 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.008037 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:58.008038 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:57.879591 24541 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:57.993432 22836 pending_rounds.cc:107] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:11:57.871363 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.010419 22836 raft_consensus.cc:2889] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.012526 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.010078 22839 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:58.012163 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.017922 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.018572 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.019039 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.021000 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.024194 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.502s (SENT)
I20260709 13:11:57.981489 24625 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:57.989105 24576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.716s	user 0.046s	sys 0.024s
W20260709 13:11:58.006942 22806 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.078s (SENT)
W20260709 13:11:58.028811 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.026328 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:57.987660 24628 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:58.029942 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.027611 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.027693 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 154) took 2291 ms. Trace:
I20260709 13:11:58.031121 22763 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.032215 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:58.027617 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 124) took 2527 ms (client timeout 2239 ms). Trace:
I20260709 13:11:58.032881 22763 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:58.027976 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:58.032968 22764 rpcz_store.cc:269] 0709 13:11:55.500639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.289948 (+1789309us) service_pool.cc:224] Handling call
0709 13:11:58.027411 (+737463us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54000028d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:58.027538 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554542336}
I20260709 13:11:58.033933 22759 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 2: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:57.980108 24478 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.035666 24616 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:58.035882 24507 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:58.036931 24608 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:58.037418 22910 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:58.039026 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:11:58.008069 24579 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.781s	user 0.013s	sys 0.000s
I20260709 13:11:58.040042 22910 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:58.034708 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:57.782526 24476 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.041112 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.059128 22758 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 2: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:58.035727 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:58.045677 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.158s (SENT)
I20260709 13:11:58.059757 22765 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
W20260709 13:11:58.060325 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101303us
I20260709 13:11:58.042299 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.031147 22837 rpcz_store.cc:276] 0709 13:11:55.725138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.725654 (+   516us) service_pool.cc:224] Handling call
0709 13:11:55.763445 (+ 37791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.851412 (+ 87967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.016349 (+2164937us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:58.016528 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146180352}
I20260709 13:11:58.061528 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.004133 24580 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:58.049587 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 196) took 2209 ms. Trace:
I20260709 13:11:57.994262 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 123) took 2680 ms. Trace:
I20260709 13:11:57.789672 24469 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:57.955518 22760 rpcz_store.cc:269] 0709 13:11:55.749244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.383891 (+1634647us) service_pool.cc:224] Handling call
0709 13:11:57.772309 (+388418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.011520 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.093855 22910 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:11:58.111368 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45124us
W20260709 13:11:58.100059 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54460.1us
I20260709 13:11:58.121704 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.029870 24562 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:58.064196 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1854) took 1671 ms. Trace:
I20260709 13:11:58.115046 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.068428 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:58.113427 23431 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.925s (SENT)
I20260709 13:11:58.057247 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.093643 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.064836 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:58.122219 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.128878 22764 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:58.117911 23462 rpcz_store.cc:276] 0709 13:11:55.819192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.819371 (+   179us) service_pool.cc:224] Handling call
0709 13:11:55.845526 (+ 26155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.028276 (+2182750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.028492 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.123142 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.052162 22759 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:11:58.125186 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:58.129822 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.130545 22765 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:58.131376 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.022058 24574 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:58.131724 22806 connection.cc:256] RPC call timeout handler was delayed by -2.293752789999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
I20260709 13:11:58.128523 22758 rpcz_store.cc:276] 0709 13:11:56.392928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.012671 (+1619743us) service_pool.cc:224] Handling call
0709 13:11:58.064114 (+ 51443us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.058266 24548 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:58.001945 24508 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:11:58.134213 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.131824 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 125) took 1080 ms. Trace:
W20260709 13:11:58.135455 22806 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.056s (SENT)
W20260709 13:11:58.098436 24635 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:11:58.113118 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:58.140475 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.141708 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 387430us
I20260709 13:11:58.142899 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.128877 22913 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:58.182595 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.097440 23461 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:58.101194 24611 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.100775 24627 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:58.050308 24620 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:58.147014 22764 rpcz_store.cc:276] 0709 13:11:57.050356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.033874 (+983518us) service_pool.cc:224] Handling call
0709 13:11:58.130172 (+ 96298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.147185 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:58.133436 23458 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:58.056883 24628 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.155781 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.120855 24507 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:11:58.156520 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.157431 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.100998 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1010) took 1087 ms. Trace:
I20260709 13:11:58.063513 24580 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:11:58.064240 24579 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.222764 24580 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.216385 24386 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.221884 24648 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.099475 24602 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.040383 24616 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:11:58.236251 24648 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.238495 24616 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:58.239032 24616 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:58.240427 24616 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.057660 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:58.270936 23457 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:58.105994 24608 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:11:58.111863 24610 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:11:58.270977 24616 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:58.274317 24610 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:58.274714 24610 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:58.275774 24610 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.139243 24605 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:58.205745 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.280658 24333 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.040827 24521 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.137s
I20260709 13:11:58.133587 24626 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:58.304105 24626 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.050699 24440 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 13:11:58.140249 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.114511 22761 rpcz_store.cc:276] 0709 13:11:55.314362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.684430 (+2370068us) service_pool.cc:224] Handling call
0709 13:11:57.985321 (+300891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.157963 24086 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:11:58.214358 22807 connection.cc:256] RPC call timeout handler was delayed by -1.1151599999999462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:11:58.214180 23466 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:11:58.160413 24469 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.219275 24641 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.224965 23462 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:58.223233 22910 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:58.225512 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.335608 22764 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.145566 24590 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:11:58.226028 24584 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.223812 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.200991 24611 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.227712 24579 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:58.229115 23464 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:58.225291 24386 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.230034 24620 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:58.373201 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.374608 22764 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.376005 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.235172 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.224373 22758 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.397962 22764 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:58.398674 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 133) took 1019 ms. Trace:
I20260709 13:11:58.100641 24588 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:57.978296 22915 rpcz_store.cc:276] 0709 13:11:55.671931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.696594 (+2024663us) service_pool.cc:224] Handling call
0709 13:11:57.700614 (+  4020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.700880 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:57.705427 (+  4547us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.877245 (+171818us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:57.877306 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:57.877360 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.877439 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.877568 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7019,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":142,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":78000,"thread_start_us":171234,"threads_started":1}
I20260709 13:11:58.398998 22764 rpcz_store.cc:276] 0709 13:11:57.379669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.375459 (+995790us) service_pool.cc:224] Handling call
0709 13:11:58.398589 (+ 23130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.246832 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1904) took 2631 ms. Trace:
I20260709 13:11:58.248122 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:58.195730 23460 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 LEARNER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:58.249765 22760 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.138773 24412 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:11:58.176831 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.157621 24433 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.224079 24580 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:58.266228 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:11:58.268215 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.268567 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.282739 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.283888 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.131664 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:58.313967 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 188894us
I20260709 13:11:58.416266 22766 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.416496 22759 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.417049 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.415761 22762 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:58.314342 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 108460us
W20260709 13:11:58.314986 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.418102 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:58.315577 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.322408 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 187002us
W20260709 13:11:58.322790 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 269484us
I20260709 13:11:58.328704 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.420866 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:58.419411 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.339243 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.309952 24610 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:58.306430 24605 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:58.350986 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.360682 22913 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:58.368304 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.361459 24611 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.221480 22834 rpcz_store.cc:276] 0709 13:11:57.013751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.317384 (+303633us) service_pool.cc:224] Handling call
0709 13:11:58.043995 (+726611us) spinlock_profiling.cc:243] Waited 9.13 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:58.057863 (+ 13868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.100517 (+ 42654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.100558 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:58.100607 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:58.100657 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.100765 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.100920 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9807,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":332,"spinlock_wait_cycles":21005184}
I20260709 13:11:58.375404 24620 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:58.384471 24585 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.657s	user 0.015s	sys 0.008s
I20260709 13:11:58.412074 22838 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:58.386324 24628 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:58.386821 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.387912 24648 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.366714 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:11:58.400627 22911 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.615s	user 0.017s	sys 0.004s
I20260709 13:11:58.341846 24590 consensus_queue.cc:1243] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:11:58.369882 23459 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:58.404522 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.434805 22757 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:58.407337 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:58.407900 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.408776 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:58.408831 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:58.454011 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.409816 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 348469us
I20260709 13:11:58.409759 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.452082 24470 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.455448 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.455621 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:58.456110 22760 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.456380 22836 raft_consensus.cc:2376] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4. Re-sending same reply.
I20260709 13:11:58.456525 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:58.434602 22835 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.458081 22757 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 1. Current term is 3. Ops: []
I20260709 13:11:58.458906 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1799) took 5072 ms. Trace:
I20260709 13:11:58.459187 22835 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:58.459268 22757 rpcz_store.cc:276] 0709 13:11:53.387016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.457197 (+5070181us) service_pool.cc:224] Handling call
0709 13:11:58.457874 (+   677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.458651 (+   777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.458847 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.459960 22838 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:58.411360 22765 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:58.409173 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.413002 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.460708 22757 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:58.461151 22912 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 2.
I20260709 13:11:58.461709 22765 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:58.461946 22764 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:58.462308 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1807) took 5070 ms. Trace:
I20260709 13:11:58.462496 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1808) took 5070 ms. Trace:
I20260709 13:11:58.426793 24627 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:11:58.415805 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.462664 22765 rpcz_store.cc:276] 0709 13:11:53.391680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.461080 (+5069400us) service_pool.cc:224] Handling call
0709 13:11:58.461369 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.462093 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.462253 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.419514 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:58.420058 22806 connection.cc:256] RPC call timeout handler was delayed by -2.0979517309999665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 13:11:58.464524 22806 connection.cc:256] RPC call timeout handler was delayed by -2.0454454369998984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24550000000000002s for a response
I20260709 13:11:58.464675 22765 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:11:58.408141 22907 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:58.465049 22765 pending_rounds.cc:85] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 2
I20260709 13:11:58.465425 22765 pending_rounds.cc:107] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:11:58.428584 24476 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.435889 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.435909 22763 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:58.434644 22840 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:58.433483 22833 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:58.440704 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:58.441988 22837 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:58.469071 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1783) took 5113 ms. Trace:
I20260709 13:11:58.469966 22913 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:58.443521 22910 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:58.338985 24641 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.445361 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.482128 24626 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.428409 24602 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.453223 22758 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.506168 22840 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:11:58.506227 24602 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.506968 24602 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.507287 22758 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:58.508008 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1810) took 5115 ms. Trace:
I20260709 13:11:58.508353 22758 rpcz_store.cc:276] 0709 13:11:53.392953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.506712 (+5113759us) service_pool.cc:224] Handling call
0709 13:11:58.507098 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.507726 (+   628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.507939 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:58.510177 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.511509 22758 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:58.429590 24588 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:58.516108 23464 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 198) took 2439 ms (client timeout 2255 ms). Trace:
I20260709 13:11:58.516124 24588 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.516932 24588 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.517880 22909 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:58.462872 22764 rpcz_store.cc:276] 0709 13:11:53.392032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.461320 (+5069288us) service_pool.cc:224] Handling call
0709 13:11:58.461653 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.462256 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.462445 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:58.519342 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.519605 22758 pending_rounds.cc:85] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 1
I20260709 13:11:58.520346 22758 pending_rounds.cc:107] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 13:11:58.465016 22806 connection.cc:256] RPC call timeout handler was delayed by -2.0310454369999889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24710000000000001s for a response
W20260709 13:11:58.521560 22806 connection.cc:256] RPC call timeout handler was delayed by -1.8631466019999152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3048s for a response
W20260709 13:11:58.522051 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4847192519999908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29500000000000004s for a response
W20260709 13:11:58.522352 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2324973730000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 13:11:58.521493 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.521236 22758 raft_consensus.cc:2889] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:58.466986 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:58.467967 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48518.9us
I20260709 13:11:58.434068 24521 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.439683 22911 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:11:58.525691 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1148) took 1003 ms. Trace:
I20260709 13:11:58.435207 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1910) took 1064 ms. Trace:
I20260709 13:11:58.401621 22906 rpcz_store.cc:276] 0709 13:11:55.615230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.285261 (+1670031us) service_pool.cc:224] Handling call
0709 13:11:57.367930 (+ 82669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.527346 (+159416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:57.532932 (+  5586us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:57.548550 (+ 15618us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.699707 (+151157us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:57.699776 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.699861 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:58.246466 (+546605us) raft_consensus.cc:1685] finished
0709 13:11:58.246504 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.246734 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92128,"raft.queue_time_us":106267,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":1131,"thread_start_us":149009,"threads_started":1,"wal-append.queue_time_us":244505}
I20260709 13:11:58.526059 22911 rpcz_store.cc:276] 0709 13:11:57.522576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.580145 (+ 57569us) service_pool.cc:224] Handling call
0709 13:11:58.525611 (+945466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.438165 24645 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:11:58.473105 22761 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:58.474412 24641 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.475349 22760 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:58.461385 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1803) took 5072 ms. Trace:
I20260709 13:11:58.480574 22766 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:58.446152 24605 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.539474 22757 rpcz_store.cc:276] 0709 13:11:53.389341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.460147 (+5070806us) service_pool.cc:224] Handling call
0709 13:11:58.460388 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.461128 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.461332 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.469996 24627 consensus_queue.cc:1243] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:58.543529 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.543694 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1789) took 5185 ms. Trace:
I20260709 13:11:58.544451 22834 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:58.484572 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.546288 22762 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:58.489034 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:58.547233 22833 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:58.544577 22760 rpcz_store.cc:276] 0709 13:11:53.358079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.456816 (+5098737us) service_pool.cc:224] Handling call
0709 13:11:58.475136 (+ 18320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.543368 (+ 68232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.543619 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.468674 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.481784 22839 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:58.550284 22766 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:58.512950 22759 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:58.515326 24386 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:58.463049 22912 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:58.465839 22765 raft_consensus.cc:2889] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.522270 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:58.469861 22763 rpcz_store.cc:276] 0709 13:11:53.356082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.435230 (+5079148us) service_pool.cc:224] Handling call
0709 13:11:58.435566 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.468712 (+ 33146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.468997 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:58.527510 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.531479 22730 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:58.530993 22764 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:11:58.533736 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.535s (SENT)
I20260709 13:11:58.577651 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1817) took 5166 ms. Trace:
I20260709 13:11:58.534267 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1816) took 5123 ms. Trace:
I20260709 13:11:58.526546 22914 rpcz_store.cc:276] 0709 13:11:57.370701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.856993 (+486292us) service_pool.cc:224] Handling call
0709 13:11:57.857312 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:57.857538 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:58.139369 (+281831us) log.cc:844] Serialized 14 byte log entry
0709 13:11:58.434814 (+295445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:58.434880 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:58.434936 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.435016 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.435137 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24687,"prepare.queue_time_us":137856,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":12536,"raft.queue_time_us":10367,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5062,"spinlock_wait_cycles":36296320,"thread_start_us":464694,"threads_started":2,"wal-append.queue_time_us":305372}
I20260709 13:11:58.577718 22806 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:58.532456 24521 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.522970 24470 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.473344 24576 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:58.551190 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1782) took 5195 ms. Trace:
I20260709 13:11:58.585785 22759 rpcz_store.cc:276] 0709 13:11:53.355789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.512383 (+5156594us) service_pool.cc:224] Handling call
0709 13:11:58.512744 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.550967 (+ 38223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.551124 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:58.574661 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52540.7us
I20260709 13:11:58.530946 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.551190 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 999) took 5087 ms. Trace:
I20260709 13:11:58.601120 22766 rpcz_store.cc:276] 0709 13:11:53.463699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.549636 (+5085937us) service_pool.cc:224] Handling call
0709 13:11:58.550005 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.550915 (+   910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.551124 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.574661 22763 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:58.578547 22758 rpcz_store.cc:276] 0709 13:11:53.410672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.509907 (+5099235us) service_pool.cc:224] Handling call
0709 13:11:58.511128 (+  1221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.533929 (+ 22801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.534207 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.578114 22764 rpcz_store.cc:276] 0709 13:11:53.410982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.519386 (+5108404us) service_pool.cc:224] Handling call
0709 13:11:58.525524 (+  6138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.577305 (+ 51781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.577582 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:58.578205 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 109671us
W20260709 13:11:58.516673 23464 rpcz_store.cc:269] 0709 13:11:56.076902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.162227 (+ 85325us) service_pool.cc:224] Handling call
0709 13:11:58.490264 (+2328037us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54005140d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:58.515981 (+ 25717us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600247680}
I20260709 13:11:58.583235 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1809) took 5190 ms. Trace:
I20260709 13:11:58.606705 22765 rpcz_store.cc:276] 0709 13:11:53.392509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.464033 (+5071524us) service_pool.cc:224] Handling call
0709 13:11:58.464357 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.580309 (+115952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.583137 (+  2828us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.607381 22764 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:58.587971 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.591709 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.592792 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.608593 22762 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:58.605039 22759 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:58.609918 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1844) took 4547 ms. Trace:
I20260709 13:11:58.585318 24576 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:11:58.580197 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46526.6us
I20260709 13:11:58.538730 24333 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
W20260709 13:11:58.588567 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57196.2us
I20260709 13:11:58.581682 24521 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.581558 24580 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:58.611086 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.609062 22840 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:58.601176 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.581184 24386 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:58.608992 22762 pending_rounds.cc:85] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 1
I20260709 13:11:58.634127 22762 pending_rounds.cc:107] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:11:58.634408 22762 raft_consensus.cc:2889] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:58.616820 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.617043 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1010) took 3869 ms. Trace:
W20260709 13:11:58.616948 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 181124us
I20260709 13:11:58.612673 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.609426 22839 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:58.622908 22758 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:58.585651 24470 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.610289 22759 rpcz_store.cc:276] 0709 13:11:54.062798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.599198 (+4536400us) service_pool.cc:224] Handling call
0709 13:11:58.599627 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.609641 (+ 10014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.609854 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.624405 23466 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:58.608074 24541 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.566s	user 0.020s	sys 0.000s
I20260709 13:11:58.600770 24627 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.604583 24626 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.608157 24512 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
W20260709 13:11:58.621479 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.455s (SENT)
I20260709 13:11:58.628135 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.605345 22763 pending_rounds.cc:85] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 1
I20260709 13:11:58.613777 22907 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:58.630076 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:58.671454 22763 pending_rounds.cc:107] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:11:58.633376 23462 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:58.637990 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.671756 22763 raft_consensus.cc:2889] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.660754 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1002) took 4569 ms. Trace:
I20260709 13:11:58.661021 22764 rpcz_store.cc:276] 0709 13:11:54.738654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.606576 (+3867922us) service_pool.cc:224] Handling call
0709 13:11:58.606903 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.607883 (+   980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.608091 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.661599 22759 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:58.672389 22758 rpcz_store.cc:276] 0709 13:11:54.091048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.602862 (+4511814us) service_pool.cc:224] Handling call
0709 13:11:58.622669 (+ 19807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.660437 (+ 37768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.660687 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.663523 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:11:58.629458 24650 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.085s	user 0.000s	sys 0.002s
I20260709 13:11:58.625171 24645 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:11:58.627905 24478 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.665006 24470 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.629309 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:58.632682 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.672640 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.661998 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.691306 22839 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:11:58.673668 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52234.1us
W20260709 13:11:58.692996 22806 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.471s (SENT)
I20260709 13:11:58.625367 24440 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:58.675261 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:58.695083 22833 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.675899 22840 raft_consensus.cc:2410] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:58.664453 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:11:58.677812 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:58.633884 24386 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.678871 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1822) took 4985 ms. Trace:
I20260709 13:11:58.680016 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.700692 22763 rpcz_store.cc:276] 0709 13:11:53.693636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.573594 (+4879958us) service_pool.cc:224] Handling call
0709 13:11:58.574127 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.678616 (+104489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.678814 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.625627 24572 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.630700 24579 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:11:58.683714 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 114) took 3901 ms. Trace:
I20260709 13:11:58.683974 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:58.704807 22759 rpcz_store.cc:276] 0709 13:11:54.774836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.660877 (+3886041us) service_pool.cc:224] Handling call
0709 13:11:58.661319 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.676206 (+ 14887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.676394 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.706673 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.707456 22759 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.667413 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1825) took 4973 ms. Trace:
I20260709 13:11:58.692135 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.693845 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:58.670521 24626 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.698454 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.732856 24626 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.733477 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:58.698748 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:58.689623 22834 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:58.699481 22664 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 131) took 7661 ms. Trace:
I20260709 13:11:58.627636 24628 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:11:58.701812 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.676923 24608 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:58.671087 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:11:58.634557 24536 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:11:58.678746 24580 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:58.703498 24386 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:58.709776 22762 rpcz_store.cc:276] 0709 13:11:53.694478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.581891 (+4887413us) service_pool.cc:224] Handling call
0709 13:11:58.608106 (+ 26215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.667147 (+ 59041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.667352 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:58.709125 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.712065 24590 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.704406 24548 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:11:58.713338 22833 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:11:58.689106 24412 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:11:58.722800 22840 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:58.705448 22836 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
W20260709 13:11:58.724766 24574 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.580s	user 0.022s	sys 0.016s
I20260709 13:11:58.725385 24641 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:58.726431 24562 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.704372 24646 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:11:58.736402 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 107184us
I20260709 13:11:58.683470 24478 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.781203 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:58.786305 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.788331 22765 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 4. Ops: []
I20260709 13:11:58.789218 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 112) took 4035 ms. Trace:
I20260709 13:11:58.789650 22765 rpcz_store.cc:276] 0709 13:11:54.754004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.607738 (+3853734us) service_pool.cc:224] Handling call
0709 13:11:58.787801 (+180063us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:11:58.788089 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.788870 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.789145 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4951296}
I20260709 13:11:58.764389 24590 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.794464 24590 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.743112 22763 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:58.669083 24541 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:58.798627 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.799383 22763 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.800007 24541 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:58.800313 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.768735 24562 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.678417 24605 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:58.758807 22759 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.703292 24440 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:11:58.745792 24610 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:58.761593 22915 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:58.750263 24469 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.697821 24620 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:11:58.772689 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 79730.7us
I20260709 13:11:58.748202 24608 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:11:58.831440 22806 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.950s (SENT)
I20260709 13:11:58.749470 24649 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.736207 22664 rpcz_store.cc:276] 0709 13:11:51.057999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.072154 (+ 14155us) service_pool.cc:224] Handling call
0709 13:11:58.699395 (+7627241us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:52.911668 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:53.019166 (+107498us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:53.019452 (+   286us) write_op.cc:435] Acquired schema lock
0709 13:11:53.019474 (+    22us) tablet.cc:662] Decoding operations
0709 13:11:53.020019 (+   545us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:53.020302 (+   283us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:53.020338 (+    36us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:11:53.020838 (+   500us) tablet.cc:701] Row locks acquired
0709 13:11:53.020888 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:11:53.021147 (+   259us) write_op.cc:270] Start()
0709 13:11:53.021308 (+   161us) write_op.cc:276] Timestamp: P: 1783602713021024 usec, L: 0
0709 13:11:53.021350 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:11:53.155037 (+133687us) log.cc:844] Serialized 907 byte log entry
0709 13:11:53.699681 (+544644us) op_driver.cc:464] REPLICATION: finished
0709 13:11:53.874049 (+174368us) write_op.cc:301] APPLY: starting
0709 13:11:53.874173 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:53.874371 (+   198us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:53.874561 (+   190us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:53.940157 (+ 65596us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:53.940458 (+   301us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:11:53.940595 (+   137us) write_op.cc:312] APPLY: finished
0709 13:11:53.941038 (+   443us) log.cc:844] Serialized 29 byte log entry
0709 13:11:53.941265 (+   227us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:53.941365 (+   100us) write_op.cc:454] Released schema lock
0709 13:11:53.941430 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:53.941868 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:53.941901 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":379862,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":173850,"apply.run_cpu_time_us":3818,"apply.run_wall_time_us":67916,"num_ops":2,"prepare.queue_time_us":525983,"prepare.run_cpu_time_us":41820,"prepare.run_wall_time_us":439965,"raft.queue_time_us":85618,"raft.run_cpu_time_us":24009,"raft.run_wall_time_us":98936,"replication_time_us":677883,"thread_start_us":377665,"threads_started":4,"wal-append.queue_time_us":302936,"wal-append.run_cpu_time_us":19171,"wal-append.run_wall_time_us":1447203}]]}
I20260709 13:11:58.740418 24572 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.764591 24548 consensus_queue.cc:1243] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:58.704900 24611 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:58.793358 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 92851us
I20260709 13:11:58.808796 24605 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.739490 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.792714 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.783403 23464 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:11:58.802580 22763 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:58.783385 24478 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:58.852329 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 112942us
I20260709 13:11:58.852922 22837 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:11:58.853621 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.784505 22834 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:58.784840 24580 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.859079 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:58.810154 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.867966 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.868795 22759 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.813581 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.869912 22762 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:11:58.870741 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 136326us
W20260709 13:11:58.872618 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80038.5us
I20260709 13:11:58.840862 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:58.805748 24562 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.845927 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.886349 22758 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:58.703282 24584 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:58.810284 24469 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.845461 22832 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:58.809985 24610 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:11:58.804543 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.858450 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.855494 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:58.888365 23463 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:58.928937 22765 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:58.929311 22763 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:58.929008 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73596.2us
I20260709 13:11:58.890266 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:58.931483 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.850443 24605 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:58.898941 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.882027 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:11:58.856843 24624 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.927642 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.932742 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:58.963239 22758 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.866529 24585 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.882483 24412 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.884053 24580 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:58.941825 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49654.7us
I20260709 13:11:58.930559 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:58.889123 24548 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:58.890029 24628 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.954253 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:58.954244 22759 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:58.894341 24584 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.902683 24469 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:58.961639 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 130269us
I20260709 13:11:58.957854 24641 consensus_queue.cc:1243] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:11:58.968986 24654 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:11:58.970141 24660 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:58.925881 24521 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.039255 24521 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:58.996358 24412 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.042152 24412 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.045959 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 997) took 5583 ms. Trace:
I20260709 13:11:58.903355 24608 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.047552 24433 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:59.047011 24469 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.048091 24433 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.050654 24469 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:58.965332 24585 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.056762 24585 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:59.060202 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.030589 24584 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:59.027716 24649 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:11:59.066826 24649 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:59.067227 24649 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.067612 24649 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:59.022841 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1021) took 1464 ms (client timeout 1833 ms). Trace:
W20260709 13:11:59.068826 22832 rpcz_store.cc:269] 0709 13:11:57.558527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.706493 (+147966us) service_pool.cc:224] Handling call
0709 13:11:59.022597 (+1316104us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:11:59.022770 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292171136}
I20260709 13:11:59.046595 22757 rpcz_store.cc:276] 0709 13:11:53.463021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.541053 (+5078032us) service_pool.cc:224] Handling call
0709 13:11:58.541560 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:58.541969 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.542011 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:58.542618 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:11:58.665614 (+122996us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:58.677184 (+ 11570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.681975 (+  4791us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.045604 (+363629us) raft_consensus.cc:1685] finished
0709 13:11:59.045637 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.045868 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1663,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":124,"thread_start_us":84671,"threads_started":1,"wal-append.queue_time_us":160872}
I20260709 13:11:58.971149 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:59.075064 22765 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:11:58.971724 24627 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:58.971897 24667 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.997335 22763 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:59.041376 24521 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.964848 22762 raft_consensus.cc:2376] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4. Re-sending same reply.
I20260709 13:11:58.935496 24610 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:58.883697 24588 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.955009 24582 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.896348 24590 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:58.960381 24086 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.004535 22660 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:59.013947 24656 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
W20260709 13:11:59.017515 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.019876 24576 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.022804 24611 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.024236 24572 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.024762 24548 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.026374 22841 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.026666 22758 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
W20260709 13:11:59.029994 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.952s (SENT)
I20260709 13:11:59.128332 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.003450 24580 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.033442 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:59.033749 24641 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.075495 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:59.047992 24608 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:59.011315 24602 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.502s	user 0.013s	sys 0.000s
I20260709 13:11:58.999205 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:11:59.142603 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 143602us
I20260709 13:11:59.107187 22763 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:59.145992 22763 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.114713 24086 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:58.951719 24440 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.094060 24521 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.131526 22764 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:11:59.132342 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 242137us
I20260709 13:11:59.133991 22759 raft_consensus.cc:2376] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4. Re-sending same reply.
I20260709 13:11:59.093627 24664 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.046977 24470 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:59.138607 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 108653us
I20260709 13:11:59.111783 24582 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.124943 22660 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:11:59.049366 24433 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.077165 24627 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.149931 22762 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:59.153100 22763 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.155800 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:11:59.163769 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 209910us
I20260709 13:11:59.137418 24576 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:59.167244 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 3.152s (SENT)
W20260709 13:11:59.167658 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.146242 24614 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:59.147296 24572 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:59.218958 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.221098 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.221549 22764 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:59.221714 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54537.5us
W20260709 13:11:59.223783 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:59.224143 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:59.224553 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:59.224891 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.223985 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1020) took 3602 ms. Trace:
I20260709 13:11:59.229694 22764 rpcz_store.cc:276] 0709 13:11:55.622137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.155149 (+3533012us) service_pool.cc:224] Handling call
0709 13:11:59.221342 (+ 66193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.223722 (+  2380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.223929 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:59.228747 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.230813 22909 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:11:59.234670 22764 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:59.235567 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1853) took 2919 ms. Trace:
I20260709 13:11:59.236253 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.237457 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1018) took 3626 ms. Trace:
I20260709 13:11:59.237771 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.238021 22762 rpcz_store.cc:276] 0709 13:11:55.611245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.131485 (+3520240us) service_pool.cc:224] Handling call
0709 13:11:59.149736 (+ 18251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.236810 (+ 87074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.237380 (+   570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:59.152724 24086 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.154045 24521 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.235970 22764 rpcz_store.cc:276] 0709 13:11:56.316287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.233052 (+2916765us) service_pool.cc:224] Handling call
0709 13:11:59.234210 (+  1158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.235232 (+  1022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.235494 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:59.240092 22660 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:59.143786 24641 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.240239 22763 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.279803 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.281193 22908 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:59.214375 23460 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:59.251444 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:59.285729 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.147861 24602 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.252588 22762 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:59.290894 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 131) took 1984 ms. Trace:
I20260709 13:11:59.291345 22762 rpcz_store.cc:276] 0709 13:11:57.306249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.251935 (+1945686us) service_pool.cc:224] Handling call
0709 13:11:59.252248 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.290511 (+ 38263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.290819 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:59.252962 24673 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.253315 24660 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.254133 24627 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:59.266178 24663 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:11:59.267817 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.269378 24674 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:59.273649 22760 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:11:59.277347 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.283244 22836 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:59.216708 22757 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:59.172645 24582 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.252156 24469 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.288707 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.287966 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:11:59.295991 22660 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 3, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:59.298120 22834 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.298772 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 996) took 5836 ms. Trace:
I20260709 13:11:59.300032 22759 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:11:59.360294 24626 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.005s
I20260709 13:11:59.301404 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.301258 24562 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.304618 24662 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.318722 24590 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.255928 24386 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.273444 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:59.231055 24433 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.232441 24611 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.357786 22763 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:11:59.405083 22763 pending_rounds.cc:85] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:11:59.161602 24664 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.389161 22660 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 3, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:59.391762 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 103870us
I20260709 13:11:59.405489 22763 pending_rounds.cc:107] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:11:59.409523 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:59.409920 22763 raft_consensus.cc:2889] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:59.394959 24674 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.392650 24668 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.393551 24678 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.384959 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:59.300772 24641 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:59.400139 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.343591 24660 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.442376 22839 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:59.342633 24673 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:59.434141 24478 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:11:59.396714 24469 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:59.445370 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.284471 24548 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.438169 22660 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 2, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 1 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:59.407716 24627 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:59.386199 24412 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.439582 24590 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.435568 24476 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:59.432248 24668 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.452989 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 120) took 4299 ms. Trace:
W20260709 13:11:59.455816 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 300136us
I20260709 13:11:59.454849 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:59.461905 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.461997 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:59.462937 22660 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 2 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:11:59.407673 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1847) took 4039 ms. Trace:
I20260709 13:11:59.461396 22763 rpcz_store.cc:276] 0709 13:11:55.153386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.087709 (+3934323us) service_pool.cc:224] Handling call
0709 13:11:59.088047 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.452726 (+364679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.452919 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:59.466427 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 228777us
I20260709 13:11:59.441514 24541 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.470584 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:11:59.467311 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.440349 24678 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.473768 24470 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:59.474395 24602 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.475076 24602 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.475374 24660 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.475778 24386 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:11:59.476250 24669 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:11:59.477396 24562 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.478116 24665 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.478629 24412 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.479893 22911 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.480010 24585 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.481338 22909 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.473358 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.484236 24412 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.484618 24412 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.484781 24585 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.485003 24412 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.487424 22837 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:11:59.487735 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.489518 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.490993 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.407135 24626 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.493059 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.504272 24584 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:59.413269 24611 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.469189 24476 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.405201 24677 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:11:59.452909 24675 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:59.475427 24602 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.491178 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.515648 22660 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:59.537884 22838 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:11:59.501780 24470 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.470862 22757 rpcz_store.cc:276] 0709 13:11:55.368348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.105177 (+3736829us) service_pool.cc:224] Handling call
0709 13:11:59.139737 (+ 34560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.383813 (+244076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.404761 (+ 20948us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:59.539516 24470 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:59.540081 24470 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.541062 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1023) took 1877 ms. Trace:
I20260709 13:11:59.470054 24668 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.408464 24662 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.522558 24602 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.523591 24664 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.460093 24649 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.525689 24611 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.517627 24476 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.527220 24588 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.527810 24386 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.527148 24626 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:59.530520 24584 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.533841 24677 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.581153 24626 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.581373 24677 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.582051 24626 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.582553 24626 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.583889 24677 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.536087 24660 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.456284 24478 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.489611 24678 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.491760 24541 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.538516 22660 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:59.492290 24641 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.584625 24660 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:11:59.553624 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60868.5us
I20260709 13:11:59.598475 22764 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:59.599125 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.542830 24469 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:59.554672 24590 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.541003 24470 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:11:59.578176 24679 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:11:59.543126 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:59.502065 24680 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:59.601780 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 109875us
W20260709 13:11:59.598338 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.584568 24626 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.605060 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.608395 22908 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:59.610819 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:11:59.611580 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.543265 22757 rpcz_store.cc:276] 0709 13:11:57.663621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.539981 (+1876360us) service_pool.cc:224] Handling call
0709 13:11:59.540403 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.540622 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:59.540657 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:59.540721 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:59.540773 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.540857 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.540991 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:59.598803 24677 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.574005 24664 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 13:11:59.610114 24605 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.001s
I20260709 13:11:59.622695 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1022) took 1971 ms. Trace:
I20260709 13:11:59.624902 24673 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.625031 24478 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:59.625489 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.628118 24649 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:59.629696 24662 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:59.635944 24541 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:59.637574 24685 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:11:59.587853 24386 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.634694 24626 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.638792 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.640805 24386 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.641852 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.642158 24664 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:59.642582 22906 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:59.642663 24664 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.642957 24548 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.642668 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.643728 24664 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.644868 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.645979 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.648432 24548 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.649282 24548 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.397105 22761 rpcz_store.cc:276] 0709 13:11:53.462705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.537587 (+5074882us) service_pool.cc:224] Handling call
0709 13:11:58.545789 (+  8202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:58.601088 (+ 55299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.607390 (+  6302us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:58.694887 (+ 87497us) log.cc:844] Serialized 26 byte log entry
0709 13:11:59.088960 (+394073us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:59.090569 (+  1609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.111206 (+ 20637us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.298505 (+187299us) raft_consensus.cc:1685] finished
0709 13:11:59.298536 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.298691 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2460,"raft.queue_time_us":11522,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":6084,"thread_start_us":338712,"threads_started":1,"wal-append.queue_time_us":486268}
I20260709 13:11:59.647536 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.650995 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:59.651154 24086 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.651739 22761 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.652274 24680 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.643790 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.654384 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:11:59.666898 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.667520 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.612421 22907 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:59.668609 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:59.669840 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.671286 24680 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.668658 24469 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.672667 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:11:59.654381 24686 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:11:59.674782 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:59.675498 22757 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.675554 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1019) took 4062 ms. Trace:
I20260709 13:11:59.676023 22759 rpcz_store.cc:276] 0709 13:11:55.613488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.153448 (+3539960us) service_pool.cc:224] Handling call
0709 13:11:59.153805 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.395396 (+241591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.675247 (+279851us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540035e654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:11:59.675487 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335470848}
I20260709 13:11:59.680748 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.682127 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:59.609859 24684 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:59.685467 22762 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:59.685298 24668 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.685770 24684 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:11:59.672524 24469 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.673389 22758 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:11:59.687428 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:59.511801 24674 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.692366 22759 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.695389 24582 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.655808 24664 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:59.656149 24605 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:59.696899 24086 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:11:59.698861 24608 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 13:11:59.707829 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.655104 22906 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:11:59.708386 24641 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.713820 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:59.706596 24580 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.005s
I20260709 13:11:59.714141 22913 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:11:59.622856 24677 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.690884 24684 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:59.718266 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:11:59.711846 24602 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.722685 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.720543 22915 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.726253 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.714695 22758 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.719732 22914 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:59.724334 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:11:59.728065 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 1.02658e+06us
I20260709 13:11:59.711937 24680 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:59.749747 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.731055 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.753643 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:59.755339 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.730810 22906 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:59.760741 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.761946 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:11:59.729694 24610 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.601s	user 0.013s	sys 0.000s
I20260709 13:11:59.764189 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 119) took 4659 ms. Trace:
I20260709 13:11:59.747493 24641 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:59.769711 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.764722 22765 rpcz_store.cc:276] 0709 13:11:55.105196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.085663 (+3980467us) service_pool.cc:224] Handling call
0709 13:11:59.108431 (+ 22768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.378968 (+270537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:59.379018 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:59.390265 (+ 11247us) log.cc:844] Serialized 26 byte log entry
0709 13:11:59.519797 (+129532us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:59.763146 (+243349us) log.cc:844] Serialized 14 byte log entry
0709 13:11:59.763718 (+   572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.763842 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.763903 (+    61us) raft_consensus.cc:1685] finished
0709 13:11:59.763922 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.764109 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20799,"prepare.queue_time_us":162083,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":28,"raft.queue_time_us":160128,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":163,"thread_start_us":549724,"threads_started":3,"wal-append.queue_time_us":188146}
I20260709 13:11:59.772590 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:11:59.772667 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.773471 22909 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:11:59.774722 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:11:59.720542 24580 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.712069 24476 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:11:59.779384 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.758746 24680 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:11:59.766108 24478 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:59.850170 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.759416 24086 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:59.783823 24333 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:11:59.782522 24616 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.853113 24677 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:11:59.842777 24694 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.123s
I20260709 13:11:59.755543 24608 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:11:59.867938 22909 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:11:59.770408 24611 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:59.765247 24585 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:11:59.833650 24469 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:11:59.796455 22759 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:59.833782 24602 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.832978 24693 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:59.853855 23432 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.576s (SENT)
I20260709 13:11:59.833307 24697 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:59.907811 24697 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.876747 22758 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:59.774670 24641 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.920246 24641 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.920217 24659 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:11:59.919894 24697 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.833221 24681 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:59.831595 24688 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:11:59.766249 24412 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.832398 24582 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.891340 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:59.931623 22759 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.894811 24668 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:11:59.845443 24689 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:11:59.806470 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:59.875212 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.907943 22832 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:59.884089 24611 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.898415 24605 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:11:59.861033 24584 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:59.859368 24662 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:11:59.782997 24665 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.111s
I20260709 13:11:59.847800 24614 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:11:59.849586 24541 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 13:11:59.903123 24521 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
I20260709 13:11:59.832727 22761 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:59.911019 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:11:59.970491 22758 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.773310 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.884817 24086 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:11:59.886904 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:59.938730 24478 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:11:59.924552 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80716.8us
I20260709 13:11:59.891276 24677 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:59.938166 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 208684us
I20260709 13:11:59.777232 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.897308 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:11:59.942822 24611 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.768061 24610 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:11:59.944481 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.853138 24680 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.946501 22757 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:59.928486 24602 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.965592 22759 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:59.932142 24616 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.968186 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:59.971849 22838 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:11:59.972995 22765 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:59.910648 24585 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:59.975103 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 121341us
I20260709 13:11:59.992748 22758 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:11:59.981748 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:11:59.982537 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6127829429999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
W20260709 13:11:59.987207 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.992094 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.994902 22838 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:11:59.995787 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:11:59.996282 22759 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:59.993988 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.978624 24440 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:00.022820 24658 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:11:59.980965 24627 raft_consensus.cc:2764] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:11:59.980357 24536 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:59.993605 24616 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.023914 22879 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.024737 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 147) took 2227 ms. Trace:
I20260709 13:11:59.978765 24667 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:12:00.025962 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.027401 24476 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.023025 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:59.978962 24677 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:11:59.986528 24674 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.038585 22765 rpcz_store.cc:276] 0709 13:11:57.797627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.991781 (+2194154us) service_pool.cc:224] Handling call
0709 13:12:00.023951 (+ 32170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.024337 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.024374 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:00.024425 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:00.024466 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.024532 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.024673 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5842,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3547}
I20260709 13:11:59.973006 24521 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.058523 22907 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:00.058817 22910 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:00.041824 24086 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.059868 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.061229 24521 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:00.033694 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.068468 24674 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.070283 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:00.038564 24412 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:00.071156 24674 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.071812 24440 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:00.072283 24440 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.072347 24576 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:00.072759 24440 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.073467 24677 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:00.073025 24576 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:11:59.978307 24478 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:00.074322 24576 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.076545 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:00.033981 22759 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.036989 24616 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.038177 24605 raft_consensus.cc:2764] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:12:00.084522 24590 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:00.085105 24590 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.044044 22757 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:00.086596 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:59.987484 24333 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:12:00.020792 24585 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.097347 24585 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.051169 22761 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:00.098254 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1024) took 2420 ms. Trace:
I20260709 13:12:00.098610 22761 rpcz_store.cc:276] 0709 13:11:57.677403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.989944 (+2312541us) service_pool.cc:224] Handling call
0709 13:12:00.042582 (+ 52638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.097980 (+ 55398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.098192 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.099684 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:11:59.996385 24659 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:12:00.099751 24641 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:00.100628 24620 raft_consensus.cc:670] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:00.099781 23464 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:00.046160 24706 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:11:59.990361 24470 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.103611 24470 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:11:59.982960 24707 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.063879 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.065047 24646 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:12:00.134071 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.134689 24646 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:00.067579 24660 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.065665 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 134) took 2631 ms. Trace:
I20260709 13:12:00.065685 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.135658 22764 rpcz_store.cc:276] 0709 13:11:57.433706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.279679 (+1845973us) service_pool.cc:224] Handling call
0709 13:11:59.295178 (+ 15499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.599191 (+304013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.065104 (+465913us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54001004d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.065360 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677163904}
I20260709 13:12:00.136754 22765 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:00.078604 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:11:59.992988 22834 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:00.079607 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.138048 22838 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.138372 22759 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:00.029110 24697 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.038208 24684 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:00.090656 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.095177 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.095459 24478 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:00.109766 24708 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
W20260709 13:12:00.110534 24705 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:00.135141 24646 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:00.111294 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.113644 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.148622 23463 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:12:00.113492 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.116662 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:00.149569 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:00.150974 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 128933us
W20260709 13:12:00.151647 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.873s (SENT)
I20260709 13:12:00.118757 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:00.152040 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.120059 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.152594 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.153071 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.153630 22764 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:00.153803 22759 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:00.153697 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.154631 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.154630 22836 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.990003 24541 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:12:00.087460 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
W20260709 13:12:00.116298 24700 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.342s	user 0.003s	sys 0.000s
I20260709 13:12:00.140074 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.162153 22911 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:00.141971 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:11:59.989531 24602 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.169844 24469 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.174608 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.176734 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.166029 24620 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:12:00.178771 24678 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.009s
I20260709 13:12:00.179576 24678 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.166685 24671 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:00.151117 22761 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:00.168341 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 158) took 2815 ms. Trace:
I20260709 13:12:00.145684 24580 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.184384 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.191466 22764 raft_consensus.cc:2376] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3. Re-sending same reply.
I20260709 13:12:00.183557 22835 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:00.188671 22841 rpcz_store.cc:276] 0709 13:11:57.353201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.988292 (+635091us) service_pool.cc:224] Handling call
0709 13:11:59.026128 (+1037836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.127108 (+100980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.168132 (+1041024us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540019f5d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:00.168276 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385682688}
I20260709 13:12:00.194538 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 169) took 1758 ms. Trace:
I20260709 13:12:00.196559 22907 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:00.201887 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:00.184778 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.988s (SENT)
I20260709 13:12:00.171375 24659 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.199499 24704 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:00.199393 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 76987.5us
I20260709 13:12:00.205580 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:00.202767 22910 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:00.201959 22839 rpcz_store.cc:276] 0709 13:11:58.435743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.697574 (+261831us) service_pool.cc:224] Handling call
0709 13:11:58.698161 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.442907 (+744746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.194204 (+751297us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54003223d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.194460 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1711003520}
I20260709 13:12:00.198843 24684 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.211798 24659 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.166951 24386 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:12:00.194164 24671 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:00.213038 24536 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:00.206292 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.213959 24671 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.214605 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.215251 22909 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:00.166374 24660 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:12:00.189090 22832 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:00.216209 24660 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:00.216572 24660 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:00.216992 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.218186 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.212236 24541 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:00.242491 24541 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.242929 24541 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.243135 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.207080 24646 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.213406 24536 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:00.244504 24646 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.170915 24641 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.220045 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.245970 22764 raft_consensus.cc:2376] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3. Re-sending same reply.
I20260709 13:12:00.203076 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.250125 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1921) took 1025 ms. Trace:
I20260709 13:12:00.250629 22915 rpcz_store.cc:276] 0709 13:11:59.225111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.225252 (+   141us) service_pool.cc:224] Handling call
0709 13:11:59.252901 (+ 27649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.725202 (+472301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.249821 (+524619us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540010e0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.250047 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074815232}
I20260709 13:11:59.633447 22763 rpcz_store.cc:276] 0709 13:11:57.651598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.466408 (+1814810us) service_pool.cc:224] Handling call
0709 13:11:59.466774 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.467003 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:59.608075 (+141072us) log.cc:844] Serialized 14 byte log entry
0709 13:11:59.622307 (+ 14232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:59.622370 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:59.622426 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.622502 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.622633 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17620,"raft.queue_time_us":20525,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":117988,"threads_started":2}
I20260709 13:12:00.257429 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.258272 22912 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.237977 22832 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.238914 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 171) took 1689 ms. Trace:
I20260709 13:12:00.170399 24469 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.284947 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.242975 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 139) took 2681 ms. Trace:
I20260709 13:12:00.244036 22914 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:00.241113 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.286139 22762 rpcz_store.cc:276] 0709 13:11:57.561915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.292319 (+1730404us) service_pool.cc:224] Handling call
0709 13:11:59.354419 (+ 62100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.685891 (+331472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.242712 (+556821us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54002dafd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.242909 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760438528}
I20260709 13:12:00.287245 22910 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:00.158093 22759 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:00.219673 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.276666 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.278295 24697 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.279065 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.204455 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.281800 24678 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:00.284232 24580 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:00.294287 22757 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:00.243180 24610 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.311489 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.312114 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.312738 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.312758 22762 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:00.290575 22915 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.292172 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.217344 24660 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.328917 24660 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.245680 24667 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:12:00.330588 24667 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:00.330935 24667 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:00.262130 22833 rpcz_store.cc:276] 0709 13:11:58.549217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.762884 (+213667us) service_pool.cc:224] Handling call
0709 13:11:58.763320 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:58.871618 (+108298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.871651 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:58.888312 (+ 16661us) log.cc:844] Serialized 52 byte log entry
0709 13:11:59.132175 (+243863us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:59.145712 (+ 13537us) log.cc:844] Serialized 14 byte log entry
0709 13:11:59.152094 (+  6382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.200303 (+1048209us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.200378 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.200451 (+    73us) raft_consensus.cc:1685] finished
0709 13:12:00.238649 (+ 38198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.238844 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16641,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":186976,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":53693,"spinlock_wait_cycles":17082112,"thread_start_us":330516,"threads_started":2,"wal-append.queue_time_us":379987}
I20260709 13:12:00.308804 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.327463 22834 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
W20260709 13:12:00.333904 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 128434us
I20260709 13:12:00.334280 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.335464 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.327808 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.335922 24459 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:00.336516 22757 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:00.337095 24459 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:00.337632 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.337984 24459 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:00.338435 24459 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:00.338555 22757 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:00.338769 24697 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.339275 24697 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.339339 24459 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.340314 24620 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:00.340809 24620 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:00.341089 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.342582 24641 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:00.343248 24684 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:00.343165 24641 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.343709 24684 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.344157 24684 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:00.346159 24459 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.346681 24469 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.341578 24697 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.339556 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.341673 24620 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.342423 24693 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.351675 22838 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.356771 22836 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
W20260709 13:12:00.332605 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 147868us
I20260709 13:12:00.426704 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.427409 22836 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:00.428187 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:00.358898 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.426671 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:00.359639 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:00.429382 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.395782 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.363111 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.430148 22764 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:00.430260 22833 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:12:00.431005 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.331745 24667 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:00.420300 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:00.433756 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70839.8us
I20260709 13:12:00.429392 22838 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:00.431054 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.434813 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.435168 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.435580 22833 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.308017 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.396610 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.393854 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.439989 22762 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:00.431720 22764 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.432447 22912 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:00.336037 22834 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:00.425371 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.440816 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.444572 22838 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:00.348235 24641 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.448052 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.358692 24678 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.452181 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.445361 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.455343 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.463569 24440 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.465624 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.467729 22834 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:00.473122 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.478056 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.479660 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.480149 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.481052 22762 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:00.491660 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:00.495108 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.496075 22758 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.498947 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1933) took 1234 ms. Trace:
I20260709 13:12:00.499380 22913 rpcz_store.cc:276] 0709 13:11:59.264051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.267464 (+  3413us) service_pool.cc:224] Handling call
0709 13:12:00.498744 (+1231280us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54003020d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:00.498882 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850828544}
I20260709 13:12:00.483170 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.501655 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.502321 22840 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:12:00.502929 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:00.488214 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.488153 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 178) took 1194 ms. Trace:
I20260709 13:12:00.489156 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1935) took 1224 ms. Trace:
I20260709 13:12:00.486948 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:00.439145 24469 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.438297 24648 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:00.503651 22763 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:12:00.503823 22908 rpcz_store.cc:276] 0709 13:11:59.264726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.341643 (+ 76917us) service_pool.cc:224] Handling call
0709 13:12:00.488886 (+1147243us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000447d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:00.489072 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398179712}
I20260709 13:12:00.503706 22837 rpcz_store.cc:276] 0709 13:11:59.293459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.293677 (+   218us) service_pool.cc:224] Handling call
0709 13:11:59.330495 (+ 36818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.487837 (+157342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.487906 (+1000069us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.488087 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360120192}
I20260709 13:12:00.523967 22841 pending_rounds.cc:85] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 1
I20260709 13:12:00.524479 22841 pending_rounds.cc:107] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:12:00.524763 22841 raft_consensus.cc:2889] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.525189 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.526057 22763 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:00.531116 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73463.5us
I20260709 13:12:00.531628 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.532238 22763 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:12:00.540398 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:00.540491 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.532177 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.543502 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.545674 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:00.464767 24590 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.547727 24605 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:00.547871 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1000) took 6927 ms. Trace:
I20260709 13:12:00.548625 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:00.549319 24677 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:00.549798 24616 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:00.549810 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:00.458880 24086 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:00.552232 22766 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 4. Ops: []
I20260709 13:12:00.498214 24521 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:00.554077 24469 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:00.486110 24478 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:00.526127 22757 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:00.486456 24585 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:00.563169 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:00.594127 22911 raft_consensus.cc:606] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.486290 24602 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.448210 24610 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:12:00.487330 24667 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.460134 24678 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.543241 24433 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:00.554075 24664 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.842s	user 0.017s	sys 0.014s
I20260709 13:12:00.449967 24693 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:00.569707 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.556689 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.650691 22762 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.558363 24677 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.453363 24620 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:00.620976 22913 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:00.629611 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.572049 24470 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.571134 24086 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.659313 22765 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:00.637537 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.683063 23466 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.684000 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1936) took 1419 ms. Trace:
I20260709 13:12:00.684531 22913 rpcz_store.cc:276] 0709 13:11:59.265031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.593408 (+1328377us) service_pool.cc:224] Handling call
0709 13:12:00.620737 (+ 27329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.683641 (+ 62904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.683924 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.686218 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.643987 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:00.584693 24659 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.553598 24648 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.594763 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1934) took 1330 ms. Trace:
I20260709 13:12:00.710292 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.711309 22759 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.677206 23462 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:00.678527 23463 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:12:00.713948 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.679522 22914 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:00.635334 22806 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.078s (SENT)
I20260709 13:12:00.686317 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.687778 22915 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:00.716511 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.646292 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:00.716946 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1938) took 1271 ms. Trace:
I20260709 13:12:00.717309 22915 rpcz_store.cc:276] 0709 13:11:59.445188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.687255 (+1242067us) service_pool.cc:224] Handling call
0709 13:12:00.687534 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.716698 (+ 29164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.716891 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.622013 24602 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:00.720638 22763 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:00.724557 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.725829 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.621779 24585 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:00.734000 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98730us
I20260709 13:12:00.682298 24677 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.736564 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.742065 22763 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.690554 24086 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:00.687790 24716 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.746201 22758 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.693468 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:00.751142 24717 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:12:00.752115 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:00.776597 24440 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
W20260709 13:12:00.788465 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96360.3us
W20260709 13:12:00.789868 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60293.1us
I20260709 13:12:00.791216 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:00.792460 22833 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.792934 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:00.750106 24691 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.143s	user 0.005s	sys 0.000s
I20260709 13:12:00.801410 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:00.646924 24433 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.802274 22764 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.807829 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.813066 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:00.815891 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.558089 22760 rpcz_store.cc:276] 0709 13:11:53.620779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.575461 (+4954682us) service_pool.cc:224] Handling call
0709 13:11:58.659336 (+ 83875us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.383595 (+724259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.530243 (+1146648us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54002233d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:00.547784 (+ 17541us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271767424}
I20260709 13:12:00.858002 22764 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 4. Ops: []
I20260709 13:12:00.858804 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1031) took 2756 ms. Trace:
I20260709 13:12:00.859123 22764 rpcz_store.cc:276] 0709 13:11:58.102802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.857349 (+2754547us) service_pool.cc:224] Handling call
0709 13:12:00.857811 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.858508 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.858745 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.860358 22914 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:00.860805 22764 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:12:00.861219 22764 pending_rounds.cc:85] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:12:00.819890 22908 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:00.862253 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 152) took 1314 ms. Trace:
I20260709 13:12:00.862481 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:00.862630 22908 rpcz_store.cc:276] 0709 13:11:59.547860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.781870 (+1234010us) service_pool.cc:224] Handling call
0709 13:12:00.819697 (+ 37827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.861971 (+ 42274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.862192 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.820492 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:00.864058 22758 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:12:00.826865 22763 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:00.722548 22906 rpcz_store.cc:276] 0709 13:11:59.264398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.841425 (+577027us) service_pool.cc:224] Handling call
0709 13:11:59.841963 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.938497 (+ 96534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:59.940454 (+  1957us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.030903 (+ 90449us) log.cc:844] Serialized 26 byte log entry
0709 13:12:00.446228 (+415325us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:00.451204 (+  4976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.487142 (+ 35938us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.552744 (+ 65602us) raft_consensus.cc:1685] finished
0709 13:12:00.558057 (+  5313us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.575686 (+ 17629us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25128,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":10818,"thread_start_us":371496,"threads_started":2,"wal-append.queue_time_us":453770}
I20260709 13:12:00.830189 24433 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:00.831930 24576 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.757s	user 0.007s	sys 0.012s
I20260709 13:12:00.865872 24576 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.866348 22906 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:12:00.866688 22758 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:00.867055 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 154) took 1299 ms. Trace:
I20260709 13:12:00.867362 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1049) took 1256 ms. Trace:
I20260709 13:12:00.833794 22833 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:00.740832 24602 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:00.867875 22758 rpcz_store.cc:276] 0709 13:11:59.610614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.865967 (+1255353us) service_pool.cc:224] Handling call
0709 13:12:00.866323 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.867095 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.867303 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.868850 24602 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.869433 23460 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:00.837857 24585 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:00.839962 24611 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.847s	user 0.024s	sys 0.015s
I20260709 13:12:00.852258 24648 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:00.810261 23464 raft_consensus.cc:2376] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3. Re-sending same reply.
I20260709 13:12:00.753412 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:00.854377 22907 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.643s	user 0.014s	sys 0.006s
I20260709 13:12:00.827814 22757 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.861436 22764 pending_rounds.cc:107] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:12:00.904165 22764 raft_consensus.cc:2889] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.876210 22758 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:00.867478 22906 rpcz_store.cc:276] 0709 13:11:59.567928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.865647 (+1297719us) service_pool.cc:224] Handling call
0709 13:12:00.866090 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.866776 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.866996 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.905011 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 153) took 2212 ms. Trace:
I20260709 13:12:00.926190 22764 rpcz_store.cc:276] 0709 13:11:58.692171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.860117 (+2167946us) service_pool.cc:224] Handling call
0709 13:12:00.860428 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.904725 (+ 44297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.904953 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.906531 22908 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:00.927516 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1052) took 1312 ms. Trace:
I20260709 13:12:00.929656 22758 rpcz_store.cc:276] 0709 13:11:59.614674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.869183 (+1254509us) service_pool.cc:224] Handling call
0709 13:12:00.875712 (+  6529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.927098 (+ 51386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.927429 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.793792 24716 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.930967 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 153) took 1379 ms. Trace:
I20260709 13:12:00.926260 24648 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:00.932634 22908 rpcz_store.cc:276] 0709 13:11:59.551236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.879197 (+1327961us) service_pool.cc:224] Handling call
0709 13:12:00.906287 (+ 27090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.928884 (+ 22597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.930890 (+  2006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:00.889701 22907 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:00.947266 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:00.947304 22758 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:12:00.932624 24576 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:00.951220 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 398283us
W20260709 13:12:00.944654 24684 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.600s	user 0.016s	sys 0.009s
I20260709 13:12:00.933979 24659 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:00.942296 24610 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:12:00.957207 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 408643us
I20260709 13:12:00.990792 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.943697 24675 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:01.003600 23466 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 1 LEARNER]: Advancing to term 2
I20260709 13:12:01.003865 22758 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:01.004460 22758 pending_rounds.cc:85] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 2
I20260709 13:12:01.004735 22758 pending_rounds.cc:107] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:12:01.005012 22758 raft_consensus.cc:2889] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.927999 24478 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:00.882138 24611 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:00.992899 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:01.008344 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:01.010321 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.011744 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:01.012049 22913 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:01.012894 23462 raft_consensus.cc:2410] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:01.015239 22758 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:12:01.016564 22758 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:00.949935 24469 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:00.952090 24590 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.026738 24478 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:00.959686 24605 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:00.946576 24086 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.020859 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:01.021204 22758 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:01.023109 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1161) took 1083 ms. Trace:
I20260709 13:12:01.026029 22908 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:12:01.026489 24541 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.783s	user 0.014s	sys 0.009s
I20260709 13:12:01.027122 24590 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:01.028290 22762 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
W20260709 13:12:00.989609 22832 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.679s	user 0.003s	sys 0.008s
I20260709 13:12:01.018391 22880 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:00.951411 23461 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:00.987941 24610 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.006779 24675 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.944425 24648 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.013142 24576 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:12:01.002373 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:00.988343 24684 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:01.026513 24707 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.019s
I20260709 13:12:01.038910 22832 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:01.052433 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1052) took 1219 ms. Trace:
I20260709 13:12:01.052865 22832 rpcz_store.cc:276] 0709 13:11:59.833300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.216674 (+383374us) service_pool.cc:224] Handling call
0709 13:12:01.052349 (+835675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:01.054322 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.040304 23466 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:12:01.040126 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:01.012588 24693 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:01.042866 22912 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:01.030839 24718 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:01.030653 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:01.044029 24610 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.046662 24675 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:01.070940 22759 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 1. Current term is 3. Ops: []
I20260709 13:12:01.057922 24707 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:01.077069 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 284189us
I20260709 13:12:01.037549 24541 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:01.085929 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 223512us
I20260709 13:12:01.098520 24614 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:01.099460 24614 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:01.069080 24646 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.012s
I20260709 13:12:01.035068 22907 rpcz_store.cc:276] 0709 13:11:59.939609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.059545 (+119936us) service_pool.cc:224] Handling call
0709 13:12:01.022919 (+963374us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540020bcd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.023044 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312692608}
I20260709 13:12:01.061410 24725 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:12:01.078923 24660 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.002s
W20260709 13:12:01.129030 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.103623 24459 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.008s
I20260709 13:12:01.100663 24614 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.142459 22833 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:01.071341 24693 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:12:01.130092 23457 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:01.159220 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1937) took 1725 ms. Trace:
I20260709 13:12:01.102437 24727 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:01.137238 22907 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:01.185357 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.118461 24718 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:12:01.143122 24611 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:01.140508 24728 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:01.142036 24716 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:01.183434 24614 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.224133 22839 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:01.074357 24605 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.155529 24588 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:01.148283 24707 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:01.130609 24732 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:12:01.196286 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.200431 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:01.223099 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 103451us
W20260709 13:12:01.206687 24724 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:12:01.161149 22913 rpcz_store.cc:276] 0709 13:11:59.433974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.685538 (+1251564us) service_pool.cc:224] Handling call
0709 13:12:00.685860 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.012721 (+326861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.155273 (+142552us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54000ea954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:01.159125 (+  3852us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749651840}
I20260709 13:12:01.185603 24727 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.138029 24737 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:12:01.130266 24723 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
W20260709 13:12:01.245483 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45182.3us
I20260709 13:12:01.246632 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:01.248698 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:01.195454 24726 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:01.260694 22762 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:01.260164 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:01.253058 24707 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.199016 24660 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:12:01.278069 24732 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.223424 24738 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:01.274987 24733 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:12:01.237814 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1946) took 1469 ms. Trace:
W20260709 13:12:01.298925 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 258886us
I20260709 13:12:01.251590 24727 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:01.282034 24659 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:01.253414 24469 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:01.253454 24671 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:01.291615 24086 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.295802 24725 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:01.297956 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:01.310789 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:01.311865 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:01.344496 22758 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:01.303158 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:01.314422 23457 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:12:01.303704 24611 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:01.317128 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1048) took 1485 ms. Trace:
I20260709 13:12:01.317317 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1948) took 1166 ms. Trace:
W20260709 13:12:01.303807 24736 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:12:01.305474 24726 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.367798 22913 rpcz_store.cc:276] 0709 13:12:00.150740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.242321 (+1091581us) service_pool.cc:224] Handling call
0709 13:12:01.242741 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.243021 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:01.243372 (+   351us) log.cc:844] Serialized 14 byte log entry
0709 13:12:01.316889 (+ 73517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:01.316958 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:01.317018 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.317099 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.317248 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":651,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":171,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":11167,"thread_start_us":72754,"threads_started":1,"wal-append.queue_time_us":93038}
I20260709 13:12:01.318352 24728 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:01.333976 24671 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.334199 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:01.295596 24459 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:12:01.323818 24469 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.338006 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1945) took 1649 ms. Trace:
I20260709 13:12:01.324267 24648 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:01.302838 24684 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:01.253278 24716 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:12:01.343650 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 45867.8us
I20260709 13:12:01.253525 24476 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:12:01.285272 24646 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:12:01.346477 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:01.290727 24704 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:12:01.308854 24590 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:01.367820 22837 rpcz_store.cc:276] 0709 13:11:59.832088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.550014 (+717926us) service_pool.cc:224] Handling call
0709 13:12:00.550382 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.550775 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.550816 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.551411 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:12:00.887659 (+336248us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:00.887721 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.887822 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.316809 (+428987us) raft_consensus.cc:1685] finished
0709 13:12:01.316841 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.317038 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1855,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":36219,"thread_start_us":335714,"threads_started":1,"wal-append.queue_time_us":513208}
I20260709 13:12:01.373662 24754 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:12:01.374155 24739 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:12:01.375072 24734 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:12:01.375456 24729 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:12:01.376101 24667 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.000s
I20260709 13:12:01.375619 24732 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:01.433472 24726 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.434360 24678 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.435204 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 148) took 1888 ms. Trace:
I20260709 13:12:01.377779 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:01.378875 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 417) took 1266 ms. Trace:
I20260709 13:12:01.307512 24748 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:12:01.380173 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.309013 24738 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.367892 24611 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.450090 24611 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:01.452093 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.380993 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1068) took 1168 ms. Trace:
I20260709 13:12:01.453092 24704 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:01.403822 24728 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.453466 24704 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:01.453805 22832 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:01.347216 24585 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:01.383648 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1059) took 1766 ms. Trace:
I20260709 13:12:01.385711 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1046) took 1554 ms. Trace:
I20260709 13:12:01.454767 22764 rpcz_store.cc:276] 0709 13:11:59.616732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.931557 (+1314825us) service_pool.cc:224] Handling call
0709 13:12:00.932043 (+   486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:00.938082 (+  6039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.938114 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:00.945248 (+  7134us) log.cc:844] Serialized 52 byte log entry
0709 13:12:01.144016 (+198768us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:01.144065 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.144144 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.383414 (+239270us) raft_consensus.cc:1685] finished
0709 13:12:01.383436 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.383584 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":2491,"thread_start_us":130690,"threads_started":1,"wal-append.queue_time_us":334980}
I20260709 13:12:01.390336 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:01.456947 22913 raft_consensus.cc:606] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:01.342959 24725 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:01.458572 22762 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:12:01.459256 24620 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.772s	user 0.009s	sys 0.004s
I20260709 13:12:01.459373 24725 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.460584 24725 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:01.405210 22909 rpcz_store.cc:276] 0709 13:11:59.688742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.096201 (+1407459us) service_pool.cc:224] Handling call
0709 13:12:01.097021 (+   820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.104132 (+  7111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.104175 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:01.139144 (+ 34969us) log.cc:844] Serialized 548 byte log entry
0709 13:12:01.306153 (+167009us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:01.306209 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.306299 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.337735 (+ 31436us) raft_consensus.cc:1685] finished
0709 13:12:01.337758 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.337935 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":166579,"threads_started":1,"wal-append.queue_time_us":195341}
W20260709 13:12:01.463686 24677 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
I20260709 13:12:01.464134 24677 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.464478 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1949) took 1313 ms. Trace:
I20260709 13:12:01.464875 22909 rpcz_store.cc:276] 0709 13:12:00.150993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.463565 (+1312572us) service_pool.cc:224] Handling call
0709 13:12:01.463904 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.464118 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.464147 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:01.464201 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:01.464244 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.464301 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.464420 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:01.377784 24541 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:01.310133 22907 rpcz_store.cc:276] 0709 13:11:59.768535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.136567 (+1368032us) service_pool.cc:224] Handling call
0709 13:12:01.136982 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.193465 (+ 56483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.233236 (+ 39771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:01.395937 24671 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:01.413820 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67458.2us
W20260709 13:12:01.468032 22757 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.005s
I20260709 13:12:01.337888 24749 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:12:01.468420 22757 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:01.468946 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1063) took 1125 ms. Trace:
W20260709 13:12:01.409241 24722 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:12:01.407963 24469 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.470166 24675 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.453147 22833 rpcz_store.cc:276] 0709 13:12:00.212557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.868568 (+656011us) service_pool.cc:224] Handling call
0709 13:12:00.868859 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.143494 (+274635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.380745 (+237251us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54003228d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:01.380931 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628786944}
I20260709 13:12:01.474054 24727 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.418040 24086 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:01.475265 24086 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.476393 24086 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.479219 24536 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.480705 24536 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.435585 22915 rpcz_store.cc:276] 0709 13:11:59.546664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.734241 (+1187577us) service_pool.cc:224] Handling call
0709 13:12:00.734653 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.735079 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.735120 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.748867 (+ 13747us) log.cc:844] Serialized 26 byte log entry
0709 13:12:01.103487 (+354620us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:01.103534 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.103608 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.434959 (+331351us) raft_consensus.cc:1685] finished
0709 13:12:01.434980 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.435134 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77617,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":47166,"thread_start_us":275228,"threads_started":1,"wal-append.queue_time_us":449458}
I20260709 13:12:01.437222 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 151) took 1889 ms. Trace:
I20260709 13:12:01.446954 24476 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:01.481897 22911 rpcz_store.cc:276] 0709 13:11:59.547551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.750003 (+1202452us) service_pool.cc:224] Handling call
0709 13:12:00.750694 (+   691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.757859 (+  7165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.757895 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.758506 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:12:01.065137 (+306631us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:01.065220 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.065332 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.436976 (+371644us) raft_consensus.cc:1685] finished
0709 13:12:01.437001 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.437155 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6670,"raft.queue_time_us":149756,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":28811,"thread_start_us":215126,"threads_started":1,"wal-append.queue_time_us":409608}
I20260709 13:12:01.483080 24476 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:01.438663 22836 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:01.484069 24476 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.441540 23464 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:01.485164 22836 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1039) took 1709 ms (client timeout 2243 ms). Trace:
W20260709 13:12:01.485647 22836 rpcz_store.cc:269] 0709 13:11:59.775335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.427971 (+652636us) service_pool.cc:224] Handling call
0709 13:12:01.484947 (+1056976us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.485103 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2322276736}
I20260709 13:12:01.445312 23460 rpcz_store.cc:276] 0709 13:12:00.112031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.112184 (+   153us) service_pool.cc:224] Handling call
0709 13:12:00.112522 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.869877 (+757355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.378625 (+508748us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54005253d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:01.378813 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740240128}
W20260709 13:12:01.487315 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1064) took 1424 ms (client timeout 1831 ms). Trace:
W20260709 13:12:01.487751 22839 rpcz_store.cc:269] 0709 13:12:00.062821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.277370 (+214549us) service_pool.cc:224] Handling call
0709 13:12:01.487146 (+1209776us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.487256 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963851008}
W20260709 13:12:01.409698 24521 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.004s	sys 0.004s
I20260709 13:12:01.411477 24660 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:01.409432 24707 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.449185 24756 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:01.349167 24588 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:01.454864 22841 rpcz_store.cc:276] 0709 13:11:59.831511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.523199 (+691688us) service_pool.cc:224] Handling call
0709 13:12:00.523694 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:00.554596 (+ 30902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.554642 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:00.555300 (+   658us) log.cc:844] Serialized 52 byte log entry
0709 13:12:00.952518 (+397218us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:00.952567 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.952645 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.385456 (+432811us) raft_consensus.cc:1685] finished
0709 13:12:01.385482 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.385645 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":12396,"thread_start_us":350665,"threads_started":1,"wal-append.queue_time_us":553301}
W20260709 13:12:01.391428 24720 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.289s	user 0.001s	sys 0.001s
I20260709 13:12:01.475351 22910 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:01.481175 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1861) took 1485 ms. Trace:
I20260709 13:12:01.506172 22765 rpcz_store.cc:276] 0709 13:11:59.996012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.137688 (+141676us) service_pool.cc:224] Handling call
0709 13:12:00.479412 (+341724us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9cce9c 00007f689e9cca5b 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:01.480941 (+1001529us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400311354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.481093 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414342528}
I20260709 13:12:01.491544 22909 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:01.508946 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1953) took 1353 ms. Trace:
I20260709 13:12:01.509336 22909 rpcz_store.cc:276] 0709 13:12:00.156075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.465843 (+1309768us) service_pool.cc:224] Handling call
0709 13:12:01.491289 (+ 25446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.508676 (+ 17387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.508877 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:01.448127 24659 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:01.492805 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:01.416465 24478 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.528513 24478 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:01.436419 24616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.016s	sys 0.009s
I20260709 13:12:01.497457 22762 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:12:01.525264 24730 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.388s	user 0.000s	sys 0.003s
I20260709 13:12:01.502974 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1047) took 1668 ms. Trace:
I20260709 13:12:01.502761 24693 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:01.504709 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.494783 24660 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:01.509367 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:01.540482 22758 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.512714 24738 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.514755 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1042) took 1732 ms. Trace:
I20260709 13:12:01.543387 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.525350 24588 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.548765 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:01.521617 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1950) took 1370 ms. Trace:
I20260709 13:12:01.549368 22914 rpcz_store.cc:276] 0709 13:12:00.151284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.465269 (+313985us) service_pool.cc:224] Handling call
0709 13:12:01.521420 (+1056151us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54001b9ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.521556 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490267904}
I20260709 13:12:01.523141 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 155) took 1930 ms. Trace:
I20260709 13:12:01.524314 22765 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:01.550529 22908 rpcz_store.cc:276] 0709 13:11:59.592534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.938907 (+1346373us) service_pool.cc:224] Handling call
0709 13:12:01.025784 (+ 86877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.039620 (+ 13836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.522917 (+483297us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b540020bcd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.523073 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174921600}
I20260709 13:12:01.469291 22757 rpcz_store.cc:276] 0709 13:12:00.343301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.680568 (+337267us) service_pool.cc:224] Handling call
0709 13:12:01.468883 (+788315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:01.494477 24476 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.527189 24659 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.530189 22832 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:01.499862 24756 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:01.495549 24521 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.004s	sys 0.004s
W20260709 13:12:01.533620 24616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.016s	sys 0.009s
W20260709 13:12:01.542169 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 154369us
I20260709 13:12:01.516057 24728 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.520710 24675 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.534662 24412 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:01.558485 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:01.595270 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.581240 24677 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.535728 24585 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:12:01.600558 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62148.8us
W20260709 13:12:01.602398 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.609404 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1037) took 1834 ms. Trace:
I20260709 13:12:01.606348 24412 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:01.607160 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:01.610843 22806 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.148s (SENT)
I20260709 13:12:01.611611 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:01.613526 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:01.613775 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:01.614380 22914 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:01.610208 22838 rpcz_store.cc:276] 0709 13:11:59.774703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.512849 (+738146us) service_pool.cc:224] Handling call
0709 13:12:00.513583 (+   734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.514099 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.514128 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.575584 (+ 61456us) log.cc:844] Serialized 26 byte log entry
0709 13:12:01.056139 (+480555us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:01.233744 (+177605us) log.cc:844] Serialized 14 byte log entry
0709 13:12:01.234205 (+   461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.234369 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.524558 (+290189us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:01.601006 (+ 76448us) raft_consensus.cc:1685] finished
0709 13:12:01.608566 (+  7560us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.608969 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5911,"prepare.queue_time_us":282533,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":14322,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":29799,"spinlock_wait_cycles":13010688,"thread_start_us":555035,"threads_started":2,"wal-append.queue_time_us":529168}
I20260709 13:12:01.616056 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.544505 24693 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.618379 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.619335 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:01.608289 24610 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.006s
I20260709 13:12:01.620332 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:01.600073 24521 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.600317 24616 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:01.607962 24611 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.601950 24660 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:01.608040 24756 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.553557 22758 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:01.557106 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:01.544972 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:01.595558 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:01.598331 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 113) took 6827 ms. Trace:
W20260709 13:12:01.558073 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1857) took 1557 ms (client timeout 1965 ms). Trace:
I20260709 13:12:01.951555 22909 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:01.623656 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:12:01.958395 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1159) took 2019 ms. Trace:
W20260709 13:12:01.957888 22761 rpcz_store.cc:269] 0709 13:11:59.994943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.099390 (+104447us) service_pool.cc:224] Handling call
0709 13:12:01.542821 (+1443431us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b540035e654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.552071 (+  9250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16026880}
I20260709 13:12:01.959118 22912 rpcz_store.cc:276] 0709 13:11:59.939045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.471902 (+532857us) service_pool.cc:224] Handling call
0709 13:12:01.957834 (+1485932us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540010f4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.958098 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308029696}
I20260709 13:12:01.952409 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1154) took 2016 ms. Trace:
I20260709 13:12:01.960235 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:01.896373 22837 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:01.633273 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.639349 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:01.964359 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.967240 24433 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.968518 24433 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.973002 24433 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:01.974793 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.929384 22908 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:01.898548 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:01.978658 22836 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:01.613611 24412 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:01.929226 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:01.900651 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:01.981024 22766 rpcz_store.cc:276] 0709 13:11:54.765901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.628256 (+3862355us) service_pool.cc:224] Handling call
0709 13:11:59.035373 (+407117us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5400223228. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a95ebc 00007f68a6acb08c 00007f68a6ab593b 00007f68a6ab53d2 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c
0709 13:12:00.552038 (+1516665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.571262 (+ 19224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.592754 (+1021492us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002233d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.592928 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3492317312}
I20260709 13:12:01.539388 22835 rpcz_store.cc:276] 0709 13:11:59.831821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.548656 (+716835us) service_pool.cc:224] Handling call
0709 13:12:00.549144 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.549423 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.549458 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.550068 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:12:00.917814 (+367746us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:00.926888 (+  9074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.933064 (+  6176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.470765 (+537701us) raft_consensus.cc:1685] finished
0709 13:12:01.470793 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.500446 (+ 29653us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":331603,"threads_started":1,"wal-append.queue_time_us":519665}
I20260709 13:12:02.027897 22835 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:01.960995 22910 rpcz_store.cc:276] 0709 13:11:59.936158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.188326 (+1252168us) service_pool.cc:224] Handling call
0709 13:12:01.188819 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.505891 (+317072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.951674 (+445783us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54001257d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:01.952092 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658452352}
W20260709 13:12:01.865257 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 248297us
I20260709 13:12:01.985160 22765 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:01.986104 24440 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:01.987385 24459 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.688844 24469 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:01.866250 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.683097 24478 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:01.976140 24433 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.878949 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:01.988801 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:01.976058 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 214) took 1328 ms. Trace:
I20260709 13:12:01.844233 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:01.989094 24521 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:01.951570 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.614292 24536 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:12:01.994350 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.817117 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 268889us
W20260709 13:12:01.874208 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.057765 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.641247 24610 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:02.059051 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.059760 22762 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:02.060568 24610 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:01.839452 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.061964 24541 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:02.062821 24541 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.067584 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.068660 22760 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:01.817739 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.071784 24590 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:02.072139 22757 raft_consensus.cc:2376] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4. Re-sending same reply.
I20260709 13:12:01.697199 24671 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.074133 22763 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:02.073945 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.075006 22757 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:02.075138 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:02.075847 22763 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.076107 24459 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:01.675760 24718 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.010146 24646 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:02.011349 24707 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.012773 24605 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.010239 24719 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.640076 24668 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:01.686662 24693 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:02.015445 22837 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:01.613561 24620 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:01.613534 24715 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.815385 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:02.017436 24704 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.022619 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.017063 22912 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:02.022989 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:01.543468 22834 rpcz_store.cc:276] 0709 13:11:59.782516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.468640 (+686124us) service_pool.cc:224] Handling call
0709 13:12:00.469109 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.471363 (+  2254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.471405 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.472506 (+  1101us) log.cc:844] Serialized 26 byte log entry
0709 13:12:00.890201 (+417695us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:00.901289 (+ 11088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.930163 (+ 28874us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:01.514518 (+584355us) raft_consensus.cc:1685] finished
0709 13:12:01.514548 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.514690 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1608,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"thread_start_us":402298,"threads_started":1,"wal-append.queue_time_us":571243}
W20260709 13:12:02.028625 24745 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.719s	user 0.000s	sys 0.005s
I20260709 13:12:02.017268 22764 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:02.028997 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.029181 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:02.093391 22833 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:02.017043 24660 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:01.834896 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1032) took 3731 ms. Trace:
I20260709 13:12:02.017117 22913 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:01.877290 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.023130 24611 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:02.040208 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:02.102981 22910 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:02.039638 23461 rpcz_store.cc:276] 0709 13:12:00.648502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.660392 (+ 11890us) service_pool.cc:224] Handling call
0709 13:12:01.975550 (+1315158us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b54006707d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:01.975791 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137161088}
I20260709 13:12:02.048802 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:02.050853 24648 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:02.053941 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:02.055023 24659 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:02.061007 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:02.072450 24590 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.076365 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:01.870445 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 255352us
I20260709 13:12:01.678540 24588 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.116402 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.117624 22764 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:02.118952 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.119482 22766 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:02.119817 22764 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:02.078644 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.121802 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.121958 22757 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:02.122741 22762 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.123759 22914 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:02.105031 24459 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.124874 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.126334 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:02.126363 22806 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.172s (SENT)
I20260709 13:12:02.127411 24620 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.095881 22761 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.129837 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.130791 22765 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:02.131565 24541 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.132238 22765 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:02.133785 22832 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:02.133787 24731 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.901s	user 0.003s	sys 0.001s
I20260709 13:12:02.097635 24470 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.134883 24727 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:02.135263 24470 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.135928 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1043) took 2353 ms. Trace:
I20260709 13:12:02.097818 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.137075 22906 raft_consensus.cc:2368] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3.
I20260709 13:12:02.136303 22840 rpcz_store.cc:276] 0709 13:11:59.782758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.502916 (+720158us) service_pool.cc:224] Handling call
0709 13:12:00.503249 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.576411 (+ 73162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.576461 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.618387 (+ 41926us) log.cc:844] Serialized 26 byte log entry
0709 13:12:01.040966 (+422579us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:01.041022 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.896714 (+855692us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b54000c71d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:01.896838 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.132577 (+235739us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:02.135461 (+  2884us) raft_consensus.cc:1685] finished
0709 13:12:02.135495 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.135731 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1397,"raft.queue_time_us":2508,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":11601,"spinlock_wait_cycles":173052032,"thread_start_us":347941,"threads_started":1,"wal-append.queue_time_us":491908}
I20260709 13:12:02.098640 22660 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 240) took 6413 ms. Trace:
I20260709 13:12:02.099116 24707 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.107077 24704 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:02.107633 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58951.1us
I20260709 13:12:02.019553 24616 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.115561 22758 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:02.094549 22759 rpcz_store.cc:276] 0709 13:11:58.103070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.832515 (+2729445us) service_pool.cc:224] Handling call
0709 13:12:00.832895 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.074907 (+242012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.834094 (+759187us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400340dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:01.834622 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546502016}
I20260709 13:12:02.129050 22766 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:12:02.108238 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:02.138805 22906 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:02.150344 22836 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.155457 22914 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.095511 24433 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.159317 24693 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:02.159965 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:02.154798 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1125) took 1009 ms. Trace:
I20260709 13:12:02.160676 22759 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:12:02.161103 24684 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.161435 22912 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:02.159999 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:02.161917 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.092852 24605 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.162899 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:02.163751 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:02.164342 24762 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:02.102469 24719 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.168563 22836 rpcz_store.cc:276] 0709 13:12:01.145367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.035479 (+890112us) service_pool.cc:224] Handling call
0709 13:12:02.093906 (+ 58427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.154525 (+ 60619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.154738 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:02.162793 22835 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.170460 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.177753 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:02.178256 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:02.077620 24718 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.182062 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.187990 22909 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:02.187856 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73113.6us
I20260709 13:12:02.178755 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.196590 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.196828 24684 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.197750 22760 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:02.198179 22912 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:02.199168 22833 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.197804 22836 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:02.199577 24684 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:02.206979 22759 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:02.207731 24718 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.208626 22840 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:02.209908 22834 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:02.212829 22835 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.213356 24476 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.214526 24620 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:02.221658 24671 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:02.230224 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 415466us
I20260709 13:12:02.112555 24440 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.231359 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:02.424788 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89873.8us
I20260709 13:12:02.427692 24756 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.430980 22907 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:02.424719 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1071) took 1144 ms. Trace:
W20260709 13:12:02.435632 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 275733us
I20260709 13:12:02.445751 22762 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:02.258648 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 63053.6us
I20260709 13:12:02.432359 22839 rpcz_store.cc:276] 0709 13:12:01.279924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.086614 (+806690us) service_pool.cc:224] Handling call
0709 13:12:02.100330 (+ 13716us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.100931 (+   601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.100962 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.101772 (+   810us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.150403 (+ 48631us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:02.150449 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.150524 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.424433 (+273909us) raft_consensus.cc:1685] finished
0709 13:12:02.424457 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.424632 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2352,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":255,"thread_start_us":48109,"threads_started":1,"wal-append.queue_time_us":224395}
I20260709 13:12:02.232064 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:02.238548 24664 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.231768 24761 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:02.400681 24412 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.238438 24541 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:02.303719 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.424906 24470 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.460876 24476 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.463543 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:02.465969 24585 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:02.466639 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:02.466799 22760 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:02.468103 24693 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.502950 22760 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:02.503585 24693 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.468103 24433 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.468297 24664 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.508388 24664 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:02.239233 24469 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.424288 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:02.408192 22761 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:02.482052 24541 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:02.340478 24611 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.152s
I20260709 13:12:02.497116 22914 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:02.501703 23457 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
W20260709 13:12:02.502589 24774 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:12:02.257872 24762 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:02.520591 22762 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:02.421566 24715 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:02.528630 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 66026us
I20260709 13:12:02.511592 24470 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:02.534826 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1895) took 1009 ms. Trace:
W20260709 13:12:02.543124 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.536993 22762 rpcz_store.cc:276] 0709 13:12:01.519087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.472626 (+953539us) service_pool.cc:224] Handling call
0709 13:12:02.520257 (+ 47631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.527913 (+  7656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.528201 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:02.548429 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.552392 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.553350 22761 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:02.554600 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:02.556564 22761 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:12:02.557627 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:02.557900 24707 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:02.558838 24605 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:02.559988 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.560634 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.561285 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:02.561661 24469 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.562458 24646 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:02.562219 24469 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.543195 22760 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:02.563673 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1897) took 1044 ms. Trace:
I20260709 13:12:02.564368 22661 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:02.562853 24646 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.569198 24664 raft_consensus.cc:1064] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
W20260709 13:12:02.569761 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.602433 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:02.602846 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.514398 24648 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:12:02.570050 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 171) took 1061 ms. Trace:
I20260709 13:12:02.571872 22835 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.574164 24536 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:02.574947 24748 raft_consensus.cc:1064] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:12:02.578799 22761 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:12:02.578277 24762 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.581699 24719 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.581857 24723 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:02.582758 24716 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.585644 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.585693 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 168) took 1077 ms. Trace:
I20260709 13:12:02.584652 24469 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.587307 24590 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.639007 24671 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 13:12:02.639864 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.588553 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.604092 22766 rpcz_store.cc:276] 0709 13:12:01.508981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.162148 (+653167us) service_pool.cc:224] Handling call
0709 13:12:02.162681 (+   533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.163258 (+   577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.163293 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.163956 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.442147 (+278191us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:02.442184 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.442252 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.569796 (+127544us) raft_consensus.cc:1685] finished
0709 13:12:02.569822 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.569973 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1698,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":279,"thread_start_us":140887,"threads_started":1,"wal-append.queue_time_us":371500}
W20260709 13:12:02.618770 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.139582 22660 rpcz_store.cc:276] 0709 13:11:55.786470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.795187 (+  8717us) service_pool.cc:224] Handling call
0709 13:12:02.098360 (+6303173us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:59.721323 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:59.726346 (+  5023us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:59.726385 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:11:59.726403 (+    18us) tablet.cc:662] Decoding operations
0709 13:11:59.905944 (+179541us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:59.906044 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:59.906074 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:11:59.920534 (+ 14460us) tablet.cc:701] Row locks acquired
0709 13:11:59.920557 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:11:59.920735 (+   178us) write_op.cc:270] Start()
0709 13:11:59.920872 (+   137us) write_op.cc:276] Timestamp: P: 1783602719920674 usec, L: 0
0709 13:11:59.920893 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:11:59.921699 (+   806us) log.cc:844] Serialized 3694 byte log entry
0709 13:12:00.731293 (+809594us) op_driver.cc:464] REPLICATION: finished
0709 13:12:01.339261 (+607968us) write_op.cc:301] APPLY: starting
0709 13:12:01.339328 (+    67us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:01.339484 (+   156us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:01.339578 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:01.340537 (+   959us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:01.340710 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:12:01.340739 (+    29us) write_op.cc:312] APPLY: finished
0709 13:12:01.340944 (+   205us) log.cc:844] Serialized 71 byte log entry
0709 13:12:01.341126 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:01.341271 (+   145us) write_op.cc:454] Released schema lock
0709 13:12:01.341318 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:01.386179 (+ 44861us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:01.396183 (+ 10004us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1291325,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":607691,"apply.run_cpu_time_us":2505,"apply.run_wall_time_us":7227,"num_ops":9,"prepare.queue_time_us":94770,"prepare.run_cpu_time_us":16636,"prepare.run_wall_time_us":222642,"raft.queue_time_us":440961,"raft.run_cpu_time_us":29250,"raft.run_wall_time_us":427561,"replication_time_us":766142,"thread_start_us":723268,"threads_started":4,"wal-append.queue_time_us":79875,"wal-append.run_cpu_time_us":111533,"wal-append.run_wall_time_us":2342260}]]}
I20260709 13:12:02.601773 24646 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.627854 22761 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.564059 22760 rpcz_store.cc:276] 0709 13:12:01.519559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.504978 (+985419us) service_pool.cc:224] Handling call
0709 13:12:02.513878 (+  8900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.563298 (+ 49420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.563581 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:02.642122 22914 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:02.642203 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.611572 24536 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:02.607633 24605 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:02.619541 24664 raft_consensus.cc:1068] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Unable to promote non-voter 74c0a02901bc45c891e42d0d1bbbf6a2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:02.647217 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:02.657752 22761 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:12:02.659704 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 492831us
I20260709 13:12:02.659987 22913 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:02.629033 22765 rpcz_store.cc:276] 0709 13:12:01.508065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.133240 (+625175us) service_pool.cc:224] Handling call
0709 13:12:02.155457 (+ 22217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.198481 (+ 43024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.198518 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.199135 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.441919 (+242784us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:02.441970 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.560768 (+118798us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:02.560817 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.585297 (+ 24480us) raft_consensus.cc:1685] finished
0709 13:12:02.585326 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.585608 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1513,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":12183,"spinlock_wait_cycles":97583232,"thread_start_us":195227,"threads_started":1,"wal-append.queue_time_us":345360}
I20260709 13:12:02.658506 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:02.682644 22661 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.683825 24590 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:02.685082 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.685775 24541 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.689807 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.693954 24536 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.695122 24536 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.697821 24718 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:02.698468 24718 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.698976 24718 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.699344 24605 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.704452 24726 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:02.708561 22908 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.650352 24693 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.641165 24777 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:02.715003 22833 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.000s
I20260709 13:12:02.715590 22833 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:02.687242 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.722281 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:02.698474 24660 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
I20260709 13:12:02.681028 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:02.726800 22839 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.737782 22835 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:02.685165 22661 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 2, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.739547 22835 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.641243 24716 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.744325 24716 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.745802 24715 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:02.758275 22763 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.682s	user 0.013s	sys 0.002s
I20260709 13:12:02.758852 22763 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:02.753238 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:02.781657 24773 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:02.782516 24773 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.785168 24773 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.786561 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:02.754642 24412 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:02.823603 22837 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:02.829855 22760 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:02.831306 24660 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:12:02.831979 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 155399us
W20260709 13:12:02.834491 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.834438 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:02.836514 24715 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:02.838227 22838 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:02.841286 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 114648us
I20260709 13:12:02.845022 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:02.846210 24620 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.846855 24620 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.847159 24588 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:02.841365 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:02.850132 23457 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.850358 24620 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.841852 24784 raft_consensus.cc:2764] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:12:02.856645 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:02.857128 23458 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:12:02.857757 23458 pending_rounds.cc:85] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 5
I20260709 13:12:02.858043 24784 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:02.858834 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:02.858047 23458 pending_rounds.cc:107] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:12:02.860184 23458 raft_consensus.cc:2889] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:02.861877 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:02.863696 24784 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:02.864571 24784 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:02.779168 24476 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.867831 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1114) took 1777 ms. Trace:
I20260709 13:12:02.868033 24784 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.868335 22841 rpcz_store.cc:276] 0709 13:12:01.090470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.904264 (+813794us) service_pool.cc:224] Handling call
0709 13:12:01.904867 (+   603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.087150 (+182283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.087185 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.140138 (+ 52953us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.525071 (+384933us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:02.525114 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.577076 (+ 51962us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54000393d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:02.577126 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.867257 (+290131us) raft_consensus.cc:1685] finished
0709 13:12:02.867282 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.867541 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25686,"raft.queue_time_us":3700,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":24674,"spinlock_wait_cycles":396352896,"thread_start_us":335710,"threads_started":1,"wal-append.queue_time_us":627392}
I20260709 13:12:02.874907 24693 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.875808 24693 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.876087 24777 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:02.877934 22832 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:02.879989 24693 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.883713 22910 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:02.884986 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:02.886420 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:02.887220 22909 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:02.887830 23465 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:12:02.901774 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 342801us
I20260709 13:12:02.792100 24605 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:02.934718 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.019s	sys 0.031s
W20260709 13:12:02.935353 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.019s	sys 0.031s
W20260709 13:12:02.935688 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 503968us
W20260709 13:12:02.957746 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.968492 24476 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:02.969475 24476 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:02.973644 24476 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:02.841912 22661 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 3, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 3 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.901751 24588 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:02.958400 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2028764000000365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 13:12:02.998358 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1677764000000934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 13:12:03.013110 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.021611 22839 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:03.029793 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.033403 22841 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:12:03.036055 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 174341us
I20260709 13:12:03.047080 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:03.034176 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:02.992035 24777 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:03.024849 24459 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.008996 24671 consensus_queue.cc:1243] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:12:03.072888 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.056608 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:03.056095 24704 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.902s	user 0.010s	sys 0.004s
I20260709 13:12:03.083508 22911 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:03.088752 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54718.8us
I20260709 13:12:03.053617 23457 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:03.065481 24659 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.096343 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:03.058786 24684 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.068780 24664 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:03.113660 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:03.071899 22661 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 2, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.128015 24727 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.122718 24659 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:03.145593 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 397157us
I20260709 13:12:03.147826 24787 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:03.128935 24727 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.156220 24746 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:03.156915 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:03.070278 24476 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.184106 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:03.093972 24521 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:03.066390 24588 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:03.193145 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:03.122706 24773 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:03.090276 24459 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.194777 24693 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:03.200505 24726 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:03.201100 24521 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:03.259198 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 175004us
W20260709 13:12:03.270568 24786 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 13:12:03.272670 24792 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:03.272568 24459 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:03.274621 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.217116 24787 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:03.257166 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:03.210757 24773 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.209333 24746 consensus_queue.cc:1050] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:12:03.209033 24684 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.292261 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:03.217515 24790 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.217617 24616 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:12:03.290864 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:03.285007 23458 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:12:03.258366 24726 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:03.276746 24727 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:03.252489 24476 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.293287 24773 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.357359 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:03.338786 23458 pending_rounds.cc:85] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 4
I20260709 13:12:03.352339 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1076) took 1400 ms. Trace:
I20260709 13:12:03.280179 24664 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:03.343753 22661 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.339516 24784 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:03.373939 24784 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:03.374497 24784 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.365206 22762 rpcz_store.cc:276] 0709 13:12:01.952446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.544705 (+592259us) service_pool.cc:224] Handling call
0709 13:12:02.619257 (+ 74552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.630476 (+ 11219us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:12:03.033905 (+403429us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.351775 (+317870us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:03.351862 (+    87us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:03.351939 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.352047 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.352252 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":578690,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":13907,"thread_start_us":596638,"threads_started":2}
I20260709 13:12:03.364720 23458 pending_rounds.cc:107] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:12:03.340456 24809 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:12:03.395504 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 145440us
I20260709 13:12:03.343277 24790 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:03.470212 24793 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:12:03.440812 24787 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:03.561686 24784 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.562002 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 191) took 1229 ms. Trace:
I20260709 13:12:03.561524 24787 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.562758 22840 rpcz_store.cc:276] 0709 13:12:02.332654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.451863 (+119209us) service_pool.cc:224] Handling call
0709 13:12:02.478277 (+ 26414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.541450 (+ 63173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.541481 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.559783 (+ 18302us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.907395 (+347612us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:02.907471 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.907583 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.561615 (+654032us) raft_consensus.cc:1685] finished
0709 13:12:03.561637 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.561925 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29476,"raft.queue_time_us":40459,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":4749,"thread_start_us":347163,"threads_started":1,"wal-append.queue_time_us":526233}
I20260709 13:12:03.564188 24784 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:03.440462 24693 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:03.456903 24808 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:03.565071 24784 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:03.490887 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.888s (SENT) [suppressed 38 similar messages]
W20260709 13:12:03.484912 23465 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.001s
W20260709 13:12:03.503079 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 253817us
W20260709 13:12:03.405887 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 150117us
W20260709 13:12:03.574308 22908 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.848s	user 0.011s	sys 0.003s
I20260709 13:12:03.574950 22908 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:03.462415 24797 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:12:03.460803 24718 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.004s
W20260709 13:12:03.458200 24801 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:12:03.584977 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:03.454226 24794 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.335s	user 0.002s	sys 0.000s
W20260709 13:12:03.468626 24646 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.800s	user 0.007s	sys 0.006s
W20260709 13:12:03.467522 22834 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.990s	user 0.010s	sys 0.003s
W20260709 13:12:03.598577 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.500188 24716 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:03.621394 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:03.443706 24707 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:03.583223 24693 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.518349 24476 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:03.406209 23458 raft_consensus.cc:2889] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:03.462581 24684 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:03.565464 24808 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:03.432781 24809 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.463183 24588 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:03.585800 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:03.478555 24814 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
I20260709 13:12:03.640694 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1958) took 2024 ms. Trace:
I20260709 13:12:03.620518 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 167) took 2078 ms. Trace:
W20260709 13:12:03.666520 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.455797 24478 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.292s
I20260709 13:12:03.456281 24664 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:03.684891 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43538.1us
I20260709 13:12:03.628288 22834 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:12:03.928859 22834 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1077) took 1768 ms (client timeout 1974 ms). Trace:
W20260709 13:12:03.691944 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 350697us
W20260709 13:12:03.929417 22834 rpcz_store.cc:269] 0709 13:12:02.160129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.162471 (+  2342us) service_pool.cc:224] Handling call
0709 13:12:03.928763 (+1766292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:03.930114 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:03.694846 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73562.6us
I20260709 13:12:03.930821 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.931907 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:03.686256 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 202508us
I20260709 13:12:03.698648 22757 rpcz_store.cc:276] 0709 13:12:01.507764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.122748 (+614984us) service_pool.cc:224] Handling call
0709 13:12:02.556670 (+433922us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:02.556899 (+   229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.572543 (+ 15644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.572581 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.573349 (+   768us) log.cc:844] Serialized 26 byte log entry
0709 13:12:03.037876 (+464527us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:03.337149 (+299273us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.371555 (+ 34406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.386435 (+ 14880us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.568352 (+181917us) raft_consensus.cc:1685] finished
0709 13:12:03.568528 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.586406 (+ 17878us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28879,"prepare.queue_time_us":353243,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":35825,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"spinlock_wait_cycles":58549888,"thread_start_us":658035,"threads_started":2,"wal-append.queue_time_us":992348}
I20260709 13:12:03.849545 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 170) took 2340 ms. Trace:
I20260709 13:12:03.622203 24718 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.932411 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.942416 22834 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:03.878587 24648 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:03.883612 24541 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:03.698370 24707 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:03.877355 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.950047 23459 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:12:03.949867 24648 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:03.950578 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.951140 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.950685 24707 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.952378 24648 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:03.954681 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:03.955417 22834 raft_consensus.cc:2376] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4. Re-sending same reply.
I20260709 13:12:03.766021 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1091) took 1717 ms. Trace:
I20260709 13:12:03.957614 22765 rpcz_store.cc:276] 0709 13:12:02.048619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.667305 (+618686us) service_pool.cc:224] Handling call
0709 13:12:02.706387 (+ 39082us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:02.706974 (+   587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.707003 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:02.707820 (+   817us) log.cc:844] Serialized 78 byte log entry
0709 13:12:03.053774 (+345954us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:03.249683 (+195909us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.250010 (+   327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.250115 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.765607 (+515492us) raft_consensus.cc:1685] finished
0709 13:12:03.765636 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.765921 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27263,"prepare.queue_time_us":389712,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":1115,"raft.queue_time_us":11143,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":39606,"thread_start_us":485337,"threads_started":2,"wal-append.queue_time_us":916997}
I20260709 13:12:03.959260 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1965) took 1359 ms. Trace:
I20260709 13:12:03.959494 24648 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.960081 24718 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:03.960047 24648 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.962598 24648 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:03.963361 24648 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:03.964200 24671 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:03.964571 24671 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:03.965598 24671 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.966971 24671 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:03.966841 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:03.881476 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1072) took 2601 ms (client timeout 2067 ms). Trace:
I20260709 13:12:03.880362 24761 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:03.969031 22838 rpcz_store.cc:269] 0709 13:12:01.280303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.615741 (+335438us) service_pool.cc:224] Handling call
0709 13:12:03.881201 (+2265460us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540028a0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:03.881380 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2702479104}
I20260709 13:12:03.972724 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:03.761955 24821 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:12:03.553248 24748 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.769699 24835 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:03.779542 24440 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:03.694180 22911 rpcz_store.cc:276] 0709 13:12:01.611607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.611808 (+   201us) service_pool.cc:224] Handling call
0709 13:12:01.612077 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:03.087159 (+1475082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.618108 (+530949us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400041354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:03.635787 (+ 17679us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3317280128}
W20260709 13:12:03.879781 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 757320us
I20260709 13:12:03.628662 24784 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.471802 24521 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.775579 24834 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.641321 24790 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:03.628449 24646 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.861810 24659 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:03.879873 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.867343 24773 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.873353 24762 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:03.703890 24723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.024s	sys 0.031s
I20260709 13:12:03.758906 24833 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:03.880985 24756 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:12:03.923702 24802 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.674s	user 0.000s	sys 0.002s
I20260709 13:12:03.924898 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.991850 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:03.996044 22839 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:03.927604 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 190) took 1673 ms. Trace:
I20260709 13:12:03.927253 24808 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:03.846697 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ad66b5c9842437d9320507b92b78fd5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:03.881906 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.758857 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:03.997105 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 215647us
I20260709 13:12:03.883460 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.826522 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:03.651762 24693 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.882679 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.690053 24809 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:03.767125 24822 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:12:03.858573 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:03.994738 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:03.636047 24716 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:03.461284 24726 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.928783 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 166) took 2422 ms. Trace:
I20260709 13:12:03.930737 24832 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:03.698939 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 426) took 1041 ms. Trace:
I20260709 13:12:03.934922 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:03.959713 22908 rpcz_store.cc:276] 0709 13:12:02.599872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.600120 (+   248us) service_pool.cc:224] Handling call
0709 13:12:03.959064 (+1358944us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b54000438d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:03.959184 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98106752}
W20260709 13:12:03.866995 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.024351 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.024748 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.970639 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:03.973702 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43648.2us
I20260709 13:12:03.933398 22758 rpcz_store.cc:276] 0709 13:12:01.508678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.144210 (+635532us) service_pool.cc:224] Handling call
0709 13:12:02.189547 (+ 45337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.437570 (+248023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.437617 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.452496 (+ 14879us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.568267 (+115771us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b180014d880. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:12:02.773907 (+205640us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:02.773969 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.774081 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.841550 (+1067469us) raft_consensus.cc:1685] finished
0709 13:12:03.841573 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.849424 (+  7851us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178299,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":12983,"spinlock_wait_cycles":33450240,"thread_start_us":384793,"threads_started":2,"wal-append.queue_time_us":566418}
I20260709 13:12:04.023464 23465 rpcz_store.cc:276] 0709 13:12:02.658366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.682658 (+ 24292us) service_pool.cc:224] Handling call
0709 13:12:02.886744 (+204086us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:02.887019 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.887402 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:03.570908 (+683506us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.692064 (+121156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:03.692148 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:03.698575 (+  6427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.698672 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.698862 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2491,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":39175,"spinlock_wait_cycles":61771520,"thread_start_us":63488,"threads_started":1,"wal-append.queue_time_us":149076}
I20260709 13:12:03.976152 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:03.946765 24478 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:03.986510 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.026867 22910 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:04.028209 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:03.989960 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59958.6us
I20260709 13:12:03.992753 24684 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:12:03.997455 22840 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 2.
I20260709 13:12:04.002326 23462 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:03.998354 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:04.003890 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:03.998003 24843 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:04.021891 23461 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:03.991550 24723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.332s	user 0.025s	sys 0.059s
I20260709 13:12:04.030092 24833 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:04.010743 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:03.974769 24761 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:03.765904 24412 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:03.996731 22836 rpcz_store.cc:276] 0709 13:12:02.254840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.286242 (+ 31402us) service_pool.cc:224] Handling call
0709 13:12:02.341482 (+ 55240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.493962 (+152480us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:02.901002 (+407040us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.862191 (+961189us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b1800095740. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:03.862445 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:03.862556 (+   111us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:03.862628 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.927326 (+ 64698us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:03.927383 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.927538 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68515,"prepare.queue_time_us":558366,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2346,"raft.queue_time_us":791431,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5490,"spinlock_wait_cycles":383051392,"thread_start_us":520802,"threads_started":2,"wal-append.queue_time_us":664763}
I20260709 13:12:04.012553 24844 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:04.018440 22839 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:04.003623 24756 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:04.002561 24762 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.019717 22837 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:03.997575 24521 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:04.031090 22841 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:03.994472 24835 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.030303 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.043702 22909 raft_consensus.cc:606] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
W20260709 13:12:04.033938 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63348.1us
W20260709 13:12:04.037813 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104282us
I20260709 13:12:04.056432 22909 raft_consensus.cc:614] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:04.057770 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:04.058185 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.026170 22835 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:04.041946 24469 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:12:04.060725 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64105us
I20260709 13:12:04.020951 22764 rpcz_store.cc:276] 0709 13:12:01.506553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.120664 (+614111us) service_pool.cc:224] Handling call
0709 13:12:02.374444 (+253780us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:02.408890 (+ 34446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.514529 (+105639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.514589 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.538405 (+ 23816us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.957878 (+419473us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:03.154556 (+196678us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.172748 (+ 18192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.928441 (+755693us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540033fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:03.928493 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.928544 (+    51us) raft_consensus.cc:1685] finished
0709 13:12:03.928559 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.928714 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17869,"prepare.queue_time_us":301951,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":29870,"raft.run_cpu_time_us":482,"raft.run_wall_time_us":27007,"spinlock_wait_cycles":280744576,"thread_start_us":507901,"threads_started":2,"wal-append.queue_time_us":681180}
I20260709 13:12:04.032588 24833 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:04.070525 22911 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:04.032685 24843 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:04.033022 24412 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:04.078894 24412 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.079457 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.080046 24762 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.065389 22835 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:04.035786 24668 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.082077 22835 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.082682 24668 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.083292 22835 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:04.083895 24841 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:12:04.085111 24668 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.007790 24834 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.061728 22834 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:04.078262 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.042229 24800 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:04.086086 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:04.087703 24761 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.109810 24761 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:04.109951 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:04.110383 24761 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.113652 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.114142 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:04.114755 22762 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:04.114804 24762 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:04.115182 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.074540 22914 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:04.117836 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:04.119697 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:04.120170 23466 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:04.120401 22834 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:04.121925 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.072088 24659 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.055642 24648 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:04.090445 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:04.090317 24843 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.091459 24716 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.041602 24777 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:12:04.036008 24704 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.085820 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:04.180034 23462 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:04.099725 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:04.041568 24829 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:04.036949 24844 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:04.042955 24831 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:04.067243 24478 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.105365 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:04.062770 24476 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:04.106897 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:04.109086 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:04.121770 24611 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:04.087807 24693 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:04.149358 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68579.1us
I20260709 13:12:04.151280 22732 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:04.153441 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:04.140707 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:04.164909 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1894) took 2646 ms. Trace:
I20260709 13:12:04.092356 24835 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.166746 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
W20260709 13:12:04.167681 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 69111.5us
I20260709 13:12:04.171016 22762 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:04.121390 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:04.180873 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:04.123843 24790 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:04.191248 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:04.191676 22765 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:04.150906 24648 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.193431 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:04.194011 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 94490.7us
I20260709 13:12:04.103854 24800 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:04.200130 22759 rpcz_store.cc:276] 0709 13:12:01.518802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.458107 (+939305us) service_pool.cc:224] Handling call
0709 13:12:02.516668 (+ 58561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:02.570215 (+ 53547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:02.570255 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:02.571016 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.985286 (+414270us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:03.359894 (+374608us) log.cc:844] Serialized 14 byte log entry
0709 13:12:03.644943 (+285049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.954536 (+309593us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b18002f0140. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:04.164541 (+210005us) spinlock_profiling.cc:243] Waited 47.8 ms on lock 0x7b540036f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:04.164608 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:04.164656 (+    48us) raft_consensus.cc:1685] finished
0709 13:12:04.164678 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:04.164835 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57861,"prepare.queue_time_us":358092,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":211848,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":283,"spinlock_wait_cycles":172148224,"thread_start_us":976605,"threads_started":3,"wal-append.queue_time_us":827162}
I20260709 13:12:04.165820 24684 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:12:04.150612 24659 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.186147 24773 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.212404 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:04.213816 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62687.4us
W20260709 13:12:04.205531 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 98715.4us
I20260709 13:12:04.210738 24660 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:04.221101 24660 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:04.221186 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:04.221702 24660 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:04.201637 24611 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:04.202276 24829 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:04.219501 22758 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:04.204685 24664 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:04.226163 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 117172us
I20260709 13:12:04.210572 24469 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:04.212697 22834 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:04.212673 24648 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:04.186772 24704 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.213975 24831 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.215101 24800 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:04.212250 24585 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:04.239405 24842 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.172s	user 0.000s	sys 0.002s
I20260709 13:12:04.218991 24693 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:04.240515 24834 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.240522 24704 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.240880 22732 leader_election.cc:400] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3.
I20260709 13:12:04.240984 24815 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:04.232002 24664 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.241618 24799 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:04.244897 22835 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:04.245750 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:04.205302 24476 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:04.275229 24840 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:12:04.237062 24469 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:04.329161 24799 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.333699 24799 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.333922 24469 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:04.334854 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 141727us
I20260709 13:12:04.324275 24815 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:04.337769 24412 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.338704 24815 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.339468 24646 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.340056 24815 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:04.249414 24590 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:04.207521 24478 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.217423 24833 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.309913 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:04.285878 24844 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:04.288394 24831 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.298758 24829 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.315106 22732 leader_election.cc:403] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:12:04.217619 24756 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.321954 24611 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.323086 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:04.327864 22840 raft_consensus.cc:2376] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5. Re-sending same reply.
I20260709 13:12:04.330585 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1086) took 1710 ms. Trace:
I20260709 13:12:04.332557 24476 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.333029 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 223) took 1714 ms. Trace:
I20260709 13:12:04.293701 24684 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:04.341010 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 108836us
I20260709 13:12:04.374871 22910 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:12:04.356529 22907 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:04.355721 24590 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:04.382081 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133587us
W20260709 13:12:04.383949 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74166.9us
I20260709 13:12:04.386305 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:04.388224 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:04.389317 22840 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:04.353492 24779 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:12:04.378240 24611 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:04.377774 24790 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:04.408250 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:04.407706 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:04.413354 22840 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:04.403386 24541 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:04.381027 24809 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.395682 24585 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:12:04.378147 24660 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.412328 24748 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:04.394393 24834 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:12:04.410656 24790 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:04.412177 24773 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.395517 24684 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.514549 24684 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.515561 24684 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3.
W20260709 13:12:04.442917 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.358739 22835 rpcz_store.cc:276] 0709 13:12:02.619940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.740387 (+120447us) service_pool.cc:224] Handling call
0709 13:12:02.740794 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.022999 (+1282205us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b540040d6a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af31c0 00007f68a1af4231 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c
0709 13:12:04.328663 (+305664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.330513 (+  1850us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769742592}
I20260709 13:12:04.375782 23459 rpcz_store.cc:276] 0709 13:12:02.618522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.626608 (+  8086us) service_pool.cc:224] Handling call
0709 13:12:02.720668 (+ 94060us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:03.950530 (+1229862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.332706 (+382176us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400587fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:04.332929 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799173504}
W20260709 13:12:04.484359 24855 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 13:12:04.488648 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.448800 24790 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:04.446475 24659 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.422231 24828 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:12:04.417155 24787 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:12:04.447346 24541 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.415385 24727 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:12:04.439208 24476 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:04.499641 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 222) took 1885 ms (client timeout 1547 ms). Trace:
I20260709 13:12:04.500558 24440 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:04.501873 24831 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:04.501947 24664 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.501739 24799 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.501345 24809 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:04.502588 24671 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.535s	user 0.015s	sys 0.000s
I20260709 13:12:04.504041 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:04.505820 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.506584 24693 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:04.507653 24748 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:04.508342 24648 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.436161 24726 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:04.417390 24859 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.507221 22840 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:04.509913 24478 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.511096 24660 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.489714 24590 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:04.535254 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119801us
I20260709 13:12:04.642339 22841 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:12:04.558051 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.643615 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.644007 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.582588 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.567384 24521 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.002s
I20260709 13:12:04.584877 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:04.534345 22834 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
W20260709 13:12:04.583137 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 359686us
W20260709 13:12:04.580544 23461 rpcz_store.cc:269] 0709 13:12:02.614570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.617877 (+  3307us) service_pool.cc:224] Handling call
0709 13:12:02.858623 (+240746us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:04.455846 (+1597223us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54006707d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:04.499551 (+ 43705us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2641447680}
I20260709 13:12:04.885922 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:04.509747 24800 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:04.613909 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.642190 24693 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.602843 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:04.668998 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.805230 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 570990us
I20260709 13:12:04.691504 22838 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:04.578514 24541 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.583772 24440 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.569391 24773 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.586683 24831 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
W20260709 13:12:04.700968 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 197028us
I20260709 13:12:04.603623 24843 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.600502 24659 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:04.581969 24476 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.648622 24684 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:04.892213 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 488960us
W20260709 13:12:04.893376 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 219288us
I20260709 13:12:04.899544 24646 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:12:04.895771 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.899641 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 314877us
I20260709 13:12:04.689881 24799 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.697228 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:04.809103 24761 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.920996 24861 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.921666 24861 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.922497 24440 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:04.923842 24861 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.924425 24861 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:04.924746 24412 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:04.905299 22834 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:04.805578 23457 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:04.886768 24668 raft_consensus.cc:2764] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:12:04.906916 22839 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:04.637904 24859 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:04.894420 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.946249 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:04.946791 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.910509 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:04.914448 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:04.932678 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:04.908915 24863 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:04.930469 23459 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.939254 23457 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:04.949298 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:04.908018 24843 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.954728 22910 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:04.958976 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:04.966331 24870 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:12:04.967865 24773 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:04.968173 22913 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:04.946099 24684 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:04.971194 24476 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.976285 24859 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.977589 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:04.977856 24541 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:04.978833 24611 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.004s
I20260709 13:12:04.979004 24693 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:04.964226 24648 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:04.979803 24874 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:04.980878 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:04.981801 24440 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:04.937155 24646 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:04.961622 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:04.894833 24748 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:04.934943 24828 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:12:04.989401 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.991005 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.951057 24863 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:04.997397 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.990659 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:04.989338 24861 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:05.000622 24869 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:12:05.035669 22910 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:05.023264 24671 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.017081 24859 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:05.021682 24874 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:05.061550 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 99999.6us
I20260709 13:12:05.046968 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:05.063730 23463 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:05.037634 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.026422 24872 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:05.043136 24440 consensus_queue.cc:1243] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:12:05.043517 24693 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.042439 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:05.079401 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.078531 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1093) took 1106 ms. Trace:
I20260709 13:12:05.040041 24863 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:05.065012 24831 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.087288 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.131932 22914 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:05.088013 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.089478 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.125517 22915 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:05.130327 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1099) took 1052 ms. Trace:
I20260709 13:12:05.055426 24521 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:05.135350 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 186144us
I20260709 13:12:05.129632 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46252 (request call id 5) took 1329 ms. Trace:
I20260709 13:12:05.127513 22838 rpcz_store.cc:276] 0709 13:12:03.972189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.972452 (+   263us) service_pool.cc:224] Handling call
0709 13:12:05.078228 (+1105776us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b540028b4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:05.078430 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412229888}
I20260709 13:12:05.136144 22909 rpcz_store.cc:276] 0709 13:12:03.780711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.780903 (+   192us) service_pool.cc:224] Handling call
0709 13:12:05.086645 (+1305742us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b54000fb254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aeb853 00007f68a6ab960a 00007f68a003f7d4 00007f68a003f762 00007f68a003f691 00007f68a003f60c 00007f68a003e8c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:05.086768 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103827200}
I20260709 13:12:05.137552 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1107) took 1028 ms. Trace:
I20260709 13:12:05.052296 24611 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:05.139842 22841 rpcz_store.cc:276] 0709 13:12:04.108617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.862934 (+754317us) service_pool.cc:224] Handling call
0709 13:12:04.877998 (+ 15064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.878327 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:05.030566 (+152239us) log.cc:844] Serialized 14 byte log entry
0709 13:12:05.137143 (+106577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:05.137203 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:05.137257 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.137333 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.137467 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":233449,"threads_started":2}
I20260709 13:12:05.070824 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:05.143543 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72859.2us
W20260709 13:12:05.144395 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 166954us
I20260709 13:12:05.067305 24874 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.082109 24412 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:12:05.136137 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:05.148110 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.148524 24787 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:05.151584 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.081939 24865 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:12:05.154325 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.181071 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.181918 22909 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:05.183319 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.184789 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.068872 24671 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.146023 24761 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:05.158108 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.158519 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:05.148034 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 233684us
I20260709 13:12:05.173810 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:05.058454 24859 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.185462 22909 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:05.136644 24871 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 13:12:05.134327 22837 rpcz_store.cc:276] 0709 13:12:04.077415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.194910 (+117495us) service_pool.cc:224] Handling call
0709 13:12:04.211338 (+ 16428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.215920 (+  4582us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:04.249214 (+ 33294us) log.cc:844] Serialized 14 byte log entry
0709 13:12:04.899543 (+650329us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b18001e61c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:05.129326 (+229783us) log.cc:844] Serialized 14 byte log entry
0709 13:12:05.129688 (+   362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:05.129748 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:05.129803 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.129873 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.130268 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24595,"prepare.queue_time_us":59676,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":26164,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":6371,"spinlock_wait_cycles":10179456,"thread_start_us":201165,"threads_started":2,"wal-append.queue_time_us":737863}
I20260709 13:12:05.193996 22910 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:05.188079 22841 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:05.197336 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:05.186118 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.217166 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:05.157665 23464 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:05.204188 24590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.023s
I20260709 13:12:05.176395 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.150933 24521 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.144835 24831 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.177857 24787 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.148653 24693 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.199021 24824 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:05.222076 24469 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:12:05.227207 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.206224 24859 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.280892 22838 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.199137 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.187675 24671 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:05.277350 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47618.4us
I20260709 13:12:05.198915 24611 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:05.263739 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.194651 24881 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:05.313867 24881 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.314540 24881 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.315202 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.264700 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.317970 22833 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:05.319200 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.320305 22833 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:05.320490 24521 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:05.272022 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54926us
I20260709 13:12:05.273777 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.330852 22837 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:05.146847 24646 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:05.277046 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.269800 24872 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:05.283742 24659 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.229664 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:05.285640 24868 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:12:05.277325 24843 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:05.280174 24879 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:12:05.291424 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1970) took 1327 ms. Trace:
I20260709 13:12:05.292105 24883 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:12:05.275664 22834 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.199677 24866 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:12:05.270844 24882 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:05.259516 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:05.299307 22914 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:05.273984 24787 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:05.302546 24877 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:12:05.298254 24684 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:05.304050 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.304337 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.305295 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1109) took 1196 ms. Trace:
I20260709 13:12:05.305873 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1102) took 1199 ms. Trace:
I20260709 13:12:05.372113 24872 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:05.374600 22840 rpcz_store.cc:276] 0709 13:12:04.106946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.645654 (+538708us) service_pool.cc:224] Handling call
0709 13:12:04.899834 (+254180us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:04.929109 (+ 29275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:04.929152 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:04.930048 (+   896us) log.cc:844] Serialized 52 byte log entry
0709 13:12:05.059113 (+129065us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:05.209738 (+150625us) log.cc:844] Serialized 14 byte log entry
0709 13:12:05.210085 (+   347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.210207 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:05.305584 (+ 95377us) raft_consensus.cc:1685] finished
0709 13:12:05.305617 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.305796 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43711,"prepare.queue_time_us":171393,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":22308,"raft.run_cpu_time_us":439,"raft.run_wall_time_us":29411,"thread_start_us":232356,"threads_started":2,"wal-append.queue_time_us":226046}
I20260709 13:12:05.373660 22836 rpcz_store.cc:276] 0709 13:12:04.109192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.924508 (+815316us) service_pool.cc:224] Handling call
0709 13:12:04.925045 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:04.931514 (+  6469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:04.931563 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:04.945252 (+ 13689us) log.cc:844] Serialized 26 byte log entry
0709 13:12:05.052019 (+106767us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:05.052095 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.052204 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:05.305059 (+252855us) raft_consensus.cc:1685] finished
0709 13:12:05.305087 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.305232 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36902,"raft.queue_time_us":16639,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7886,"thread_start_us":53253,"threads_started":1,"wal-append.queue_time_us":172505}
I20260709 13:12:05.377215 23457 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.378382 23457 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:12:05.378635 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:05.379191 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:05.380620 24790 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.000s
I20260709 13:12:05.382055 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 433) took 1031 ms. Trace:
I20260709 13:12:05.332556 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:05.257654 24590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.003s	sys 0.023s
I20260709 13:12:05.346642 22838 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:05.277117 24824 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:05.347967 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:05.350611 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.351819 22910 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:05.317433 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:05.356439 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.328286 24831 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:05.371203 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:05.383028 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50625.4us
I20260709 13:12:05.382515 23466 rpcz_store.cc:276] 0709 13:12:04.351189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.351372 (+   183us) service_pool.cc:224] Handling call
0709 13:12:04.383485 (+ 32113us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:04.419079 (+ 35594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:04.419114 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:04.419677 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:12:04.596970 (+177293us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:04.597051 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.597166 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:05.381775 (+784609us) raft_consensus.cc:1685] finished
0709 13:12:05.381798 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.381970 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28445,"raft.queue_time_us":31622,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5252,"thread_start_us":113117,"threads_started":1,"wal-append.queue_time_us":549737}
I20260709 13:12:05.381072 24790 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.357847 24659 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.388079 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.528546 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 210) took 1004 ms. Trace:
I20260709 13:12:05.476869 22909 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:05.529595 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 306744us
I20260709 13:12:05.533929 22911 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:05.429831 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 209) took 1048 ms. Trace:
I20260709 13:12:05.533742 22836 rpcz_store.cc:276] 0709 13:12:04.524461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.527493 (+1003032us) service_pool.cc:224] Handling call
0709 13:12:05.527902 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.528135 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.528163 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:05.528212 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:05.528249 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.528318 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.528436 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:05.528826 22806 connection.cc:256] RPC call timeout handler was delayed by -3.2225997269998743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
W20260709 13:12:05.537021 22806 connection.cc:256] RPC call timeout handler was delayed by -3.1721997269999065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 13:12:05.441844 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.536005 22841 rpcz_store.cc:276] 0709 13:12:04.381776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.404694 (+1022918us) service_pool.cc:224] Handling call
0709 13:12:05.405104 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.405299 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.405328 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:05.405375 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:05.405408 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.405468 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.429724 (+ 24256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:05.539160 24831 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:05.539970 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:05.406458 24888 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:12:05.540817 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 192912us
I20260709 13:12:05.546087 22908 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:05.534327 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:05.370036 24882 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:05.454526 24726 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.002s
W20260709 13:12:05.390628 24852 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.038s	user 0.001s	sys 0.003s
I20260709 13:12:05.542706 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 211) took 1017 ms. Trace:
I20260709 13:12:05.556231 22838 rpcz_store.cc:276] 0709 13:12:04.524882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.534942 (+1010060us) service_pool.cc:224] Handling call
0709 13:12:05.537147 (+  2205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.537327 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.537366 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:05.537419 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:05.537470 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.542049 (+  4579us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:05.542639 (+   590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:05.540732 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.500465 24784 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:05.397710 24684 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:05.571271 24684 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.571827 24684 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.542265 24881 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.559434 22909 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:05.412720 23459 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:05.499306 24863 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.561853 22836 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:05.562453 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.562798 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:05.570640 24873 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.004s	sys 0.008s
I20260709 13:12:05.562386 24726 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:05.363890 22907 rpcz_store.cc:276] 0709 13:12:03.963726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.963896 (+   170us) service_pool.cc:224] Handling call
0709 13:12:03.964333 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.390428 (+426095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.291034 (+900606us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54001d0e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:05.291343 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865494912}
I20260709 13:12:05.414014 24611 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.577368 22906 raft_consensus.cc:2410] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:05.528873 24859 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.605283 24875 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:05.561417 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.628716 22911 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:05.625801 24659 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:05.552292 24882 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:05.630687 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 212) took 1096 ms. Trace:
I20260709 13:12:05.651281 22836 rpcz_store.cc:276] 0709 13:12:04.534652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.549019 (+1014367us) service_pool.cc:224] Handling call
0709 13:12:05.561655 (+ 12636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.630376 (+ 68721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.630618 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:05.550482 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.625s (SENT)
W20260709 13:12:05.634135 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.636612 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:05.652516 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:05.640867 22837 raft_consensus.cc:2410] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:12:05.630211 24873 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.005s	sys 0.009s
I20260709 13:12:05.656805 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.657778 22836 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:05.649570 22913 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:05.659241 24872 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:05.645702 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.659946 24859 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:05.552821 24831 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.629626 24787 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:05.657773 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:05.663168 22759 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:05.664501 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.656011 24659 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:05.655877 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:05.605157 24896 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:12:05.658030 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:05.655479 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105066us
I20260709 13:12:05.653436 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:05.665927 22836 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:05.668776 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:05.670723 22913 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:05.670661 22765 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:05.670717 22758 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:12:05.645435 24875 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.675987 24843 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:05.687337 24875 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.691639 24881 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:05.693817 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:05.695598 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:05.695686 24590 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:05.697252 24872 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:05.697254 24835 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:05.698364 24726 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:05.698927 24726 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:05.700103 24726 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.680749 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.702404 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.703017 22836 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.686591 24863 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:05.687451 24843 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:05.708630 24693 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:05.708840 24843 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.709679 23457 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:12:05.710423 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 224) took 2127 ms. Trace:
I20260709 13:12:05.683975 22834 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:05.702790 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:05.712531 24668 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:05.712607 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:05.712960 24668 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.713326 24668 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:05.713673 24719 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:12:05.714727 24660 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:05.715298 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.715480 24660 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:05.716516 24660 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.710757 23457 rpcz_store.cc:276] 0709 13:12:03.582859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.584423 (+  1564us) service_pool.cc:224] Handling call
0709 13:12:03.584810 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.805426 (+1220616us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54006115a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af31c0 00007f68a1af4231 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c
0709 13:12:05.376812 (+571386us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b54006115a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6f06f 00007f68a1ae7f60 00007f68a1ae97de 00007f68a1af3b49 00007f68a1af4416 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62
0709 13:12:05.710088 (+333276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.710327 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165047040}
I20260709 13:12:05.698457 24872 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:05.719147 24872 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:05.719408 24874 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.004s
I20260709 13:12:05.719787 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:05.719916 24874 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.720456 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.720248 24872 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.721702 24829 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:05.722265 24829 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:05.723212 22912 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:05.723552 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.679919 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:05.724157 22833 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:05.679128 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.728479 22836 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:05.729423 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.730053 22836 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:05.650872 24784 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.655516 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.742182 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:05.672468 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:05.743175 23462 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:05.753602 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:05.755659 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.759756 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.761132 22911 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:05.645399 24824 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.766880 24824 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:05.757753 23459 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
W20260709 13:12:05.767560 24761 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.008s
I20260709 13:12:05.768193 24761 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:05.768440 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:05.769132 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.770167 24829 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.759357 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:05.773537 23457 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:05.733997 24707 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:05.784736 24707 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.770833 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.797161 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.797425 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:05.802928 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:05.810730 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:05.811048 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.811685 22907 raft_consensus.cc:606] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:05.812042 22907 raft_consensus.cc:614] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:05.815637 24881 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:05.816522 24900 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:12:05.819720 24585 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:05.820315 24585 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:05.821420 24585 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.823043 24585 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:05.809551 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.745281 24784 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.831707 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:05.833773 23462 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:05.833880 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.807502 24859 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:05.834920 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:05.842788 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 208163us
W20260709 13:12:05.832986 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 156630us
I20260709 13:12:05.848410 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.849851 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.850960 22911 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:05.851971 24808 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.608s	user 0.008s	sys 0.022s
I20260709 13:12:05.867491 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.868960 22911 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:05.867525 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.847378 24881 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:05.873271 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.874042 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:05.836685 24859 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:05.876207 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:05.882563 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.885880 24459 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:05.886479 24459 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:05.887528 24459 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.877954 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:05.894435 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.853s (SENT)
I20260709 13:12:05.881708 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:05.869306 24808 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.624s	user 0.009s	sys 0.023s
I20260709 13:12:05.911921 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:05.926268 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.928364 22838 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:05.931478 22907 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 4. Ops: []
I20260709 13:12:05.934537 22907 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:12:05.932150 24901 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:12:05.937985 22907 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:05.879118 24859 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:05.944832 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.952389 22838 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:12:05.957199 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 323539us
I20260709 13:12:05.966372 24902 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:05.967128 24875 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:05.913808 24881 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:05.985406 22838 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:05.904292 24668 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:05.989156 24719 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:05.996357 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:05.998119 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.239898 22758 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:05.998620 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.241631 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.243106 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.954344 24832 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:05.988026 24835 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.004246 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:05.968250 24863 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.011106 24784 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:06.023229 24843 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.262094 24881 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.264685 24843 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:06.267591 24719 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:06.268467 24719 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:06.268795 24719 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:06.041369 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.270411 22765 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:06.042083 24693 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.272043 24693 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:06.044199 24867 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.234s	user 0.002s	sys 0.003s
I20260709 13:12:06.095433 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.275419 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:06.275787 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 432) took 1932 ms. Trace:
W20260709 13:12:06.276144 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.126603 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.276368 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:06.144901 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.277032 22914 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:06.277792 22759 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:06.163470 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.278877 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.177856 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.225365 22911 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:06.226089 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 331690us
W20260709 13:12:06.227123 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 315291us
W20260709 13:12:06.228998 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 382762us
W20260709 13:12:06.284235 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.232750 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.233745 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:06.227809 24902 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:06.236510 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1114) took 1061 ms. Trace:
I20260709 13:12:06.234566 24875 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:06.254376 24863 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.254884 24835 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:06.310241 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.255914 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:06.257891 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:06.262308 24684 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.001s
W20260709 13:12:06.277375 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.276304 23465 rpcz_store.cc:276] 0709 13:12:04.343254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.343467 (+   213us) service_pool.cc:224] Handling call
0709 13:12:04.356064 (+ 12597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:04.403602 (+ 47538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:04.403662 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:04.409932 (+  6270us) log.cc:844] Serialized 26 byte log entry
0709 13:12:04.445183 (+ 35251us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:04.445235 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.445325 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.043262 (+1597937us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:06.275397 (+232135us) raft_consensus.cc:1685] finished
0709 13:12:06.275429 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.275706 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33576,"raft.queue_time_us":43878,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4372,"spinlock_wait_cycles":9313664,"thread_start_us":16034,"threads_started":1,"wal-append.queue_time_us":259126}
W20260709 13:12:06.261229 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.297940 22838 rpcz_store.cc:276] 0709 13:12:05.175570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.953763 (+778193us) service_pool.cc:224] Handling call
0709 13:12:05.981547 (+ 27784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.236137 (+254590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.236430 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:06.301131 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:06.306267 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:06.293489 24832 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.313261 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:06.314284 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.314924 22915 raft_consensus.cc:2410] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:06.311867 22763 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.316093 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:06.282344 24844 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.326005 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1106) took 2217 ms. Trace:
I20260709 13:12:06.329756 22839 rpcz_store.cc:276] 0709 13:12:04.108283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.813153 (+704870us) service_pool.cc:224] Handling call
0709 13:12:04.901765 (+ 88612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.944951 (+ 43186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.325773 (+1380822us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b54000c6cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:06.325942 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11333376}
I20260709 13:12:06.326014 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1991) took 1285 ms. Trace:
I20260709 13:12:06.275259 24829 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.331245 22908 rpcz_store.cc:276] 0709 13:12:05.040965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.041119 (+   154us) service_pool.cc:224] Handling call
0709 13:12:05.057881 (+ 16762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.547009 (+489128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.325762 (+778753us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54000cf654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:06.325944 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122195200}
W20260709 13:12:06.344877 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.301752 24660 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.011s	sys 0.043s
W20260709 13:12:06.302570 24882 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.005s
I20260709 13:12:06.275522 24872 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:06.367180 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.368829 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:06.369695 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:06.369892 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:06.370443 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.300194 24727 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.300952 24902 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.302462 24828 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.386330 23462 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:06.384174 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:06.388618 24693 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:06.441604 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44911.3us
I20260709 13:12:06.443454 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:06.396905 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74507.4us
I20260709 13:12:06.325376 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:06.404078 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.446254 22907 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:12:06.446440 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.409721 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.418258 23463 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:06.447554 22806 connection.cc:256] RPC call timeout handler was delayed by -2.6054709229999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25730000000000003s for a response
I20260709 13:12:06.422752 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:06.447899 22806 connection.cc:256] RPC call timeout handler was delayed by -2.2118997269999454s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3032s for a response
I20260709 13:12:06.448321 22757 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.430827 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 207) took 2093 ms. Trace:
W20260709 13:12:06.398089 24660 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.013s	sys 0.044s
I20260709 13:12:06.303591 24719 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.433334 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.451470 22833 rpcz_store.cc:276] 0709 13:12:04.337714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.321097 (+983383us) service_pool.cc:224] Handling call
0709 13:12:05.723949 (+402852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:05.727014 (+  3065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.430504 (+703490us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54000c67d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:06.430739 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907131264}
I20260709 13:12:06.467242 22758 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.434836 23465 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:12:06.437595 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:06.388629 22759 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:06.318353 24684 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.473021 22766 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:12:06.416074 22763 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.466167 22839 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:06.399152 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:06.387274 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.477869 22765 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:06.470160 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:06.471843 22806 connection.cc:256] RPC call timeout handler was delayed by -1.6411079499999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:12:06.479785 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4980079499999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 13:12:06.480291 22806 connection.cc:256] RPC call timeout handler was delayed by -1.440407950000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 13:12:06.480662 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4053079499999512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 13:12:06.481042 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2937079499999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
W20260709 13:12:06.481427 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2280079500000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 13:12:06.481822 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0930079500000147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2384s for a response
I20260709 13:12:06.474785 22907 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:06.474251 24756 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.386580 24829 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.486805 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1120) took 1305 ms. Trace:
I20260709 13:12:06.487066 22761 raft_consensus.cc:2376] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4. Re-sending same reply.
I20260709 13:12:06.440456 24918 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.522167 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:06.525559 22839 rpcz_store.cc:276] 0709 13:12:05.181085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.426536 (+1245451us) service_pool.cc:224] Handling call
0709 13:12:06.465969 (+ 39433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.477158 (+ 11189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.486744 (+  9586us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:06.526234 23463 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:06.522264 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.527704 22766 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.419366 24875 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:12:06.530191 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:06.530871 22763 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:12:06.531083 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44384us
I20260709 13:12:06.534801 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.488775 22757 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:06.474108 24784 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:06.529275 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 85903.4us
I20260709 13:12:06.406080 24882 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.434944 24727 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.427060 24872 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.466842 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:06.468453 24831 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:06.482954 24756 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.543710 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.465917 24440 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:12:06.537088 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.550468 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:06.552644 23461 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:06.553200 22760 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:06.554013 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:06.554857 24916 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:06.557920 24727 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:06.558682 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1129) took 2680 ms. Trace:
I20260709 13:12:06.547784 24719 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:06.561256 24863 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:06.483956 24908 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:06.546860 22833 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:06.572512 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1122) took 1389 ms. Trace:
I20260709 13:12:06.576838 22765 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:06.576802 22766 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.579234 23465 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:06.623620 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.623140 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.440968 24828 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.631263 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.628189 24831 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.639521 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.487813 24815 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:12:06.650658 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1121) took 1468 ms. Trace:
I20260709 13:12:06.667295 22833 rpcz_store.cc:276] 0709 13:12:05.182009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.485986 (+1303977us) service_pool.cc:224] Handling call
0709 13:12:06.537943 (+ 51957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.650288 (+112345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.650594 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:06.654111 22757 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.664211 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:06.664997 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125144us
I20260709 13:12:06.670348 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:06.671725 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:06.673709 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:12:06.660333 24859 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.703s	user 0.011s	sys 0.003s
I20260709 13:12:06.677234 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:06.638006 24668 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:06.547464 24918 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.678610 22758 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:06.570752 24784 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.687453 23465 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:12:06.688162 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 216528us
I20260709 13:12:06.664189 24440 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.570690 24881 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:12:06.689859 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.693017 22763 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
W20260709 13:12:06.684368 24911 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:12:06.698792 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:06.720220 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.856210 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.697008 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:06.842648 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.847728 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.858405 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.677244 22760 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.859375 22761 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:06.679929 24903 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:06.870180 22757 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.876256 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.629364 22832 rpcz_store.cc:276] 0709 13:12:03.877984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.882372 (+  4388us) service_pool.cc:224] Handling call
0709 13:12:06.558422 (+2676050us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54000398d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:06.558605 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5107095296}
I20260709 13:12:06.882745 22760 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:06.883875 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.884938 22760 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:06.887888 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.889199 22757 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:06.895550 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.897598 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.893870 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.901639 22766 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:06.695612 24440 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:06.675870 24831 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:06.912761 22766 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:06.684799 24859 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.928865 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:06.932592 24882 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:06.930765 22766 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:06.934855 24784 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:06.935701 23457 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:06.937074 22766 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:06.941937 24829 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:06.943842 24756 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:06.946156 24828 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:06.947533 24872 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:06.876221 24832 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:06.948799 24440 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:06.887261 24917 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.951763 24903 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:06.952348 24903 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.952775 24903 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.965077 22758 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:07.047325 22758 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:07.050195 22757 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:07.053869 22765 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:07.093866 22766 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:12:06.972708 24440 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.005719 24684 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.131721 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.131994 22766 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:07.132483 22758 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:07.139204 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 465563us
I20260709 13:12:07.140403 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:07.150816 22908 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.004s	sys 0.009s
W20260709 13:12:07.151542 22908 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.005s	sys 0.010s
I20260709 13:12:07.154943 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:07.121281 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:07.167066 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:07.176326 24902 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.000s
I20260709 13:12:07.164448 24881 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:07.188617 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.200636 22766 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:07.204206 22880 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.006s	sys 0.019s
W20260709 13:12:07.204669 22880 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.006s	sys 0.019s
W20260709 13:12:07.204967 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 544588us
W20260709 13:12:07.153352 24459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.001s	sys 0.013s
I20260709 13:12:07.156126 24756 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.191143 24872 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:07.200057 24844 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.001s
I20260709 13:12:07.222657 22758 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:07.231499 22833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.014s	sys 0.020s
W20260709 13:12:07.215934 24459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.002s	sys 0.013s
W20260709 13:12:07.230557 22880 leader_election.cc:336] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.896s (SENT)
I20260709 13:12:07.274529 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:07.273468 22766 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:07.275735 22762 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:12:07.275173 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:07.272747 22833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.019s	sys 0.033s
I20260709 13:12:07.276504 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.277175 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:07.206038 24902 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:07.285408 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54947.5us
W20260709 13:12:07.238603 24916 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.010s	sys 0.027s
I20260709 13:12:07.288373 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:07.208022 24881 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.290652 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.285203 24923 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:07.292778 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 137956us
I20260709 13:12:07.294617 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.294822 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.278448 24707 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.278959 24844 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:07.289572 24916 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.015s	sys 0.037s
I20260709 13:12:07.277544 24882 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:07.281878 24933 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.332502 22766 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.286552 24829 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:07.239490 24872 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:07.337025 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:07.339795 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.232659 24756 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:07.332248 23432 connection.cc:256] RPC call timeout handler was delayed by -1.1146215429998847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
I20260709 13:12:07.342913 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.305086 24927 ts_tablet_manager.cc:933] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.300391 24800 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:07.347050 23432 connection.cc:256] RPC call timeout handler was delayed by -1.0607936099999051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
I20260709 13:12:07.347265 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1998) took 1345 ms. Trace:
W20260709 13:12:07.345935 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57622.6us
I20260709 13:12:07.352160 24756 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.348441 22908 rpcz_store.cc:276] 0709 13:12:06.002168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.419075 (+416907us) service_pool.cc:224] Handling call
0709 13:12:06.440574 (+ 21499us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:06.465841 (+ 25267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.465870 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:06.488061 (+ 22191us) log.cc:844] Serialized 52 byte log entry
0709 13:12:07.152411 (+664350us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:07.287663 (+135252us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.345876 (+ 58213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.345995 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.346951 (+   956us) raft_consensus.cc:1685] finished
0709 13:12:07.346975 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.347188 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":284,"prepare.queue_time_us":193637,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":19046,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":15810,"spinlock_wait_cycles":7552,"thread_start_us":753173,"threads_started":3,"wal-append.queue_time_us":636010}
W20260709 13:12:07.363297 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.364224 22766 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:07.301192 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1101) took 3188 ms. Trace:
I20260709 13:12:07.366137 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:07.366377 22758 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:12:07.369450 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.290827 24815 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:12:07.299901 24875 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:12:07.345299 24844 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:07.372040 24925 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:12:07.372881 22758 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:07.373560 22763 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:07.347977 24859 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.336798 24882 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.368656 24784 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:07.355962 24929 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:12:07.342933 24934 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:07.394972 24829 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:07.363179 24707 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:07.354393 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.441700 24784 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:07.362071 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:07.446017 23457 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:07.446311 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1124) took 2102 ms. Trace:
I20260709 13:12:07.446527 24784 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:07.408509 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.404609 22761 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:07.452522 24844 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.453722 24881 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.453908 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1123) took 2241 ms. Trace:
I20260709 13:12:07.436767 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 215) took 2057 ms. Trace:
I20260709 13:12:07.454365 22761 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:07.454658 22841 rpcz_store.cc:276] 0709 13:12:05.379821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.400239 (+2020418us) service_pool.cc:224] Handling call
0709 13:12:07.436229 (+ 35990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.436428 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.436455 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.436510 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:07.436539 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.436595 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.436707 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.455838 22761 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:12:07.456111 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:07.457090 22761 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:07.375720 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.458016 22908 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:07.458421 22761 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:07.459649 24726 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.459767 22761 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:07.446774 22832 rpcz_store.cc:276] 0709 13:12:05.343666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.882626 (+1538960us) service_pool.cc:224] Handling call
0709 13:12:06.883081 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:06.883495 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.883530 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:06.884136 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.274824 (+390688us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:07.278345 (+  3521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.283999 (+  5654us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.446033 (+162034us) raft_consensus.cc:1685] finished
0709 13:12:07.446064 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.446238 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2219,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":89,"thread_start_us":374146,"threads_started":1,"wal-append.queue_time_us":414752}
I20260709 13:12:07.370071 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:07.449101 24902 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:07.440363 24934 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:07.450142 24903 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.306279 24923 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:07.373963 24831 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:07.346392 24933 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:07.465736 23432 leader_election.cc:336] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.668s (SENT)
I20260709 13:12:07.482807 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:07.461764 24937 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:12:07.450162 24931 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:12:07.382668 24808 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:12:07.492769 24808 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:07.493472 24808 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:07.495008 24919 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.942s	user 0.029s	sys 0.045s
W20260709 13:12:07.467214 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 97268us
I20260709 13:12:07.496400 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:07.497957 24829 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:07.498569 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:07.462659 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 216) took 1995 ms. Trace:
I20260709 13:12:07.499351 24412 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.499584 22841 rpcz_store.cc:276] 0709 13:12:05.466934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.457057 (+1990123us) service_pool.cc:224] Handling call
0709 13:12:07.457315 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.457486 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.462266 (+  4780us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.462316 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:07.462355 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.462416 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.462588 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.472143 24934 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.501580 22841 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:07.502164 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1126) took 1786 ms. Trace:
I20260709 13:12:07.502463 22841 rpcz_store.cc:276] 0709 13:12:05.716147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.500824 (+1784677us) service_pool.cc:224] Handling call
0709 13:12:07.501193 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.501900 (+   707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.502106 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.472244 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.474447 22832 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:12:07.505790 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1140) took 1960 ms. Trace:
I20260709 13:12:07.475543 24918 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.478251 22763 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:12:07.506760 24920 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.946s	user 0.011s	sys 0.002s
I20260709 13:12:07.506183 22832 rpcz_store.cc:276] 0709 13:12:05.547430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.473191 (+1925761us) service_pool.cc:224] Handling call
0709 13:12:07.473876 (+   685us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:07.505438 (+ 31562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.505728 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.472759 24815 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:07.509119 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.494629 24808 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:07.509841 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1119) took 2329 ms. Trace:
I20260709 13:12:07.510195 22838 rpcz_store.cc:276] 0709 13:12:05.180141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.369286 (+1189145us) service_pool.cc:224] Handling call
0709 13:12:06.390910 (+ 21624us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:06.391856 (+   946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.391893 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:06.436790 (+ 44897us) log.cc:844] Serialized 518 byte log entry
0709 13:12:06.690256 (+253466us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:06.690320 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.690434 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.509608 (+819174us) raft_consensus.cc:1685] finished
0709 13:12:07.509630 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.509784 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":598,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":22919,"thread_start_us":160920,"threads_started":1,"wal-append.queue_time_us":668362}
I20260709 13:12:07.511011 24939 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.512472 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1130) took 1795 ms. Trace:
I20260709 13:12:07.512661 24939 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:07.454258 22833 rpcz_store.cc:276] 0709 13:12:05.212745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.677198 (+1464453us) service_pool.cc:224] Handling call
0709 13:12:06.682528 (+  5330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:06.683134 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.683169 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:06.683963 (+   794us) log.cc:844] Serialized 52 byte log entry
0709 13:12:07.276584 (+592621us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:07.399070 (+122486us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.402319 (+  3249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.402443 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.453675 (+ 51232us) raft_consensus.cc:1685] finished
0709 13:12:07.453696 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.453838 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2820,"prepare.queue_time_us":167503,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":7432,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":43762,"thread_start_us":705431,"threads_started":2,"wal-append.queue_time_us":669847}
I20260709 13:12:07.436681 24859 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.489312 24902 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:07.498656 24829 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:07.509404 23432 leader_election.cc:336] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.966s (SENT)
W20260709 13:12:07.526721 23432 leader_election.cc:336] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.056s (SENT)
I20260709 13:12:07.513998 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 1995) took 1513 ms. Trace:
W20260709 13:12:07.514350 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.514681 22837 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:07.488790 24933 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.512851 22838 rpcz_store.cc:276] 0709 13:12:05.717296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.511711 (+1794415us) service_pool.cc:224] Handling call
0709 13:12:07.511991 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.512172 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.512201 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.512243 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:07.512279 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.512332 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.512420 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.485411 22761 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:07.488595 24923 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.513212 24939 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:07.524222 22914 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:07.530370 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1129) took 1813 ms. Trace:
I20260709 13:12:07.534694 22837 rpcz_store.cc:276] 0709 13:12:05.717016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.509051 (+1792035us) service_pool.cc:224] Handling call
0709 13:12:07.509335 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.529988 (+ 20653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.530291 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.535218 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1118) took 2356 ms. Trace:
I20260709 13:12:07.510432 24808 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.527387 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:07.540457 23432 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.106s (SENT)
I20260709 13:12:07.509163 22832 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:07.522099 24761 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.542580 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.535588 22840 rpcz_store.cc:276] 0709 13:12:05.178814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.011927 (+833113us) service_pool.cc:224] Handling call
0709 13:12:06.012464 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:06.013815 (+  1351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.013862 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:06.045653 (+ 31791us) log.cc:844] Serialized 26 byte log entry
0709 13:12:06.432020 (+386367us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:06.432093 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.432199 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.506055 (+1073856us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:07.513294 (+  7239us) raft_consensus.cc:1685] finished
0709 13:12:07.513319 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.535150 (+ 21831us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":879,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":144,"spinlock_wait_cycles":24758016,"thread_start_us":191790,"threads_started":1,"wal-append.queue_time_us":335168}
I20260709 13:12:07.479846 24928 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:12:07.551479 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1132) took 1749 ms. Trace:
I20260709 13:12:07.552222 22837 rpcz_store.cc:276] 0709 13:12:05.797738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.535529 (+1737791us) service_pool.cc:224] Handling call
0709 13:12:07.546598 (+ 11069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.546808 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.546835 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.546880 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:07.546914 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.546973 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.547104 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.554100 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 218) took 1274 ms. Trace:
I20260709 13:12:07.554388 22837 rpcz_store.cc:276] 0709 13:12:06.280064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.553343 (+1273279us) service_pool.cc:224] Handling call
0709 13:12:07.553729 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.553875 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.553890 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.553919 (+    29us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:07.553937 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.553970 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.554056 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:07.542263 24719 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.896s	user 0.010s	sys 0.005s
I20260709 13:12:07.365413 22835 rpcz_store.cc:276] 0709 13:12:04.106586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.549094 (+442508us) service_pool.cc:224] Handling call
0709 13:12:04.616748 (+ 67654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:04.974695 (+357947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:04.974754 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:04.975354 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:12:05.076816 (+101462us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b18002c0aa0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:12:05.205604 (+128788us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:05.211773 (+  6169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.278274 (+2066501us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54000a5354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:07.282080 (+  3806us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.284122 (+  2042us) raft_consensus.cc:1685] finished
0709 13:12:07.286249 (+  2127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.292323 (+  6074us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29173,"raft.queue_time_us":8766,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4191,"spinlock_wait_cycles":731214592,"thread_start_us":101130,"threads_started":1,"wal-append.queue_time_us":316926,"wal-append.run_cpu_time_us":2168,"wal-append.run_wall_time_us":1007528}
I20260709 13:12:07.560673 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:07.561766 24748 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.564112 24940 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:12:07.565651 24875 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.570642 24872 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.572345 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:07.540634 24815 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:07.546406 24902 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:07.527344 24859 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.569144 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1128) took 1852 ms. Trace:
I20260709 13:12:07.575563 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.579548 22907 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:07.582759 24835 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.580938 24611 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:12:07.551713 24761 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.584463 24939 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.584996 24787 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:07.583804 24831 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.586410 22840 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:12:07.587718 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:07.578048 24917 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.004s
I20260709 13:12:07.582798 24903 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.582798 24815 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.592223 24611 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:07.592828 24611 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:07.595954 24934 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.581354 24707 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:12:07.602845 22761 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:07.602927 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1911) took 1307 ms. Trace:
I20260709 13:12:07.606343 22759 rpcz_store.cc:276] 0709 13:12:06.295652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.295853 (+   201us) service_pool.cc:224] Handling call
0709 13:12:07.602713 (+1306860us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b54003408d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:07.602825 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168983808}
I20260709 13:12:07.610440 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.611222 22761 raft_consensus.cc:2368] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5.
I20260709 13:12:07.579070 22832 rpcz_store.cc:276] 0709 13:12:05.716740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.508434 (+1791694us) service_pool.cc:224] Handling call
0709 13:12:07.508795 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.543836 (+ 35041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.569011 (+ 25175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:07.612056 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.612579 22761 raft_consensus.cc:2376] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4. Re-sending same reply.
I20260709 13:12:07.613131 24590 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.613524 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.604976 24917 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:07.612738 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.614840 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1112) took 1127 ms. Trace:
W20260709 13:12:07.617676 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68829.4us
I20260709 13:12:07.615204 22765 rpcz_store.cc:276] 0709 13:12:06.487605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.055970 (+568365us) service_pool.cc:224] Handling call
0709 13:12:07.384113 (+328143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.384353 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:07.436520 (+ 52167us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.522845 (+ 86325us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.612102 (+ 89257us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b18001289e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:07.612253 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.612300 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:07.612351 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.612421 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.613444 (+  1023us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54003403d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:07.614780 (+  1336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3285,"prepare.queue_time_us":53926,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":98,"raft.queue_time_us":3686,"spinlock_wait_cycles":772837248,"thread_start_us":107758,"threads_started":2,"wal-append.queue_time_us":138913}
I20260709 13:12:07.618558 22907 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:12:07.576906 24748 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:12:07.628032 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.629355 22907 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:07.585146 24829 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.631378 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.641932 22907 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:07.624415 24927 tablet_copy_client.cc:323] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
W20260709 13:12:07.633963 24440 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.006s
I20260709 13:12:07.629961 24707 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:12:07.637694 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.638484 24934 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.627121 24590 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:12:07.712796 24834 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.706074 24875 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:07.727257 24875 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:07.727434 24787 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:12:07.728335 24875 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.705703 24611 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.731067 24611 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.732425 22908 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:12:07.763263 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.772801 24833 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.773113 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.774601 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:07.778035 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.775s (SENT)
I20260709 13:12:07.778839 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:07.785740 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:07.786314 23432 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.696s (SENT)
I20260709 13:12:07.786769 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:07.803799 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:07.804322 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:07.806778 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 203191us
I20260709 13:12:07.807466 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.810148 24756 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:07.810958 24756 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.812772 24440 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:07.813167 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:07.815519 24872 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:07.815929 24872 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:07.816254 24872 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.824579 24872 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.803267 24875 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.827507 24872 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.829710 24944 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.830052 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:07.836272 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49971.7us
I20260709 13:12:07.843972 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:07.845419 23458 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:07.845733 24944 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:07.846010 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 439) took 1462 ms. Trace:
I20260709 13:12:07.846320 24944 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:07.846506 23458 rpcz_store.cc:276] 0709 13:12:06.383772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.396908 (+ 13136us) service_pool.cc:224] Handling call
0709 13:12:07.843744 (+1446836us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:07.845926 (+  2182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58833664}
I20260709 13:12:07.847769 24944 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:07.807175 24947 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:07.850193 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:07.850720 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72747us
W20260709 13:12:07.851522 22806 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.909s (SENT)
I20260709 13:12:07.854763 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1127) took 2138 ms. Trace:
I20260709 13:12:07.802798 24761 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.803344 24859 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.855279 22841 rpcz_store.cc:276] 0709 13:12:05.716473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.503166 (+1786693us) service_pool.cc:224] Handling call
0709 13:12:07.503508 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:07.515270 (+ 11762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.515304 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:07.515886 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:12:07.590474 (+ 74588us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:07.590542 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.590647 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:07.854517 (+263870us) raft_consensus.cc:1685] finished
0709 13:12:07.854546 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.854695 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1261,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":40349,"threads_started":1,"wal-append.queue_time_us":313894}
W20260709 13:12:07.857064 22807 connection.cc:256] RPC call timeout handler was delayed by -1.2184288549999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 13:12:07.855921 22909 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:12:07.857510 22807 connection.cc:256] RPC call timeout handler was delayed by -1.2040863600000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
W20260709 13:12:07.858431 22807 connection.cc:256] RPC call timeout handler was delayed by -1.1995863600000121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:12:07.858909 22807 connection.cc:256] RPC call timeout handler was delayed by -1.0861863600000561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
W20260709 13:12:07.859305 22807 connection.cc:256] RPC call timeout handler was delayed by -1.0096863600000461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 13:12:07.861760 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:07.850203 24947 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.864140 24947 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.864842 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:07.858211 24828 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.861984 22760 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:07.865700 24828 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:07.868383 24947 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.877775 24756 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:07.877936 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.879320 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.869091 24828 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.825284 23457 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:07.876011 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.880476 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.881119 22906 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:07.881841 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:07.888598 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.889356 24944 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:07.892069 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 300637us
I20260709 13:12:07.895134 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:07.895496 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.896754 22765 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:07.897100 22761 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:12:07.921995 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.924261 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.924934 22763 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.927955 22765 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:12:07.927917 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75486.8us
W20260709 13:12:07.929973 22806 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.965s (SENT)
I20260709 13:12:07.930357 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:07.931470 22759 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:07.932744 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:07.933014 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.161s (SENT)
I20260709 13:12:07.933380 22759 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:07.933465 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:07.934182 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:07.934397 24947 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:07.935385 24947 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:07.935513 22759 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:07.932641 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:07.938374 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.384s (SENT)
I20260709 13:12:07.939687 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:07.941619 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:07.942694 23457 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:07.942685 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:07.939976 24921 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.243s	user 0.010s	sys 0.002s
I20260709 13:12:07.927167 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 217) took 1647 ms. Trace:
I20260709 13:12:07.950515 22840 rpcz_store.cc:276] 0709 13:12:06.279711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.548079 (+1268368us) service_pool.cc:224] Handling call
0709 13:12:07.557915 (+  9836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.586862 (+ 28947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.926879 (+340017us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:07.927099 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64639744}
I20260709 13:12:07.947132 22760 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:12:07.947072 22806 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.579s (SENT)
I20260709 13:12:07.951653 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.954516 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:07.960230 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.962301 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:07.962637 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.962982 22832 raft_consensus.cc:2376] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5. Re-sending same reply.
I20260709 13:12:07.963886 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:07.968495 24790 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:07.969148 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:07.969944 22835 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
W20260709 13:12:07.971189 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.971791 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:07.959352 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:07.963949 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:07.983661 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:07.984598 22763 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:07.984436 22840 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
W20260709 13:12:07.986385 22912 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 166) took 2925 ms (client timeout 1966 ms). Trace:
I20260709 13:12:07.986765 22907 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:12:07.986855 22912 rpcz_store.cc:269] 0709 13:12:05.060920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.061085 (+   165us) service_pool.cc:224] Handling call
0709 13:12:07.986148 (+2925063us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400041854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:07.986307 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356468096}
I20260709 13:12:07.989923 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:07.990819 22760 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:07.999706 22764 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:08.008652 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1173) took 2301 ms. Trace:
I20260709 13:12:08.006352 23465 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:08.006078 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 174) took 2790 ms (client timeout 3109 ms). Trace:
W20260709 13:12:07.997522 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.993270 22914 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:08.007182 22758 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:08.042305 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:08.056082 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:08.056559 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:08.051017 22911 rpcz_store.cc:276] 0709 13:12:05.707074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.929693 (+222619us) service_pool.cc:224] Handling call
0709 13:12:05.950535 (+ 20842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.281356 (+330821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.008282 (+1726926us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54003c87d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:08.008562 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631523072}
I20260709 13:12:08.054656 22912 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:08.066439 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:08.051427 22909 rpcz_store.cc:269] 0709 13:12:05.215356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.758062 (+542706us) service_pool.cc:224] Handling call
0709 13:12:08.005776 (+2247714us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540020a3d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:08.005983 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4798268800}
I20260709 13:12:08.072082 22836 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:08.051867 24832 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.074651 24832 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:08.004967 23459 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:08.076310 22909 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:12:08.076787 22909 pending_rounds.cc:85] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 4
I20260709 13:12:08.076874 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 230) took 1774 ms. Trace:
I20260709 13:12:08.077134 22909 pending_rounds.cc:107] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:12:08.077579 22909 raft_consensus.cc:2889] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:08.077366 23459 rpcz_store.cc:276] 0709 13:12:06.302381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.303594 (+  1213us) service_pool.cc:224] Handling call
0709 13:12:08.004337 (+1700743us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:08.004658 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:08.076581 (+ 71923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.076805 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33548160}
I20260709 13:12:08.078042 24875 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.078548 24875 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.079033 24875 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.079959 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.082185 24947 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:08.082389 24761 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.082842 24947 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:08.082968 24761 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:08.088476 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.091228 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:08.092114 22911 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:08.009739 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:08.117555 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:08.119035 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.119684 22836 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:08.000473 24882 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:08.006816 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.001987 23457 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:08.085290 24947 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:07.994414 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet 0ad66b5c9842437d9320507b92b78fd5 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
I20260709 13:12:08.089711 22906 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:08.129052 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:08.129196 24835 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:08.129647 24874 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:08.092999 24726 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:12:08.136936 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.134683 24835 consensus_queue.cc:1243] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:08.134696 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet 0ad66b5c9842437d9320507b92b78fd5 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-0ad66b5c9842437d9320507b92b78fd5
I20260709 13:12:08.085377 24761 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:08.085747 24828 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:12:08.071375 24829 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:12:08.172366 24829 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:08.172850 24829 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:08.173940 24748 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:08.168216 23465 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:08.168532 24773 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:08.168334 24808 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:08.174010 24829 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:08.184258 24829 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:08.185792 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50327.1us
I20260709 13:12:08.186920 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.123937 24833 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.187814 24761 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.187708 24833 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:08.188396 22806 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.188625 24761 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:08.187039 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1907) took 2092 ms (client timeout 2384 ms). Trace:
I20260709 13:12:08.189635 24761 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:08.189693 22762 rpcz_store.cc:269] 0709 13:12:06.094816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.095020 (+   204us) service_pool.cc:224] Handling call
0709 13:12:08.186792 (+2091772us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54000811d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:08.186952 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300180736}
I20260709 13:12:08.190836 22912 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:06.650972 22839 rpcz_store.cc:276] 0709 13:12:05.182933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.544407 (+1361474us) service_pool.cc:224] Handling call
0709 13:12:06.571498 (+ 27091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.571974 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:06.572017 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:06.572076 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:06.572125 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.572230 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.572424 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4461,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":72085}
W20260709 13:12:08.201601 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 1990) took 3181 ms (client timeout 1711 ms). Trace:
I20260709 13:12:08.197041 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:08.202183 22913 rpcz_store.cc:269] 0709 13:12:05.020688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.020939 (+   251us) service_pool.cc:224] Handling call
0709 13:12:08.201312 (+3180373us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54000cfb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:08.201482 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345593600}
I20260709 13:12:08.215624 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.216921 22836 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:08.127286 24939 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:08.133198 24590 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.200664 24800 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.191720 24773 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:08.191715 24917 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:12:08.236007 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.224107 24590 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.240147 22836 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:08.226902 24800 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:08.226456 24939 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:08.264508 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67573.2us
I20260709 13:12:08.266758 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.240445 24590 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.261778 24800 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:08.291466 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 163942us
I20260709 13:12:08.304567 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.305290 22838 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:08.301920 24947 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:08.309314 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:08.286655 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.303845 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:08.312217 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.314447 22839 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:08.316588 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:08.317396 22838 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.321712 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.322744 22837 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:08.324513 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:08.321760 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.327265 22837 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.330513 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.333055 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.341856 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.342923 23465 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 2.
I20260709 13:12:08.346459 22839 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:12:08.347931 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.348529 22839 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:08.351513 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.352238 23457 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:08.353145 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.358227 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.359602 23457 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:08.361454 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.364166 22839 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:08.365576 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.367280 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:08.362855 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.368907 24940 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:08.369331 24940 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.369779 24940 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:08.370285 22839 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:08.370886 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:08.372581 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:08.373787 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:08.373745 24957 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:08.374943 24790 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.375571 24828 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:12:08.378041 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45125us
I20260709 13:12:08.302637 24459 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:08.309894 24967 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.389653 22840 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:08.312325 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 440) took 1916 ms. Trace:
I20260709 13:12:08.319211 24833 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:08.393939 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.399669 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:08.318679 22832 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:08.379247 24859 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.407513 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.408689 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 236) took 1997 ms. Trace:
W20260709 13:12:08.408779 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 141332us
I20260709 13:12:08.299283 24773 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:08.387398 24790 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.391774 24967 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:08.412303 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.388986 24861 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.379320 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:08.389997 24828 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.409166 23462 rpcz_store.cc:276] 0709 13:12:06.411841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.474851 (+ 63010us) service_pool.cc:224] Handling call
0709 13:12:08.407341 (+1932490us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:08.408606 (+  1265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3299072}
I20260709 13:12:08.426759 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.434752 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.440737 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:08.419750 24944 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.003s
W20260709 13:12:08.443655 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68415.9us
I20260709 13:12:08.419516 24790 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.447358 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:08.399483 24664 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:12:08.448375 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:08.434089 23460 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:08.451046 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:08.387610 24939 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:08.420847 24967 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.454653 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.455859 22840 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:08.448421 22913 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:08.437556 24942 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:08.450910 22761 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:12:08.443464 24835 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.458811 22761 pending_rounds.cc:85] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 1
I20260709 13:12:08.443243 24828 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:08.459342 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:08.459379 22761 pending_rounds.cc:107] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:12:08.463688 22761 raft_consensus.cc:2889] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:08.464087 22761 pending_rounds.cc:107] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:08.464347 22761 raft_consensus.cc:2889] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:08.456403 24970 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.467160 24970 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:08.393648 23461 rpcz_store.cc:276] 0709 13:12:06.396240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.640781 (+244541us) service_pool.cc:224] Handling call
0709 13:12:06.641295 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.641672 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:07.153171 (+511499us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.996413 (+843242us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b180026f9e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:08.287014 (+290601us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:08.287959 (+   945us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:08.294438 (+  6479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.300618 (+  6180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:08.311073 (+ 10455us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9948,"prepare.queue_time_us":520345,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":16687,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":6731,"spinlock_wait_cycles":79753344,"thread_start_us":701512,"threads_started":2,"wal-append.queue_time_us":1138466}
I20260709 13:12:08.467602 24970 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.467905 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.468890 22834 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:12:08.469872 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80698.6us
I20260709 13:12:08.471540 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.455420 24957 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:08.473100 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.474465 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:08.476756 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.458597 24942 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:08.458673 24964 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:08.464423 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:08.458900 24815 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:12:08.490869 24979 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:08.493104 24979 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:08.494205 24979 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:08.495455 24983 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.496111 24983 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.496582 24983 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.498111 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:08.498497 24979 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:08.475694 24944 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.503691 24980 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.482411 24861 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.506680 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:08.504297 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45086us
I20260709 13:12:08.484596 24859 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:08.575846 24977 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:12:08.577461 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.578315 24908 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:08.578888 24908 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:08.579988 24908 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:08.582186 24971 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:12:08.583395 24910 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 2.030s	user 0.010s	sys 0.017s
W20260709 13:12:08.584873 24962 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:12:08.586305 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:08.509676 24980 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:08.592418 22837 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:08.598111 22840 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:08.593571 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 442) took 1102 ms. Trace:
I20260709 13:12:08.465265 24940 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:08.606554 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94271.2us
I20260709 13:12:08.626765 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 426) took 2975 ms. Trace:
W20260709 13:12:08.572062 24968 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:12:08.611027 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:08.605494 23463 rpcz_store.cc:276] 0709 13:12:07.491351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.497620 (+  6269us) service_pool.cc:224] Handling call
0709 13:12:07.498010 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.534101 (+ 36091us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:07.552602 (+ 18501us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.784866 (+232264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.784950 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:07.785017 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.528621 (+743604us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b54000b6654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:08.593309 (+ 64688us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:08.593485 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94102,"prepare.queue_time_us":21768,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2853,"raft.queue_time_us":72971,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":28293,"spinlock_wait_cycles":82352384,"thread_start_us":159343,"threads_started":1,"wal-append.queue_time_us":301957}
I20260709 13:12:08.627452 23466 rpcz_store.cc:276] 0709 13:12:05.651271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.651428 (+   157us) service_pool.cc:224] Handling call
0709 13:12:05.651816 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:05.724722 (+ 72906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:05.724770 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:05.740961 (+ 16191us) log.cc:844] Serialized 52 byte log entry
0709 13:12:06.228566 (+487605us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:06.228616 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.228697 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:08.582657 (+2353960us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:08.589251 (+  6594us) raft_consensus.cc:1685] finished
0709 13:12:08.589277 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:08.626649 (+ 37372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2265,"raft.queue_time_us":56616,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":14753,"spinlock_wait_cycles":39199488,"thread_start_us":458518,"threads_started":1,"wal-append.queue_time_us":723004}
W20260709 13:12:08.606051 24965 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:12:08.581437 24908 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.607084 24980 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:08.674360 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 186802us
I20260709 13:12:08.589529 24981 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:08.676549 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:08.643503 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:08.641276 24861 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:08.734068 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 236101us
W20260709 13:12:08.641911 24963 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
W20260709 13:12:08.730844 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:08.735812 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59383.5us
I20260709 13:12:08.739250 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:08.735052 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.725407 24981 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:12:08.742767 22730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.519s (SENT) [suppressed 41 similar messages]
W20260709 13:12:08.743525 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.741750 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:08.771032 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1120) took 1842 ms. Trace:
I20260709 13:12:08.771556 22758 rpcz_store.cc:276] 0709 13:12:06.929039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.440092 (+511053us) service_pool.cc:224] Handling call
0709 13:12:08.006851 (+566759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:08.052951 (+ 46100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.770815 (+717864us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b540035dc50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:08.770942 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1203041664}
I20260709 13:12:08.776819 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1203) took 1226 ms. Trace:
I20260709 13:12:08.777292 22908 rpcz_store.cc:276] 0709 13:12:07.550091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.550234 (+   143us) service_pool.cc:224] Handling call
0709 13:12:08.776600 (+1226366us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540020b2d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:08.776755 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365877504}
I20260709 13:12:07.527628 22915 rpcz_store.cc:276] 0709 13:12:06.001062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.333233 (+332171us) service_pool.cc:224] Handling call
0709 13:12:06.333698 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.333987 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:06.397314 (+ 63327us) log.cc:844] Serialized 14 byte log entry
0709 13:12:07.499610 (+1102296us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b1800381ac0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a
0709 13:12:07.513675 (+ 14065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.513729 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:07.513775 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:07.513839 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:07.513941 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13828,"prepare.queue_time_us":57004,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":26008,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4937,"spinlock_wait_cycles":16444672,"thread_start_us":68885,"threads_started":1,"wal-append.queue_time_us":461664}
W20260709 13:12:08.783205 24761 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.593s	user 0.008s	sys 0.003s
I20260709 13:12:08.783710 24761 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:08.784638 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.785710 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:08.796538 22761 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:08.798998 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:08.744496 22925 tablet_copy_source_session.cc:215] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:08.808490 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:08.739012 24861 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:08.810393 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:08.809219 22910 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:08.812332 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:08.816776 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:08.817444 22914 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:08.819358 22911 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:08.627609 24942 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:08.822881 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.747431 24981 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.825356 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:08.841619 22908 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:08.845813 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 103075us
I20260709 13:12:08.800442 24987 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.822376 24942 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.862416 24521 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:08.863116 24835 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:08.863708 24835 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.864080 24835 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:08.864295 24521 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.868235 24834 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:08.868829 24834 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:08.870016 24834 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:08.876407 22759 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:08.877893 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:08.878944 22759 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:08.842340 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:08.882476 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 271538us
W20260709 13:12:08.885875 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 143837us
I20260709 13:12:08.846315 24985 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.883821 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:08.900321 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.904259 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:08.921324 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:08.884510 24987 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:12:08.925231 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 190314us
I20260709 13:12:08.857214 24986 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:12:08.894654 24761 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:08.931897 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:08.948701 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:08.923377 24800 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.005s
I20260709 13:12:08.931491 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:08.900516 24984 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:08.970674 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:08.973631 22766 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:08.990578 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:08.947403 24979 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 13:12:08.992040 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87884.2us
W20260709 13:12:08.979228 24875 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.900s	user 0.015s	sys 0.000s
I20260709 13:12:08.993739 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:09.000190 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 138591us
I20260709 13:12:08.999480 24985 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:09.010823 24993 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:09.011922 24993 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:12:09.014364 24970 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
W20260709 13:12:09.014793 24990 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:12:09.014961 24970 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.016733 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1148) took 1060 ms. Trace:
I20260709 13:12:09.017381 24970 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.018283 24970 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:09.018659 24970 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:09.017155 22760 rpcz_store.cc:276] 0709 13:12:07.956500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.586795 (+630295us) service_pool.cc:224] Handling call
0709 13:12:08.587178 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:08.587625 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:08.587655 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:08.588373 (+   718us) log.cc:844] Serialized 52 byte log entry
0709 13:12:08.734815 (+146442us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:08.757224 (+ 22409us) log.cc:844] Serialized 14 byte log entry
0709 13:12:08.894380 (+137156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.896151 (+  1771us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.016393 (+120242us) raft_consensus.cc:1685] finished
0709 13:12:09.016416 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.016655 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3014,"prepare.queue_time_us":13039,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":98450,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":30302,"spinlock_wait_cycles":9428736,"thread_start_us":275427,"threads_started":2,"wal-append.queue_time_us":258197}
I20260709 13:12:09.019517 24970 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.019873 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:09.020537 22760 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:09.021603 24939 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:09.021914 22834 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:09.027271 22839 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:09.027733 22839 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:09.028527 22839 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.029045 24964 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.029616 24964 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.030593 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:09.031741 24964 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.032831 24964 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:09.033149 24590 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.001s
I20260709 13:12:09.033375 24964 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.033716 24997 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:09.034902 24997 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:09.035532 24964 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:09.033718 24829 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.002s
I20260709 13:12:09.032896 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:09.039252 22913 raft_consensus.cc:2368] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
I20260709 13:12:08.974474 24800 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.029448 22839 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:08.997573 24761 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:08.997057 24940 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:09.046135 24979 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:09.114743 24939 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:09.126145 24915 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 2.576s	user 0.010s	sys 0.006s
I20260709 13:12:09.137455 22760 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:09.114495 24964 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.004086 24875 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.117975 24773 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.144836 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:09.141610 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 235353us
I20260709 13:12:09.147396 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:09.148296 22760 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:09.153836 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:09.124766 24985 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:09.152843 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:09.046341 24590 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:09.138200 24790 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.000s
W20260709 13:12:09.160265 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 228913us
W20260709 13:12:09.141772 24828 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.004s
I20260709 13:12:09.170359 24987 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:09.139858 24861 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.165442 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:09.175825 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.178807 22839 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:12:09.183151 22912 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:12:09.199276 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:09.200699 22839 pending_rounds.cc:85] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:12:09.201095 22839 pending_rounds.cc:107] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:12:09.201465 22839 raft_consensus.cc:2889] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:09.200930 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:09.203629 22839 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:09.164490 24940 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:09.124927 24761 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:09.209079 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:09.163476 25005 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:09.212702 22910 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:09.216352 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:09.167562 24790 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.215183 25005 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.218205 25005 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:09.147234 24939 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.222707 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:09.223613 22908 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:09.222714 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:09.227989 24790 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:09.231166 24980 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.003s
I20260709 13:12:09.231630 24980 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.212493 24986 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:12:09.234875 25011 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:09.235347 25011 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.235698 25011 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:09.238314 22910 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:09.205487 24998 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:09.159991 24964 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.157454 24773 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.176755 24979 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.214212 24940 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.175369 24828 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:09.252461 24908 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.000s
I20260709 13:12:09.163803 24985 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.266100 22759 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:09.276078 24980 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:09.275705 25009 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:09.281646 24940 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:09.341987 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.307358 24800 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.357242 24832 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.354708 25009 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.296080 25001 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.388051 24831 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:12:09.388808 25004 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:12:09.389647 24831 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:09.391600 25008 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:12:09.390988 24831 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.296036 25013 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:09.392481 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:09.395632 25006 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:12:09.284324 24979 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.383975 25009 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:09.503978 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.570917 25015 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:12:09.435091 24875 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.436036 25013 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:09.599589 22759 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:12:09.603827 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.611405 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.612149 22759 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:09.602869 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 437540us
I20260709 13:12:09.603824 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:09.615525 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:09.615963 22759 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:09.562500 24590 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:09.305976 24987 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.657460 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:09.661337 22806 leader_election.cc:400] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5.
I20260709 13:12:09.394586 25001 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.663517 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1144) took 1710 ms. Trace:
I20260709 13:12:09.661741 22806 leader_election.cc:403] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:12:09.672379 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.657971 24980 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:12:09.709285 25013 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.708381 24979 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:09.665899 24967 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.202s	user 0.012s	sys 0.007s
I20260709 13:12:09.658535 24761 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.660316 24831 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:09.693941 25020 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:12:09.721913 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.727478 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.662863 24756 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:09.703202 22763 rpcz_store.cc:276] 0709 13:12:07.953140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.991496 (+ 38356us) service_pool.cc:224] Handling call
0709 13:12:08.051400 (+ 59904us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:08.059149 (+  7749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:08.059201 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:08.087192 (+ 27991us) log.cc:844] Serialized 52 byte log entry
0709 13:12:08.302970 (+215778us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:08.400816 (+ 97846us) log.cc:844] Serialized 14 byte log entry
0709 13:12:08.412658 (+ 11842us) log.cc:844] Serialized 14 byte log entry
0709 13:12:08.412858 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.412947 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.024913 (+611966us) spinlock_profiling.cc:243] Waited 7.95 ms on lock 0x7b48002ac0d8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:12:09.387958 (+363045us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b1800394f00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:09.663211 (+275253us) raft_consensus.cc:1685] finished
0709 13:12:09.663245 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.663440 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10234,"prepare.queue_time_us":213058,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":2109,"raft.queue_time_us":22027,"raft.run_cpu_time_us":8102,"raft.run_wall_time_us":970633,"spinlock_wait_cycles":32010624,"thread_start_us":202165,"threads_started":2,"wal-append.queue_time_us":278881}
W20260709 13:12:09.664216 24835 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.800s	user 0.018s	sys 0.014s
I20260709 13:12:09.703101 24939 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.704413 24684 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.711223 24980 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:12:09.737972 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1152) took 1776 ms. Trace:
W20260709 13:12:09.746790 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 85601.9us
I20260709 13:12:09.717932 24964 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.748449 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:09.738415 22761 rpcz_store.cc:276] 0709 13:12:07.961702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.805603 (+843901us) service_pool.cc:224] Handling call
0709 13:12:08.806205 (+   602us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:08.884810 (+ 78605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:08.884878 (+    68us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:08.901162 (+ 16284us) log.cc:844] Serialized 52 byte log entry
0709 13:12:08.995499 (+ 94337us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:09.046950 (+ 51451us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.047197 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.047280 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.737703 (+690423us) raft_consensus.cc:1685] finished
0709 13:12:09.737731 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.737903 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31064,"prepare.queue_time_us":51095,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":117722,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":97992,"thread_start_us":154035,"threads_started":2,"wal-append.queue_time_us":222968}
I20260709 13:12:09.756089 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1131) took 4027 ms. Trace:
I20260709 13:12:09.757225 22833 rpcz_store.cc:276] 0709 13:12:05.728676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.532911 (+1804235us) service_pool.cc:224] Handling call
0709 13:12:07.543142 (+ 10231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.543455 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:07.552999 (+  9544us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.693089 (+2140090us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b1800334ae0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:09.755721 (+ 62632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:09.755759 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:09.755793 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.755877 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.756017 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9335,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"spinlock_wait_cycles":8411904,"thread_start_us":61996,"threads_started":1}
I20260709 13:12:09.730405 24756 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:09.759593 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1150) took 1802 ms. Trace:
I20260709 13:12:09.733423 24908 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:09.756775 24773 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.733917 25027 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5.
I20260709 13:12:09.747696 24828 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.729087 24967 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.764249 22758 rpcz_store.cc:276] 0709 13:12:07.957250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.772898 (+815648us) service_pool.cc:224] Handling call
0709 13:12:08.773351 (+   453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:08.860525 (+ 87174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:08.860561 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:08.920839 (+ 60278us) log.cc:844] Serialized 52 byte log entry
0709 13:12:09.060066 (+139227us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:09.125667 (+ 65601us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.125896 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.125974 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.759366 (+633392us) raft_consensus.cc:1685] finished
0709 13:12:09.759388 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.759525 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82054,"prepare.queue_time_us":71320,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":10258,"raft.queue_time_us":78798,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":19230,"spinlock_wait_cycles":3000192,"thread_start_us":135149,"threads_started":2,"wal-append.queue_time_us":208363}
I20260709 13:12:09.766593 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:09.757009 24947 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:09.758271 24684 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.765074 24761 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:09.784197 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:09.787456 24998 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:09.794349 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:09.766554 22805 connection.cc:256] RPC call timeout handler was delayed by -1.3197379069999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3118s for a response
I20260709 13:12:09.737320 24835 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.800906 22833 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:09.764619 24991 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:09.814013 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1155) took 1351 ms. Trace:
I20260709 13:12:09.814170 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1150) took 1457 ms. Trace:
I20260709 13:12:09.814570 22839 rpcz_store.cc:276] 0709 13:12:08.462770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.204635 (+741865us) service_pool.cc:224] Handling call
0709 13:12:09.215635 (+ 11000us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.241640 (+ 26005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.241674 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.259212 (+ 17538us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.561480 (+302268us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:09.561562 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.561673 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.813747 (+252074us) raft_consensus.cc:1685] finished
0709 13:12:09.813770 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.813945 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21293,"raft.queue_time_us":101623,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"thread_start_us":268729,"threads_started":1,"wal-append.queue_time_us":513561}
W20260709 13:12:09.817435 25012 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.390s	user 0.000s	sys 0.000s
I20260709 13:12:09.818823 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1153) took 1442 ms. Trace:
I20260709 13:12:09.791165 24918 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.819182 22841 rpcz_store.cc:276] 0709 13:12:08.376803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.981333 (+604530us) service_pool.cc:224] Handling call
0709 13:12:09.004700 (+ 23367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.004960 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.004985 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.005581 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.165369 (+159788us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:09.165426 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.179706 (+ 14280us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.818581 (+638875us) raft_consensus.cc:1685] finished
0709 13:12:09.818605 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.818761 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":151381,"threads_started":1,"wal-append.queue_time_us":245271}
I20260709 13:12:09.820240 24835 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.819944 24918 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.814656 22832 rpcz_store.cc:276] 0709 13:12:08.356430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.606474 (+250044us) service_pool.cc:224] Handling call
0709 13:12:08.723962 (+117488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.216803 (+492841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.220827 (+  4024us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.278882 (+ 58055us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.740010 (+461128us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:09.756612 (+ 16602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.762108 (+  5496us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.813946 (+ 51838us) raft_consensus.cc:1685] finished
0709 13:12:09.813972 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.814112 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23996,"raft.queue_time_us":235904,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5801,"thread_start_us":776075,"threads_started":2,"wal-append.queue_time_us":485780}
I20260709 13:12:09.821026 22760 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:09.789515 24908 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.765847 24756 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:09.795902 25022 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:09.830502 22758 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:12:09.832551 22805 connection.cc:256] RPC call timeout handler was delayed by -1.1546693039999809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 13:12:09.832544 25032 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:09.842821 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:09.844686 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:09.810925 24998 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:09.858958 24967 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.859628 25024 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:09.860714 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:09.871109 22758 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:09.801630 24985 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.794736 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1136) took 1485 ms. Trace:
I20260709 13:12:09.866901 22832 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:09.847693 24980 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:09.883090 22833 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:09.886144 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:09.866372 25032 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:09.900424 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:09.901773 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:09.794690 25019 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:12:09.837334 25009 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.885847 24967 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:09.908862 24987 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.916865 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:09.917768 22758 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:09.922293 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:09.923367 22758 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:09.923889 25043 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:09.924237 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:09.924841 22758 raft_consensus.cc:2376] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4. Re-sending same reply.
I20260709 13:12:09.911111 25027 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:09.938267 24983 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.441s	user 0.015s	sys 0.003s
I20260709 13:12:09.938882 24983 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.941783 22837 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:09.938194 24918 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:09.920742 24773 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.941210 24983 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.036657 22833 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:12:09.973191 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:09.982772 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:09.940943 24981 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.081s	user 0.005s	sys 0.008s
I20260709 13:12:10.052888 22758 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4.
I20260709 13:12:10.052083 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1156) took 1581 ms. Trace:
I20260709 13:12:10.054235 22839 rpcz_store.cc:276] 0709 13:12:08.470943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.816499 (+1345556us) service_pool.cc:224] Handling call
0709 13:12:09.816888 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.838345 (+ 21457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.838386 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.856898 (+ 18512us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.908720 (+ 51822us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:09.908766 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.908837 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.051844 (+143007us) raft_consensus.cc:1685] finished
0709 13:12:10.051866 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.052020 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29557,"raft.queue_time_us":20771,"raft.run_cpu_time_us":894,"raft.run_wall_time_us":58524,"thread_start_us":29166,"threads_started":1,"wal-append.queue_time_us":102487}
I20260709 13:12:09.924609 25043 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:09.995662 24942 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.135s	user 0.002s	sys 0.012s
W20260709 13:12:10.010614 25005 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.003s
W20260709 13:12:10.001809 25023 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 13:12:09.981878 25026 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:12:09.969877 25001 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:09.943655 25009 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:10.056985 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:09.986893 24940 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.003s
I20260709 13:12:10.072254 24940 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.074864 24940 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:10.062757 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:10.043188 24882 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:10.002440 24684 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.009279 24908 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
W20260709 13:12:10.067301 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1908) took 3970 ms (client timeout 2579 ms). Trace:
I20260709 13:12:09.928846 24967 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.925846 24987 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:09.994510 25024 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:10.016136 25011 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.005s
I20260709 13:12:10.060197 24918 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:12:10.058202 24981 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.078926 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1152) took 1711 ms. Trace:
I20260709 13:12:10.133059 22834 rpcz_store.cc:276] 0709 13:12:08.367489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.741746 (+374257us) service_pool.cc:224] Handling call
0709 13:12:08.822570 (+ 80824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.022676 (+200106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.078646 (+1055970us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:10.078855 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457864576}
I20260709 13:12:10.077654 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:09.993284 25025 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.173s
I20260709 13:12:10.090830 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:10.115312 22764 rpcz_store.cc:269] 0709 13:12:06.226028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.226164 (+   136us) service_pool.cc:224] Handling call
0709 13:12:10.067040 (+3840876us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002dbed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:10.067202 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3942386944}
I20260709 13:12:10.068783 25022 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:10.125209 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:10.054800 24773 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:09.969404 25027 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.089175 24882 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:10.069357 25005 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.127560 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:10.135566 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.089391 24982 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.152006 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.152477 22758 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:10.153060 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:10.155211 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:10.155618 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.155488 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:10.115196 25009 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.091485 24942 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.143880 24908 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:10.174008 24908 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:10.174023 24942 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.174685 22914 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:12:10.175079 24908 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.175971 23459 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.176631 24908 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.145432 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.146015 25043 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.146080 25025 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:10.128203 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:10.150987 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60214us
W20260709 13:12:10.158213 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.177564 24942 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:10.158443 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.158887 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:10.161828 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:10.162385 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.167347 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1167) took 1885 ms. Trace:
I20260709 13:12:10.168754 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.170462 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:10.165251 25040 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.172660 25005 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.131911 25011 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.179559 22806 leader_election.cc:400] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4.
I20260709 13:12:10.162307 24982 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:10.203323 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.236986 22806 leader_election.cc:403] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:10.250651 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:10.184629 22908 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:10.184818 22836 rpcz_store.cc:276] 0709 13:12:08.285333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.285514 (+   181us) service_pool.cc:224] Handling call
0709 13:12:10.151803 (+1866289us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:10.167252 (+ 15449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2418560}
W20260709 13:12:10.238031 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 150295us
I20260709 13:12:10.252390 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.253077 22836 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:10.253989 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.254585 22836 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
W20260709 13:12:10.183845 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:10.257475 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.177956 25043 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.242547 24459 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:10.243535 25011 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.245265 23458 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:12:10.272419 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92976.7us
I20260709 13:12:10.127748 25024 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.247008 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 199) took 1465 ms. Trace:
I20260709 13:12:10.274245 22913 rpcz_store.cc:276] 0709 13:12:08.781465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.785370 (+  3905us) service_pool.cc:224] Handling call
0709 13:12:10.246713 (+1461343us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:10.246920 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581317888}
W20260709 13:12:10.275295 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.246446 24882 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:10.180014 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.177850 22764 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:10.182025 22907 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:10.186899 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.125697 25001 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:12:10.259598 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:10.280908 22836 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.166189 24590 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:10.184020 25022 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:10.269264 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:10.279191 22909 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:12:09.876035 22835 rpcz_store.cc:276] 0709 13:12:08.305215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.310045 (+  4830us) service_pool.cc:224] Handling call
0709 13:12:09.037390 (+727345us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:09.037646 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.037892 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:09.038210 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.751392 (+713182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:09.756270 (+  4878us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:09.758709 (+  2439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.765863 (+  7154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.790626 (+ 24763us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5623,"prepare.queue_time_us":614,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":124825,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":12020864,"thread_start_us":674815,"threads_started":2,"wal-append.queue_time_us":255666}
I20260709 13:12:10.279471 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:10.276975 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.180047 23457 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Advancing to term 4
I20260709 13:12:10.178380 23464 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.178124 25025 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:10.177781 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.286852 22758 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.287688 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.291450 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:10.292173 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.323386 22909 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:10.295133 23465 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:10.318482 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:10.271174 24459 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.274541 24964 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.269004 24987 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.323649 22908 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:10.181434 24756 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.326357 23463 raft_consensus.cc:2410] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:12:10.288353 25022 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:09.944061 25032 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.288146 24590 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.333976 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.334323 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:10.335788 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:10.257763 25013 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:12:10.294422 24882 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.324725 24790 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:10.347143 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.312398 23459 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:10.355031 22836 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:10.356555 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.380038 22908 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:10.357991 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.380724 22909 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:10.339161 25022 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.271883 25043 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.324976 24874 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.280359 25044 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:10.348924 24831 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:10.337275 25049 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.384698 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.392431 22910 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:10.388962 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:10.377767 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:10.393059 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.393653 23463 raft_consensus.cc:2410] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:12:10.393826 22764 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:10.372289 25001 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.375984 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:10.356786 24590 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:10.391145 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 99844.3us
I20260709 13:12:10.337478 24964 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.332849 24998 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.443686 23466 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:10.460412 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:10.428565 23465 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:12:10.451402 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.396518 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.424615 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.464334 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.483637 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:10.503226 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:10.424113 24918 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:10.395964 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.506910 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:10.506844 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:10.460965 25045 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:12:10.514869 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:10.496440 22758 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:10.513881 24908 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:12:10.512706 25050 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:12:10.390296 24939 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:10.520049 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:10.526028 22730 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.736s (SENT)
I20260709 13:12:10.394586 24981 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.397259 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:10.527531 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.530172 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:10.526104 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:10.532939 22764 raft_consensus.cc:2368] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5.
I20260709 13:12:10.533366 22758 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:10.535478 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.537062 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:10.538146 22836 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:10.540514 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:10.541719 22760 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:10.543790 22760 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:10.455222 24967 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:10.505421 23461 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:10.519459 24918 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:10.557055 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.558061 22908 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.394860 24874 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:10.562125 22764 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:10.547510 22758 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:10.481673 24726 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:12:10.526710 24773 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.554345 22760 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:10.520627 25019 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:12:10.397893 24831 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.548259 24756 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.427520 25025 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.535080 24942 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:10.549116 22835 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.424491 25049 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.525103 25001 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.576545 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:10.577288 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.577672 22760 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:10.548188 24982 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:10.553198 25058 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:10.573324 22906 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:10.585058 22760 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:10.586228 24927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad66b5c9842437d9320507b92b78fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:10.586727 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.593758 22911 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:10.588768 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73993.1us
I20260709 13:12:10.592723 22909 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.594594 22760 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:12:10.595880 23432 leader_election.cc:400] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
I20260709 13:12:10.597918 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.599491 25043 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:10.600435 24831 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:10.601343 25054 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:12:10.602073 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:10.604054 25001 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:10.394740 25044 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.623828 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:10.625026 22911 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:10.606868 22908 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:10.607265 23432 leader_election.cc:403] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:10.628366 24998 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:10.612253 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 72049.7us
I20260709 13:12:10.582150 24828 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:10.616411 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1110) took 4131 ms. Trace:
I20260709 13:12:10.580433 24942 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:10.633896 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:10.634323 22757 rpcz_store.cc:276] 0709 13:12:06.473692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.051190 (+577498us) service_pool.cc:224] Handling call
0709 13:12:07.086014 (+ 34824us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:07.086994 (+   980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.087038 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:08.865730 (+1778692us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:08.866516 (+   786us) log.cc:844] Serialized 52 byte log entry
0709 13:12:08.965655 (+ 99139us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:08.989611 (+ 23956us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.120186 (+130575us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.122457 (+  2271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.122539 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.531512 (+1408973us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b180007fd80. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:10.605119 (+ 73607us) raft_consensus.cc:1685] finished
0709 13:12:10.605142 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.605293 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107158,"prepare.queue_time_us":114760,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":37874,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":7391,"spinlock_wait_cycles":129301632,"thread_start_us":98646,"threads_started":1,"wal-append.queue_time_us":274505}
I20260709 13:12:10.640379 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:10.641963 22914 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.587062 25053 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:12:10.646486 22833 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:10.641183 24942 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.683290 25053 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:10.683811 25053 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:10.647704 25043 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:10.687183 22833 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:10.648149 25040 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:10.693908 22833 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:10.694283 22833 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:10.697808 22838 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
W20260709 13:12:10.698308 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 58024us
W20260709 13:12:10.700908 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.700587 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:10.688918 25043 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:10.699604 22833 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.693106 24970 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.704018 25043 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.708701 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1143) took 2363 ms. Trace:
I20260709 13:12:10.708770 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:10.709444 22914 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:10.679594 25055 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:10.647786 25062 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
I20260709 13:12:10.711004 22840 rpcz_store.cc:276] 0709 13:12:08.345022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.464059 (+119037us) service_pool.cc:224] Handling call
0709 13:12:08.597751 (+133692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:08.598705 (+   954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.708503 (+2109798us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540025ec50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:10.708640 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299825536}
I20260709 13:12:10.696892 23464 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:10.714140 25027 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.684728 25053 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.714921 22840 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:10.715299 25053 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.699204 24964 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:10.717362 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81441.1us
I20260709 13:12:10.704998 24828 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:10.642289 24918 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.720477 22805 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:10.694483 25044 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.722340 25001 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:10.726380 25040 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:10.726946 25040 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.727257 25040 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.723496 25027 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:12:10.727807 22909 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:10.726434 25049 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.730403 25001 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:10.725733 24874 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:10.731426 25001 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.721580 24834 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.748361 23457 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEARNER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:10.721516 25032 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:12:10.764012 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1164) took 1758 ms. Trace:
I20260709 13:12:10.764464 22763 rpcz_store.cc:276] 0709 13:12:09.005966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.750995 (+745029us) service_pool.cc:224] Handling call
0709 13:12:09.753056 (+  2061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.753285 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:09.756404 (+  3119us) log.cc:844] Serialized 14 byte log entry
0709 13:12:10.763635 (+1007231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:10.763691 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:10.763744 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.763814 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.763949 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":897411,"prepare.queue_time_us":195,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":897673,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":109320,"threads_started":1,"wal-append.queue_time_us":132926}
I20260709 13:12:10.764417 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1165) took 1727 ms. Trace:
I20260709 13:12:10.766804 22761 rpcz_store.cc:276] 0709 13:12:09.037377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.756806 (+719429us) service_pool.cc:224] Handling call
0709 13:12:09.757492 (+   686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.757862 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:09.877749 (+119887us) log.cc:844] Serialized 14 byte log entry
0709 13:12:10.764072 (+886323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:10.764125 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:10.764171 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.764237 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.764353 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":885996,"prepare.queue_time_us":13533,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":886324,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":59,"thread_start_us":114916,"threads_started":1}
I20260709 13:12:10.771051 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:12:10.774312 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 248306us
W20260709 13:12:10.775032 22730 connection.cc:256] RPC call timeout handler was delayed by -1.7793797010000389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 13:12:10.775401 22730 connection.cc:256] RPC call timeout handler was delayed by -1.7567055790000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:12:10.775921 22730 connection.cc:256] RPC call timeout handler was delayed by -1.7531259610000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 13:12:10.778306 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:10.779662 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 200) took 1997 ms. Trace:
I20260709 13:12:10.780103 22912 rpcz_store.cc:276] 0709 13:12:08.781762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.785591 (+  3829us) service_pool.cc:224] Handling call
0709 13:12:08.785905 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.200176 (+414271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.779356 (+1579180us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b540020a3d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:10.779592 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912753536}
I20260709 13:12:10.793321 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:10.795768 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.796907 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:10.796509 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:10.745676 24942 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.775729 24859 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.739997 25055 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:12:10.779491 24984 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.773602 24982 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 13:12:10.792114 25009 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.007s
I20260709 13:12:10.790609 24684 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:10.787366 24985 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:10.840456 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45983.4us
I20260709 13:12:10.789678 25067 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:10.799863 24874 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:10.809823 24861 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:10.815514 24824 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:12:10.856010 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79506.2us
I20260709 13:12:10.791903 24835 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
W20260709 13:12:10.874588 22730 connection.cc:256] RPC call timeout handler was delayed by -1.5614259610000545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
I20260709 13:12:10.845813 24843 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:10.846740 25009 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:10.851733 25067 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:12:10.879596 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.833441 24859 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:10.856132 24861 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:10.879374 22730 connection.cc:256] RPC call timeout handler was delayed by -1.5425387260000889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:12:10.872067 24824 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:10.936546 22730 connection.cc:256] RPC call timeout handler was delayed by -1.4205151400000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:12:10.938014 22730 connection.cc:256] RPC call timeout handler was delayed by -1.4205151400000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 13:12:10.938581 22730 connection.cc:256] RPC call timeout handler was delayed by -1.3881151400000817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:12:10.938983 22730 connection.cc:256] RPC call timeout handler was delayed by -1.3254617059999418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
W20260709 13:12:10.939589 22730 connection.cc:256] RPC call timeout handler was delayed by -1.0849740720000227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
I20260709 13:12:10.846746 24984 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:10.852941 24985 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:10.956935 22730 connection.cc:256] RPC call timeout handler was delayed by -1.0233797010000671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2886s for a response
I20260709 13:12:10.948206 25067 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.893749 25032 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.007977 25032 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:11.009608 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 207613us
W20260709 13:12:10.880628 25037 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.698s	user 0.000s	sys 0.002s
W20260709 13:12:11.010614 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.011305 25032 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:11.011653 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.894061 24800 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.971088 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1149) took 3014 ms. Trace:
I20260709 13:12:10.969116 22834 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:10.944170 24824 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:11.001556 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:10.937492 24861 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.005815 24835 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.953673 22915 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:10.887995 24882 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.003s
I20260709 13:12:11.012321 22762 rpcz_store.cc:276] 0709 13:12:07.956985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.588634 (+631649us) service_pool.cc:224] Handling call
0709 13:12:08.588995 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:08.634666 (+ 45671us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:12:08.680372 (+ 45706us) log.cc:844] Serialized 14 byte log entry
0709 13:12:10.061345 (+1380973us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b18001c1860. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:10.765475 (+704130us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:10.774131 (+  8656us) log.cc:844] Serialized 26 byte log entry
0709 13:12:10.774295 (+   164us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:10.774333 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.774415 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.951747 (+177332us) raft_consensus.cc:1685] finished
0709 13:12:10.970762 (+ 19015us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.970996 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":635396,"prepare.queue_time_us":99062,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":1524,"raft.queue_time_us":674570,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":5245,"spinlock_wait_cycles":34244992,"thread_start_us":68420,"threads_started":1,"wal-append.queue_time_us":1394985}
W20260709 13:12:11.027030 25071 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:12:11.031062 24872 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:11.033843 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1155) took 2245 ms. Trace:
I20260709 13:12:11.034399 22766 rpcz_store.cc:276] 0709 13:12:08.788284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.991941 (+203657us) service_pool.cc:224] Handling call
0709 13:12:09.009322 (+ 17381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:09.118420 (+109098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.118466 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:09.135275 (+ 16809us) log.cc:844] Serialized 52 byte log entry
0709 13:12:09.277727 (+142452us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:09.299745 (+ 22018us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.300087 (+   342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.300183 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.248819 (+948636us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b1800394c60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1
0709 13:12:10.622150 (+373331us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800394c60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:10.750325 (+128175us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800394c60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f68a59c558f 00007f689ea04571 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd
0709 13:12:11.026562 (+276237us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b1800394c60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:11.033536 (+  6974us) raft_consensus.cc:1685] finished
0709 13:12:11.033567 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:11.033762 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40864,"prepare.queue_time_us":27874,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":63437,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":23532,"spinlock_wait_cycles":179294592,"thread_start_us":124460,"threads_started":1,"wal-append.queue_time_us":213213}
I20260709 13:12:11.038308 24942 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
W20260709 13:12:11.038960 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.029076 24664 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:11.040640 24664 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:11.041870 24664 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.042573 25058 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:11.045275 25025 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.972541 24684 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:11.048146 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.004289 25049 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.029183 24859 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:10.898339 24590 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.013576 25005 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4.
I20260709 13:12:11.015308 24800 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.056411 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.057673 24967 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.059366 24991 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.064287 22912 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:10.894347 24843 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.053197 24684 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:11.084002 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.085913 25040 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.124255 22913 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:11.024501 24835 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.127499 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1160) took 2123 ms. Trace:
W20260709 13:12:11.128347 24459 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.792s	user 0.012s	sys 0.000s
I20260709 13:12:11.128499 24987 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:11.125226 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.133467 22765 rpcz_store.cc:276] 0709 13:12:09.004499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.700359 (+695860us) service_pool.cc:224] Handling call
0709 13:12:09.700757 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:10.754418 (+1053661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.754453 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:10.754970 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:12:11.009834 (+254864us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:11.009922 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.010132 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:11.127199 (+117067us) raft_consensus.cc:1685] finished
0709 13:12:11.127226 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:11.127426 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1053204,"raft.queue_time_us":1057123,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"thread_start_us":211918,"threads_started":1,"wal-append.queue_time_us":282400}
I20260709 13:12:11.051024 24882 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.149436 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.131357 25009 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:11.150516 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.150592 25009 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:11.151533 25009 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.151758 22806 leader_election.cc:400] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5.
I20260709 13:12:11.065531 25049 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.140763 24459 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.144326 25040 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:11.119014 24991 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.146692 24684 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:11.147571 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.177269 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.148409 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.182122 22838 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:11.139822 24987 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:11.152140 22806 leader_election.cc:403] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:11.090416 25032 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.157929 25017 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:11.190366 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.149210 24882 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:11.184590 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.148645 24985 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:11.184579 22835 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.580s	user 0.004s	sys 0.008s
I20260709 13:12:11.215754 22839 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:11.216629 24993 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:11.223232 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71603.5us
I20260709 13:12:11.187983 25017 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:11.225344 24967 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:12:11.233692 24987 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.216310 24684 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:11.199837 25032 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5.
I20260709 13:12:11.219774 24459 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.214828 25049 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.246554 24843 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.163041 25067 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.221297 24993 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:11.260877 25022 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.000s
I20260709 13:12:11.264132 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:11.261394 25022 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:11.254199 24834 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 13:12:11.268033 24940 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.268723 24940 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.270076 24590 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.217026 22835 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:11.271040 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1165) took 1151 ms. Trace:
I20260709 13:12:11.255964 25009 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:12:11.271517 22835 rpcz_store.cc:276] 0709 13:12:10.119575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.346238 (+226663us) service_pool.cc:224] Handling call
0709 13:12:11.270963 (+924725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:11.272716 24824 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.273471 24835 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.283386 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.284251 22836 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:11.274039 24800 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.276684 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.237135 24970 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:12:11.287582 22838 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:11.272754 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.300348 25022 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.253062 25017 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.315455 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 224) took 1445 ms. Trace:
I20260709 13:12:11.320966 22832 rpcz_store.cc:276] 0709 13:12:09.870628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.893763 (+ 23135us) service_pool.cc:224] Handling call
0709 13:12:09.894102 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:10.139875 (+245773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.139905 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:10.139946 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:10.139987 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.315115 (+1175128us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b540013abd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:11.315175 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:11.315368 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564764544}
I20260709 13:12:11.279378 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:11.287252 24984 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:11.341418 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.352901 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 447) took 1044 ms. Trace:
I20260709 13:12:11.335458 24835 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.353524 23460 rpcz_store.cc:276] 0709 13:12:10.308795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.329852 (+ 21057us) service_pool.cc:224] Handling call
0709 13:12:11.279181 (+949329us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:11.352784 (+ 73603us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7567616}
I20260709 13:12:11.361967 22805 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.337617 24843 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.381719 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 444) took 1102 ms. Trace:
I20260709 13:12:11.382213 23466 rpcz_store.cc:276] 0709 13:12:10.279126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.279251 (+   125us) service_pool.cc:224] Handling call
0709 13:12:10.279551 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:10.512622 (+233071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.381383 (+868761us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54002c7754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:11.381648 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343939456}
I20260709 13:12:11.382705 25044 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:11.383306 25044 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:11.385097 25070 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.425s	user 0.000s	sys 0.005s
I20260709 13:12:11.387570 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1172) took 1149 ms. Trace:
I20260709 13:12:11.370360 24834 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.390026 24834 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:11.390982 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2013) took 1004 ms. Trace:
I20260709 13:12:11.391270 25067 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.391387 22913 rpcz_store.cc:276] 0709 13:12:10.386253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.687172 (+300919us) service_pool.cc:224] Handling call
0709 13:12:10.698117 (+ 10945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.124886 (+426769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.390479 (+265593us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000438d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:11.390906 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979421952}
I20260709 13:12:11.392541 24834 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.360519 24835 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.393318 24882 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.394508 25022 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.395054 25022 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.395260 24835 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:11.397725 22834 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1162) took 1307 ms (client timeout 1620 ms). Trace:
W20260709 13:12:11.398408 22834 rpcz_store.cc:269] 0709 13:12:10.090334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.142359 (+ 52025us) service_pool.cc:224] Handling call
0709 13:12:11.397470 (+1255111us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54000c71d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:11.397643 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725163904}
W20260709 13:12:11.405172 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.388249 22840 rpcz_store.cc:276] 0709 13:12:10.237894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.715711 (+477817us) service_pool.cc:224] Handling call
0709 13:12:10.716746 (+  1035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:10.717647 (+   901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.717678 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:10.722032 (+  4354us) log.cc:844] Serialized 26 byte log entry
0709 13:12:10.781103 (+ 59071us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:10.781129 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.781179 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:11.286958 (+505779us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:11.387120 (+100162us) raft_consensus.cc:1685] finished
0709 13:12:11.387152 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:11.387484 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3715,"spinlock_wait_cycles":6056704,"thread_start_us":40862,"threads_started":1,"wal-append.queue_time_us":138025}
I20260709 13:12:11.425887 24843 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.442323 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.443967 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:11.394646 24824 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.445123 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:11.445840 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.446986 24843 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:11.447850 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:11.378840 24790 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:11.469376 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.482236 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.487362 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:11.490516 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.487649 24790 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.494200 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1151) took 3137 ms. Trace:
I20260709 13:12:11.494808 22837 rpcz_store.cc:276] 0709 13:12:08.356776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.600616 (+243840us) service_pool.cc:224] Handling call
0709 13:12:08.601126 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.942194 (+1341068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.493899 (+1551705us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540027ea54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:11.494128 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3082692864}
I20260709 13:12:11.488746 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.397609 25005 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.499975 22910 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:12:11.491436 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.425643 24835 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.503571 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.504506 22913 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:11.503796 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.505489 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:11.506163 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.503444 24983 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.489578 24947 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.512156 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.489849 24824 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:11.517596 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.523658 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:11.526404 25005 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.531973 24824 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.534540 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:11.535197 23466 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:11.535534 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.021s	sys 0.044s
W20260709 13:12:11.536252 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.021s	sys 0.044s
I20260709 13:12:11.536634 22807 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.540454 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:11.542147 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:11.542239 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.015s	sys 0.007s
W20260709 13:12:11.545001 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.015s	sys 0.007s
W20260709 13:12:11.547868 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 532048us
I20260709 13:12:11.550148 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2010) took 1166 ms. Trace:
I20260709 13:12:11.551437 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:11.552826 22909 rpcz_store.cc:276] 0709 13:12:10.384184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.384341 (+   157us) service_pool.cc:224] Handling call
0709 13:12:11.549930 (+1165589us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003ca0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:11.550074 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459673984}
I20260709 13:12:11.516726 24947 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.555920 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:11.555143 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:11.558660 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.561122 22912 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:11.562155 24843 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.566116 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.563143 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.570926 22766 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:11.571810 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.559511 22757 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:11.574225 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:11.570089 22760 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:11.578951 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.579735 22760 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:11.580715 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.569377 25005 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:11.592566 25005 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.593040 25005 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.573475 24800 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:11.517360 24835 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.567485 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.568003 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:11.533949 25040 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.554975 25042 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:12:11.614615 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47232.1us
I20260709 13:12:11.600831 22760 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:12:11.510596 24861 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:11.561050 25043 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.004s
I20260709 13:12:11.609958 23466 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:11.564457 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:11.552260 24459 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:11.612946 22763 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:11.635213 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 70862.5us
I20260709 13:12:11.615715 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.611255 24800 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:11.616693 24875 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:11.617412 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.645366 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:11.605388 25022 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.668560 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.644260 24875 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.674867 24875 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:11.646386 22760 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:11.648202 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.634122 24459 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.678170 24459 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:11.678790 24790 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.679267 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.679971 22760 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:11.680954 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.682003 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:11.683360 22760 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:12:11.683175 25005 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:11.684032 24835 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.684985 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:11.686111 22766 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:11.687126 24861 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.618800 25042 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.690809 25042 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.643306 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.648728 24967 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:11.651006 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.651744 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.699131 22840 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:11.698987 22837 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:11.652576 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.654385 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.655958 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:11.702559 22835 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:11.659936 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:11.703258 24800 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.704217 22841 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:11.644752 25009 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:11.707193 25009 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:11.662528 25040 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:12:11.711199 25040 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:11.711697 25040 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:11.611364 23464 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:11.672145 25049 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:11.714461 25049 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.670925 24808 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.714946 25049 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:11.717618 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74419.8us
I20260709 13:12:11.718843 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.647804 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.627976 24983 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.712803 25040 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.722255 24882 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.723505 24983 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.735117 22833 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:11.693491 25025 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.648s	user 0.014s	sys 0.000s
I20260709 13:12:11.726749 24808 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:11.727098 24684 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.739208 24987 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:11.724269 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76569us
I20260709 13:12:11.742123 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:11.708705 25009 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.747584 25001 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:11.750027 24590 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:12:11.750660 24927 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.006s
I20260709 13:12:11.754292 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 203) took 1597 ms. Trace:
I20260709 13:12:11.754570 24808 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.754788 22906 rpcz_store.cc:276] 0709 13:12:10.157101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.163126 (+  6025us) service_pool.cc:224] Handling call
0709 13:12:11.754042 (+1590916us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54000cfb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:11.754216 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446602496}
I20260709 13:12:11.756474 24824 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.758837 25044 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.760053 25044 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:11.762531 22914 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:11.630954 24991 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.698084 22763 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:11.640762 25043 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.771740 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1159) took 1687 ms. Trace:
I20260709 13:12:11.775084 22839 rpcz_store.cc:276] 0709 13:12:10.083883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.083998 (+   115us) service_pool.cc:224] Handling call
0709 13:12:10.084317 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.216367 (+1132050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.771418 (+555051us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400038ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:11.771658 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600908544}
I20260709 13:12:11.780557 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:11.781483 22910 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:11.721024 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:11.799108 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78194.9us
I20260709 13:12:11.800746 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.802682 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.803834 24983 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:11.804428 24824 raft_consensus.cc:2764] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:12:11.770648 24940 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.810218 25001 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:11.820719 22833 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:12:11.823395 25007 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:12:11.828195 24875 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.837934 23460 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:11.840489 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52072.1us
I20260709 13:12:11.844784 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.876734 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:11.880615 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1166) took 1755 ms. Trace:
I20260709 13:12:11.881230 22833 rpcz_store.cc:276] 0709 13:12:10.125137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.692863 (+567726us) service_pool.cc:224] Handling call
0709 13:12:10.693226 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.734687 (+1041461us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b5800660bd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a4d0aa 00007f68a1acf447 00007f68a1ae7fe7 00007f68a1ae97de 00007f68a1af3b49 00007f68a1af4416 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4
0709 13:12:11.822208 (+ 87521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.880539 (+ 58331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13674240}
W20260709 13:12:11.881930 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.883491 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.740636 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:11.798272 25025 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:12:11.872268 24824 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.871084 25043 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.904453 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.904755 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:11.905285 22914 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:11.895547 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:11.897854 22758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.067s	sys 0.103s
W20260709 13:12:11.906893 22758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.067s	sys 0.103s
I20260709 13:12:11.908917 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.879848 24882 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:11.913445 24843 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.915916 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.900718 24824 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:11.892211 24927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40710 (request call id 235) took 5505 ms. Trace:
I20260709 13:12:11.946381 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.960748 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:11.953169 22914 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:11.882686 24861 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:12:11.962899 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:11.903057 24947 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:11.931880 24835 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:11.943279 25005 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:11.937688 24875 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:11.972545 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:11.986430 22832 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:11.956208 24824 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:11.931926 24459 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:11.966012 24882 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:11.975273 24843 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.975642 24835 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.966349 25044 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:11.993073 24824 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:12.034267 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:12.034751 23461 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:12:12.035219 22730 connection.cc:256] RPC call timeout handler was delayed by -1.2076205710000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
W20260709 13:12:12.035748 22730 connection.cc:256] RPC call timeout handler was delayed by -1.1149205709999706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 13:12:12.036239 22730 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.519s (SENT)
I20260709 13:12:12.036677 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:12.037325 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.012s	sys 0.013s
W20260709 13:12:12.037779 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.012s	sys 0.013s
W20260709 13:12:12.038240 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.01849e+06us
I20260709 13:12:12.039647 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:12.040524 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:12.041337 22762 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:12.041265 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:12.042631 22763 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:11.998591 24835 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:12.046837 22915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.027s	user 0.040s	sys 0.054s
I20260709 13:12:12.047446 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:12.048946 22761 raft_consensus.cc:2368] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4.
W20260709 13:12:12.049773 22915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.041s	sys 0.054s
I20260709 13:12:12.020951 24882 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:12.087245 22762 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:12.112526 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 76313.2us
I20260709 13:12:12.056501 24861 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:12.045938 24843 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:12:12.128036 24991 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:12.128140 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:12.128428 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:12.128999 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:12.129670 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:12.032469 25044 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:12.146257 24947 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:12.147498 25084 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:11.958384 24927 rpcz_store.cc:276] 0709 13:12:06.397476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.537135 (+139659us) service_pool.cc:224] Handling call
0709 13:12:07.357373 (+820238us) ts_tablet_manager.cc:934] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
0709 13:12:07.501219 (+143846us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea00caa 00007f689ea00bd2 00007f68a6a65131
0709 13:12:10.441962 (+2940743us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:12:11.119385 (+677423us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:11.123345 (+  3960us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:12:11.751184 (+627839us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:12:11.751547 (+   363us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:11.871350 (+119803us) ts_tablet_manager.cc:989] Replica registered.
0709 13:12:11.892072 (+ 20722us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":36401,"spinlock_wait_cycles":3318528,"tablet-copy.queue_time_us":679839,"thread_start_us":698357,"threads_started":2}
I20260709 13:12:12.270741 22766 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:12.270946 24927 tablet_copy_client.cc:806] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:12.148242 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:12.271657 24927 tablet_copy_client.cc:670] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:12.151754 25049 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:12.091508 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:12.273861 23464 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:12.155308 22761 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:12.274712 25042 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.000s
I20260709 13:12:12.271937 22762 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 3: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:12.266491 24459 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:12.310050 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.000s	sys 0.040s
W20260709 13:12:12.310563 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.000s	sys 0.040s
W20260709 13:12:12.310902 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.4622e+06us
I20260709 13:12:12.313335 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:12.116830 24882 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:12.281809 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:12.316623 22760 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:12.302593 22763 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:12.120025 24861 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:12.275171 25042 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:12.309860 23461 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:12.320946 22761 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:12.318781 22760 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:12.321877 24861 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:12.308736 24993 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:12.326620 25049 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:12.346123 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:12.353726 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.549s	user 0.003s	sys 0.066s
W20260709 13:12:12.354112 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.549s	user 0.003s	sys 0.066s
I20260709 13:12:12.350540 25081 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:12.365844 24982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.512s	user 0.009s	sys 0.031s
W20260709 13:12:12.361907 25064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.428s	user 0.039s	sys 0.062s
W20260709 13:12:12.372720 25064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.479s	user 0.039s	sys 0.062s
W20260709 13:12:12.366039 25055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.508s	user 0.009s	sys 0.031s
W20260709 13:12:12.440119 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 126836us
I20260709 13:12:12.452833 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:12.466666 24833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.416s	user 0.017s	sys 0.049s
W20260709 13:12:12.467212 24833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.444s	user 0.017s	sys 0.049s
W20260709 13:12:12.452257 24874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.561s	user 0.040s	sys 0.044s
W20260709 13:12:12.452010 25055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.606s	user 0.010s	sys 0.032s
W20260709 13:12:12.377669 24982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.518s	user 0.009s	sys 0.032s
I20260709 13:12:12.462477 24521 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:12.355085 24882 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:12.368090 24824 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:12.469223 25081 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:12.475075 24874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.592s	user 0.040s	sys 0.045s
W20260709 13:12:12.481487 25053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.351s	user 0.028s	sys 0.015s
I20260709 13:12:12.529558 24521 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:12.522435 25055 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:12:12.527186 24664 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.447s	user 0.027s	sys 0.060s
I20260709 13:12:12.732481 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:12.715682 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 207) took 2015 ms. Trace:
I20260709 13:12:12.684899 22761 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:12:12.738004 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.185s	user 0.017s	sys 0.004s
I20260709 13:12:12.729219 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:12.731262 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:12.738572 23431 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.563s	user 0.005s	sys 0.039s
W20260709 13:12:12.753230 24664 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.677s	user 0.027s	sys 0.060s
I20260709 13:12:12.754302 24761 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.565191 24835 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.002s
I20260709 13:12:12.754806 24824 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:12.702279 22807 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.005s	sys 0.017s
W20260709 13:12:12.501673 22911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.266s	user 0.070s	sys 0.162s
I20260709 13:12:12.751369 24521 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:12.684628 22759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.361s	user 0.035s	sys 0.030s
I20260709 13:12:12.536322 24874 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:12.527674 24918 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.440s	user 0.043s	sys 0.055s
W20260709 13:12:12.920852 22759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.597s	user 0.043s	sys 0.050s
W20260709 13:12:12.929917 23457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.069s	user 0.032s	sys 0.032s
W20260709 13:12:12.930433 23457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.032s	sys 0.032s
I20260709 13:12:12.931193 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1163) took 3925 ms. Trace:
I20260709 13:12:12.930660 24874 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:12.642776 24979 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.614s	user 0.013s	sys 0.089s
W20260709 13:12:12.932672 24979 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.906s	user 0.013s	sys 0.089s
I20260709 13:12:12.640949 25017 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:12.933904 24993 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.922215 22730 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.010s	sys 0.011s
I20260709 13:12:12.934835 24993 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:12.572407 25055 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:12.936815 24833 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.605166 25063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.482s	user 0.011s	sys 0.041s
W20260709 13:12:12.938050 25063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.061s	user 0.012s	sys 0.048s
I20260709 13:12:12.937636 24833 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:12.938752 25079 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.535531 25076 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:12.939607 25079 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:12.940150 25084 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:12.940279 25076 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:12.941080 25088 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.941872 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:12.941943 25088 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:12.573908 25082 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.545667 25053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.414s	user 0.028s	sys 0.015s
I20260709 13:12:12.941917 25067 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.934849 25025 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.033s	sys 0.040s
W20260709 13:12:12.690672 22764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.538s	user 0.003s	sys 0.037s
W20260709 13:12:12.950269 25025 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.034s	sys 0.040s
W20260709 13:12:12.950554 22764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.798s	user 0.006s	sys 0.048s
I20260709 13:12:12.848328 22915 rpcz_store.cc:276] 0709 13:12:10.700548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.018700 (+318152us) service_pool.cc:224] Handling call
0709 13:12:11.019112 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:11.019492 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.019525 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:11.020010 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:12:12.053312 (+1033302us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:12.053383 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.053489 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:12.714649 (+661160us) raft_consensus.cc:1685] finished
0709 13:12:12.714877 (+   228us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:12.715391 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134698,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":9066,"thread_start_us":1032888,"threads_started":1,"wal-append.queue_time_us":1555718}
W20260709 13:12:12.690776 22765 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.510s	user 0.015s	sys 0.043s
W20260709 13:12:12.877574 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.325s	user 0.017s	sys 0.007s
W20260709 13:12:12.881175 23431 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.706s	user 0.005s	sys 0.042s
W20260709 13:12:12.904042 22807 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.120s	user 0.007s	sys 0.022s
I20260709 13:12:12.923589 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:12.924472 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.675s	user 0.023s	sys 0.038s
I20260709 13:12:12.924831 25087 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:12.925571 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.353s	user 0.011s	sys 0.031s
I20260709 13:12:12.927690 25085 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.928287 22911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.693s	user 0.070s	sys 0.162s
W20260709 13:12:12.928628 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:12.929476 24918 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.847s	user 0.043s	sys 0.056s
I20260709 13:12:12.620898 25042 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:12.526324 25044 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:12.935459 22730 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.010s	sys 0.011s
W20260709 13:12:12.946319 25009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.031s	sys 0.081s
I20260709 13:12:12.650269 25081 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:12.634557 25086 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:12:12.959520 25009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.032s	sys 0.081s
I20260709 13:12:12.960702 24993 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:12.961542 24800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.081s	sys 0.127s
I20260709 13:12:12.952173 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1171) took 2872 ms. Trace:
W20260709 13:12:12.959105 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 845551us
I20260709 13:12:12.963191 22764 rpcz_store.cc:276] 0709 13:12:10.080211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.563273 (+483062us) service_pool.cc:224] Handling call
0709 13:12:10.581389 (+ 18116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:10.581638 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:11.151931 (+570293us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b180010e400. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf
0709 13:12:11.152096 (+   165us) log.cc:844] Serialized 14 byte log entry
0709 13:12:12.951464 (+1799368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:12.951606 (+   142us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:12.951763 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.951898 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:12.952102 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11691,"prepare.queue_time_us":14343,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":590,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":236,"spinlock_wait_cycles":11808512,"thread_start_us":1822757,"threads_started":2}
W20260709 13:12:12.955607 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.401s	user 0.011s	sys 0.032s
W20260709 13:12:12.965826 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.41441e+06us
W20260709 13:12:12.966181 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.41202e+06us
I20260709 13:12:12.966518 25076 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:12.967911 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:12.957268 22765 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.776s	user 0.023s	sys 0.050s
W20260709 13:12:12.957343 22758 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.000s	sys 0.029s
I20260709 13:12:12.961952 24835 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:12.931653 22759 rpcz_store.cc:276] 0709 13:12:09.005620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.706612 (+700992us) service_pool.cc:224] Handling call
0709 13:12:09.707036 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.718396 (+ 11360us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:09.719258 (+   862us) log.cc:844] Serialized 14 byte log entry
0709 13:12:11.323542 (+1604284us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b18002a5740. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:12.930407 (+1606865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:12.930518 (+   111us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:12.930644 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.930759 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:12.931060 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":508,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":2951,"raft.queue_time_us":1603,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":292,"spinlock_wait_cycles":10336128,"thread_start_us":1606290,"threads_started":1}
I20260709 13:12:12.962275 23465 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:12:12.956030 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.729s	user 0.023s	sys 0.038s
W20260709 13:12:12.969796 22763 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.651s	user 0.015s	sys 0.009s
I20260709 13:12:12.979074 24684 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:12.979254 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:12.979540 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:12.980557 22909 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:12.982951 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:12.922787 25027 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:12.983824 22914 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:12.982338 22758 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.000s	sys 0.031s
I20260709 13:12:12.957653 25082 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:12.997344 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 1.82387e+06us
W20260709 13:12:12.977981 24800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.244s	user 0.083s	sys 0.128s
W20260709 13:12:12.984784 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.20135e+06us
I20260709 13:12:13.030190 25055 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.048007 24918 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.048285 24835 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.049463 25110 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.048969 24918 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.050468 25110 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.050732 22664 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 4, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 4 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.052521 24918 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.054428 24970 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:13.056751 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.057636 22833 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.057404 24993 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:12:13.058611 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.059424 24970 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:13.059923 24970 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:13.056625 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.062245 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.062390 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1181) took 1405 ms. Trace:
I20260709 13:12:13.063086 22841 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.063087 22832 rpcz_store.cc:276] 0709 13:12:11.657118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.659622 (+  2504us) service_pool.cc:224] Handling call
0709 13:12:13.061815 (+1402193us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:13.062106 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649298176}
I20260709 13:12:12.946620 24761 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.005170 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 453) took 1482 ms. Trace:
I20260709 13:12:12.950315 25067 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.062443 22839 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:13.001731 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 1.77493e+06us
I20260709 13:12:12.997243 22763 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:13.080484 22909 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:13.081635 23460 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:12:13.053258 25110 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:13.085456 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101200us
I20260709 13:12:13.086905 25067 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:12.998376 25093 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:13.075152 24834 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.028s	sys 0.030s
I20260709 13:12:13.004702 25085 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.066813 25027 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.005331 25087 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.103417 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.103605 25088 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:13.104295 25009 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.107859 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.109714 22835 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.111150 22834 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.112725 22836 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.116056 24979 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.116304 25102 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:13.100747 24834 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.029s	sys 0.031s
I20260709 13:12:13.125785 25102 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.126309 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.127503 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.127700 22832 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:13.128535 25102 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.128870 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.128984 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.129693 22832 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:13.129762 22836 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.122433 25009 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.061442 24970 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.117586 25079 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.138203 22841 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:13.152498 22912 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.153091 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.155269 22840 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:13.151014 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.122645 24833 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:13.165719 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.174988 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.182422 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.183434 22909 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.185702 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:13.187145 24979 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:13.179198 22840 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.193739 24833 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 13:12:13.196645 25042 raft_consensus.cc:670] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } [suppressed 4 similar messages]
I20260709 13:12:13.125131 25093 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.191576 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1176) took 3110 ms. Trace:
I20260709 13:12:13.197335 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.203500 22910 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:12:13.203555 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.174947 24800 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.209564 25082 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.209985 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.210357 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.201994 24918 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:13.197765 22758 rpcz_store.cc:276] 0709 13:12:10.081571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.569535 (+487964us) service_pool.cc:224] Handling call
0709 13:12:10.572296 (+  2761us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:10.787024 (+214728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.787069 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:10.795214 (+  8145us) log.cc:844] Serialized 52 byte log entry
0709 13:12:11.907505 (+1112291us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:13.084547 (+1177042us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.153432 (+ 68885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.166818 (+ 13386us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.169757 (+  2939us) raft_consensus.cc:1685] finished
0709 13:12:13.169772 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.191523 (+ 21751us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":219178,"prepare.queue_time_us":1043145,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":64,"raft.queue_time_us":216887,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":12443,"thread_start_us":2139776,"threads_started":2,"wal-append.queue_time_us":1629127}
I20260709 13:12:13.203332 25102 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.226306 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.227579 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.228322 22835 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.228565 22837 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:13.229923 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:13.230391 23460 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:13.230806 22910 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.235769 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 441) took 2721 ms. Trace:
I20260709 13:12:13.236366 25085 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:13.215029 24918 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.239873 24918 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.242655 25027 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.244980 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.218678 24761 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.245381 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.245959 23461 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:13.249280 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.251485 25102 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.252162 22833 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:13.254242 23460 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:13.253746 22836 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:13.224915 22841 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:13.215247 24979 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.291149 25087 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.236284 23465 rpcz_store.cc:276] 0709 13:12:10.513931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.518434 (+  4503us) service_pool.cc:224] Handling call
0709 13:12:10.546228 (+ 27794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.041544 (+2495316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.235365 (+193821us) spinlock_profiling.cc:243] Waited 2.41 s on lock 0x7b54002c8b54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:13.235668 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5554251136}
I20260709 13:12:13.279654 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.288275 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1185) took 2110 ms. Trace:
I20260709 13:12:13.290444 25009 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.288818 22762 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:13.292027 22765 rpcz_store.cc:276] 0709 13:12:11.178374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.178577 (+   203us) service_pool.cc:224] Handling call
0709 13:12:11.179086 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:11.179597 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.179642 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:11.180303 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:12:12.974773 (+1794470us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:12.974870 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.975020 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.287941 (+312921us) raft_consensus.cc:1685] finished
0709 13:12:13.287966 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.288170 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83420,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":3002,"thread_start_us":1788972,"threads_started":1,"wal-append.queue_time_us":2029570}
I20260709 13:12:13.279959 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:13.224864 25073 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:13.294605 25073 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:13.294994 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1967) took 1164 ms. Trace:
I20260709 13:12:13.295099 25093 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.239502 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.283916 24833 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:13.296268 24833 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.296370 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:13.296669 24833 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.297086 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 444) took 1270 ms. Trace:
I20260709 13:12:13.284530 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:13.287124 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3732133309999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:12:13.298017 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1053675240000302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
I20260709 13:12:13.287798 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.279448 24970 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:13.303584 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:13.304515 24861 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.962s	user 0.015s	sys 0.010s
I20260709 13:12:13.295876 24800 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:13.306146 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 106649us
W20260709 13:12:13.305917 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49124.3us
I20260709 13:12:13.299078 22839 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:13.301033 25102 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.301862 22915 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.309306 22840 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:13.309545 22910 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:13.306496 25078 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 13:12:13.321112 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:13.321832 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.313753 25042 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:13.314142 24957 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:13.314539 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.295919 25073 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.315397 23459 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:12:13.342828 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.343026 25082 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:13.295488 22762 rpcz_store.cc:276] 0709 13:12:12.130501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.310610 (+180109us) service_pool.cc:224] Handling call
0709 13:12:13.288291 (+977681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.293421 (+  5130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.294781 (+  1360us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b540036f1d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:13.294931 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2247390464}
I20260709 13:12:13.343591 23459 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:13.317469 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.319352 22908 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:13.305114 24861 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.316234 22912 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:12:13.335935 24979 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:13.337726 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:13.316656 24800 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.340215 23462 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:13.340620 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.349709 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.335810 25087 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:12:13.341167 25040 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:13.343315 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63576.9us
I20260709 13:12:13.338507 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:13.379402 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.382910 22914 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:13.354506 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.404165 22839 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.350641 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.409933 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.410279 22833 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.410691 22841 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.411332 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.411499 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.412093 22833 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:13.409035 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.413714 25073 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.414773 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.416234 22839 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.069569 23457 rpcz_store.cc:276] 0709 13:12:11.522743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.533752 (+ 11009us) service_pool.cc:224] Handling call
0709 13:12:11.837204 (+303452us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccf8c 00007f689e9ccef6 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:11.837409 (+   205us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:11.837899 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.837937 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:11.838675 (+   738us) log.cc:844] Serialized 52 byte log entry
0709 13:12:12.931236 (+1092561us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:12.931367 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.931517 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.004416 (+ 72899us) raft_consensus.cc:1685] finished
0709 13:12:13.004454 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.004869 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107449,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":12193,"spinlock_wait_cycles":14434688,"thread_start_us":1091777,"threads_started":1,"wal-append.queue_time_us":1162216}
I20260709 13:12:13.397739 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:13.417402 25092 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:12:13.417953 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.354249 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.419157 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.412286 22841 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.398182 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.344115 25082 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:13.404565 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:13.405419 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:13.405306 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.405369 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.407696 22837 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.394804 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.344990 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.332679 24957 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:13.427513 22840 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.426390 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:13.349138 24993 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:13.347597 24834 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:13.344623 25042 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:13.346486 22906 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.344954 25104 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:13.429075 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 117228us
I20260709 13:12:13.429862 22836 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.430297 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.438663 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.418294 23460 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:13.444975 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.451279 22837 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:13.297776 23458 rpcz_store.cc:276] 0709 13:12:12.026799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.045226 (+ 18427us) service_pool.cc:224] Handling call
0709 13:12:13.239043 (+1193817us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:13.296985 (+ 57942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13230464}
I20260709 13:12:13.447481 22915 raft_consensus.cc:2376] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5. Re-sending same reply.
I20260709 13:12:13.453375 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.456168 25105 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:13.456984 22908 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:13.433929 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:13.431124 24993 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:13.458981 22664 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.440188 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:13.440296 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.434703 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.428391 25082 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:13.435194 24834 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:13.467151 25084 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.471113 25084 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:13.471985 22661 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 263) took 12343 ms (client timeout 15000 ms). Trace:
I20260709 13:12:13.473673 25084 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.474732 22839 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:12:13.476253 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:13.475844 22664 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.476858 22758 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.478063 22907 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:13.478749 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2012) took 3092 ms. Trace:
I20260709 13:12:13.480020 22910 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:12:13.480381 22910 pending_rounds.cc:85] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:12:13.479142 22911 rpcz_store.cc:276] 0709 13:12:10.386012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.639289 (+253277us) service_pool.cc:224] Handling call
0709 13:12:10.699404 (+ 60115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.235043 (+535639us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:11.235339 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 13:12:12.997663 (+1762324us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:12.997797 (+   134us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:12.997940 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.305063 (+307123us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b48000b0c58. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:12:13.478460 (+173397us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54002111d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:13.478508 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.478678 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":618,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":233,"raft.run_cpu_time_us":3293,"raft.run_wall_time_us":307401,"spinlock_wait_cycles":1312544384,"thread_start_us":1734568,"threads_started":1,"wal-append.queue_time_us":1970694}
I20260709 13:12:13.481596 25102 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:13.482936 24861 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.480635 22910 pending_rounds.cc:107] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:12:13.486554 22910 raft_consensus.cc:2889] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:13.487002 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:13.453141 25110 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:13.490270 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 166984us
I20260709 13:12:13.498219 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.500752 25084 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:13.500394 24957 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.501273 25110 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:13.501928 25110 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:13.502291 24957 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.503026 25110 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.505640 24861 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:13.506282 24861 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.500352 22664 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 3, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.507622 22762 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [3.3-4.5]
I20260709 13:12:13.508402 24861 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.508961 24861 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:13.508219 22664 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:12:13.510677 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.511507 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.501552 25084 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:13.514672 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 176265us
I20260709 13:12:13.509073 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:13.526533 22914 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:13.528913 22833 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
W20260709 13:12:13.531064 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66255.1us
I20260709 13:12:13.527875 23458 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:13.530464 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:13.537230 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.534955 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:13.498049 25105 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:13.538837 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67327.3us
W20260709 13:12:13.540921 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 182672us
I20260709 13:12:13.539177 22664 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.544256 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.541005 24835 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.536896 25102 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:13.548893 22664 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.547320 25076 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:13.536224 24861 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.551259 24834 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.552513 24834 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.553830 24861 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:13.554103 25121 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:12:13.554715 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:13.557737 22758 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:13.559250 22762 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.559373 25105 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.561184 25025 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:13.570246 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61380us
I20260709 13:12:13.574009 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.575532 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.576048 25102 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:13.577235 25102 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.578027 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.578797 22841 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:13.579751 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.580732 22761 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.570039 24835 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.583685 24835 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.584545 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.590376 22766 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.611943 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.613132 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:13.568100 24800 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:13.617653 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 153361us
W20260709 13:12:13.472452 22661 rpcz_store.cc:269] 0709 13:12:01.132771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.197238 (+ 64467us) service_pool.cc:224] Handling call
0709 13:12:13.471687 (+12274449us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:03.780777 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:03.792900 (+ 12123us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:03.792966 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:12:03.792987 (+    21us) tablet.cc:662] Decoding operations
0709 13:12:03.794221 (+  1234us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:03.794350 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:03.794387 (+    37us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:12:03.795165 (+   778us) tablet.cc:701] Row locks acquired
0709 13:12:03.795222 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:12:03.795472 (+   250us) write_op.cc:270] Start()
0709 13:12:03.795648 (+   176us) write_op.cc:276] Timestamp: P: 1783602723795378 usec, L: 0
0709 13:12:03.795698 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:12:03.854392 (+ 58694us) log.cc:844] Serialized 2492 byte log entry
0709 13:12:04.641250 (+786858us) op_driver.cc:464] REPLICATION: finished
0709 13:12:05.228924 (+587674us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a5527ef0
0709 13:12:05.571183 (+342259us) write_op.cc:301] APPLY: starting
0709 13:12:05.571332 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:05.571570 (+   238us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:05.571610 (+    40us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:05.620844 (+ 49234us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:05.621152 (+   308us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:12:05.621198 (+    46us) write_op.cc:312] APPLY: finished
0709 13:12:05.621570 (+   372us) log.cc:844] Serialized 53 byte log entry
0709 13:12:05.621816 (+   246us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:05.621941 (+   125us) write_op.cc:454] Released schema lock
0709 13:12:05.622005 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:05.656604 (+ 34599us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:05.656644 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1114,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":910468,"apply.run_cpu_time_us":3550,"apply.run_wall_time_us":51258,"num_ops":6,"prepare.queue_time_us":313929,"prepare.run_cpu_time_us":14060,"prepare.run_wall_time_us":250513,"raft.queue_time_us":194320,"raft.run_cpu_time_us":15444,"raft.run_wall_time_us":1016268,"replication_time_us":845247,"spinlock_wait_cycles":64069888,"thread_start_us":1384155,"threads_started":4,"wal-append.queue_time_us":175411,"wal-append.run_cpu_time_us":40540,"wal-append.run_wall_time_us":2614722}]]}
I20260709 13:12:13.621737 22836 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:13.567726 25084 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.624346 22839 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:13.626137 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:13.627774 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:13.628415 22910 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:13.637687 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.638809 22760 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:13.639956 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.617383 24800 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.643916 22758 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:13.550073 25042 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.650466 22761 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:13.652235 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:13.653220 22762 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:13.654029 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.655439 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.658232 23457 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:13.658409 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132512us
I20260709 13:12:13.658756 25044 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:12:13.659564 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:13.659734 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:13.660939 22834 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:13.661660 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:13.661051 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:13.667279 24957 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.667963 24611 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.667922 24957 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.668669 24611 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.670931 24611 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.672737 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.673928 22758 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:13.559270 24993 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:13.667476 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.678146 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.658380 24808 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.673700 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.683946 22762 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
W20260709 13:12:13.691577 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55702.1us
I20260709 13:12:13.646710 24800 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.692767 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.710259 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:13.641489 25009 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:13.694862 22664 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 4, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 4 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.718418 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.720549 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:13.714599 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 144461us
I20260709 13:12:13.722097 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:13.722713 22758 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:13.723433 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60632.3us
I20260709 13:12:13.725198 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.728698 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.696346 25084 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:13.729103 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.730507 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.711151 24993 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.734409 22833 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.731182 22837 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:13.649531 24761 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:12:13.738164 22880 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.726011 25027 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:13.753701 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57119.2us
I20260709 13:12:13.754052 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:13.698551 25132 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:12:13.766212 25009 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:13.766659 25009 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:13.766968 25134 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:13.767552 25134 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:13.767628 25009 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.769106 25009 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.771571 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:13.733529 25105 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:13.698376 24808 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.758409 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.749739 24835 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:13.760701 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.761209 24684 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.761714 24957 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:13.762094 22806 leader_election.cc:400] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4.
I20260709 13:12:13.730609 25084 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:13.763329 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.764272 25139 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:13.764963 25135 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:13.800841 22765 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.801034 22837 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:13.686942 25082 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.756502 22761 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:13.768147 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.756793 22759 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:13.775918 22839 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:13.783846 22910 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
W20260709 13:12:13.785892 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87121.8us
W20260709 13:12:13.792853 24985 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.518s	user 0.025s	sys 0.021s
I20260709 13:12:13.790100 22834 raft_consensus.cc:2376] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4. Re-sending same reply.
I20260709 13:12:13.795951 22806 leader_election.cc:403] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:12:13.750514 25110 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:13.761744 24861 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:12:13.806820 22840 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.816764 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:13.790942 22664 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.788561 24835 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.793722 24611 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:13.820920 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.850504 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:13.807587 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 163576us
I20260709 13:12:13.797966 25084 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:13.843664 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:13.818653 24985 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.540s	user 0.025s	sys 0.021s
W20260709 13:12:13.853173 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51607us
I20260709 13:12:13.792999 25132 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:12:13.801286 25135 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.852890 22840 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:12:13.852278 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 304236us
I20260709 13:12:13.852898 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.856864 22761 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:13.859220 22839 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.852726 24611 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.859544 24808 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:13.812652 25139 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:13.769208 25134 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:13.855702 22879 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.780s (SENT) [suppressed 15 similar messages]
I20260709 13:12:13.843531 24835 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.853907 25145 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:13.887212 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:13.899441 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:13.890817 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 139462us
W20260709 13:12:13.892318 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 120871us
I20260709 13:12:13.893694 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.852381 24970 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 3. Current term is 4.
W20260709 13:12:13.893893 24918 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.005s
I20260709 13:12:13.893790 22664 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:13.894827 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 92383.9us
I20260709 13:12:13.806071 25027 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.895401 24979 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:13.888248 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:13.899734 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.819360 25110 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.853343 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:13.913019 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.928287 22839 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:13.899039 25082 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.884323 24611 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.920387 22840 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:13.938620 22766 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:13.939658 22760 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:13.941017 22763 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:13.912074 24833 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.615s	user 0.014s	sys 0.000s
I20260709 13:12:13.899281 25135 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:13.942426 22664 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.944305 22758 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:13.942747 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96520.5us
I20260709 13:12:13.823975 24684 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.949168 22839 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:13.927304 25110 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.953401 25078 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:13.954018 25078 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:13.954470 25078 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.955324 25139 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.959160 25027 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:13.950037 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:13.960593 22837 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:13.962886 25063 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:13.963423 24808 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:13.964946 24872 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:13.965395 24872 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:13.966508 24872 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:13.967862 24872 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:13.969981 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57420.3us
I20260709 13:12:13.971043 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:13.972733 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:13.973699 25027 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:13.974474 25027 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:13.975780 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:13.977473 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.983594 24684 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.984385 25044 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:13.984911 25044 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.985342 25044 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:13.989634 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.990808 22807 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:13.992084 24957 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:13.992940 24761 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:13.993593 24761 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:13.994750 24761 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:13.998000 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:13.998765 24957 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.031800 24957 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:13.950347 24833 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.035053 22841 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:14.035836 24833 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:13.999611 25053 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:13.911176 24918 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.004974 25027 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:13.942137 22764 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:13.977697 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:13.983319 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:14.046480 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:13.983131 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:14.018492 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:14.047432 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:14.029219 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104878us
I20260709 13:12:14.030426 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:14.031699 22664 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:14.038159 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 153019us
I20260709 13:12:13.999179 25043 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:12:14.049243 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:14.051573 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.950536 23458 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:12:14.050462 24808 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:12:14.053632 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.054279 25040 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.054854 25040 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.055343 25134 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:14.055897 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.057058 25040 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:14.057806 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:14.058437 23463 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:12:14.058785 23463 pending_rounds.cc:85] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 5
I20260709 13:12:14.058611 25040 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:14.059052 23463 pending_rounds.cc:107] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:12:14.059386 23463 raft_consensus.cc:2889] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:14.059541 25135 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.060552 24947 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:14.060920 24947 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.061232 24947 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.051023 25043 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:14.061923 25084 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.062851 25043 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.063962 25043 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.064289 25084 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:14.064967 25084 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:14.052380 24940 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.057155 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:13.980899 25102 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:12:14.076768 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.096737 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:14.099146 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.098827 22664 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 5, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 5 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:14.103905 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46826.1us
I20260709 13:12:14.081348 25135 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:14.105676 24983 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.106299 24983 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.078138 24940 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.107412 24940 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.108629 24983 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:14.109659 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.116391 25102 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:14.116978 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 144411us
I20260709 13:12:14.095808 25078 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:14.118934 25147 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
W20260709 13:12:14.126221 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48970us
I20260709 13:12:14.068600 25084 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.122078 24790 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:12:14.129166 22664 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 4, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:14.044117 24970 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:12:14.114816 23464 raft_consensus.cc:2368] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6.
I20260709 13:12:14.178972 23465 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:12:14.146175 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.096239 25132 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.152295 24947 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.153980 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:14.154778 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:14.155565 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:14.156396 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:14.156282 24940 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.156880 23459 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:12:14.158105 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 59560.8us
W20260709 13:12:14.157559 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 123512us
I20260709 13:12:14.103091 24833 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.141477 25105 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.139213 22765 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:14.180706 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:14.134456 25110 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.184144 25078 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:14.184753 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:14.185920 24790 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.187085 24808 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:14.186520 22664 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.187748 25082 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:14.187361 24993 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:14.189791 25044 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:14.190887 24834 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.002s
I20260709 13:12:14.205780 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:14.217095 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 71058us
I20260709 13:12:14.210736 24872 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.202864 23458 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:14.197737 25132 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:14.244797 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76154.9us
W20260709 13:12:14.246508 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61875.8us
I20260709 13:12:14.216804 24808 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.216359 24790 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.214069 24970 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.229696 23457 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:14.263765 22664 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:14.241344 24800 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.519s	user 0.006s	sys 0.006s
I20260709 13:12:14.209039 24611 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:14.271915 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:14.244480 25027 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.275776 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.250718 24918 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:14.276569 22914 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:14.278869 24790 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.267807 24808 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.279458 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.280225 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.302927 22910 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:14.281060 24947 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:14.269662 23461 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:14.282121 25157 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:12:14.282774 25040 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:14.283771 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:14.316429 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 286077us
I20260709 13:12:14.290077 24875 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.327340 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.271076 24970 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:14.341202 24859 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:14.341729 24970 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.295285 22907 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:14.244412 25084 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.297859 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:14.300294 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 94622us
I20260709 13:12:14.263753 24957 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.316730 22908 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:14.287637 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.290871 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.272800 25063 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:12:14.326023 24875 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.334918 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 256) took 1090 ms. Trace:
I20260709 13:12:14.334720 24983 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:14.265893 25102 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.335186 24835 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.335793 24611 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:14.338682 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59308.7us
I20260709 13:12:14.534576 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.509825 23464 rpcz_store.cc:276] 0709 13:12:13.244574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.244836 (+   262us) service_pool.cc:224] Handling call
0709 13:12:14.334646 (+1089810us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b540039e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:14.334835 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952744832}
I20260709 13:12:14.280167 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.339675 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:14.391750 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.537906 24872 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:14.538242 25044 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:14.342106 25082 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:12:14.473819 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 156377us
I20260709 13:12:14.275012 22664 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 1 to 3, now has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:14.274787 24800 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.226413 25105 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.506543 22841 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:12:14.507504 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 225226us
W20260709 13:12:14.513021 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 224073us
W20260709 13:12:14.514866 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 217070us
I20260709 13:12:14.511442 22915 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:14.522987 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:14.526110 24944 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.528461 24833 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.483438 25078 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:12:14.529798 25053 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.514662 24947 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:12:14.531531 24983 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.583822 24983 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:14.584206 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:14.532845 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.579732 24790 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.532778 25110 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:14.587334 25044 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.587917 25044 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.588966 24993 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:12:14.574906 24859 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:12:14.590166 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:14.482968 24957 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.481796 25040 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:14.578485 25084 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:14.592697 25040 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.548210 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:14.593976 23459 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.534590 24875 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.547837 25027 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:12:14.551028 24800 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:14.551525 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:14.552040 25132 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:14.553223 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.614974 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:14.616087 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:14.618302 22841 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:12:14.621248 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:14.556365 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:14.557953 25164 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:14.559413 25158 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:14.647894 25158 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.648272 25158 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.649771 25169 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.560704 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:14.650337 24875 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:14.650779 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 90207.7us
I20260709 13:12:14.651810 25073 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.652201 25073 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.652557 25073 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.652720 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.561170 24835 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:14.653839 25078 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:14.654548 25160 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:14.655051 25078 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:14.655268 25160 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.655628 25078 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.656435 25160 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.561785 23461 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:12:14.563807 24872 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.658380 24872 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:14.593168 25084 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:14.548233 22915 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:14.606643 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:14.578570 25043 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.007s
I20260709 13:12:14.580265 25053 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.596055 25105 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:14.617373 22906 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.646536 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:14.646780 25164 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:14.610641 24940 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:14.659864 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 103598us
I20260709 13:12:14.660807 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:14.664230 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:14.713661 22758 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:14.714179 22765 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:12:14.713972 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53268.7us
I20260709 13:12:14.684387 22766 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.714963 22761 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:14.695075 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.715791 22908 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:14.698943 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:14.699234 25166 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.700381 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:14.717581 22760 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:14.701598 23463 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:12:14.705096 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 125438us
I20260709 13:12:14.661288 24800 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.708704 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:14.709281 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:14.659408 24872 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.709627 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:14.723165 22764 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:14.716521 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:14.708202 22910 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:14.697760 25053 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.721557 22835 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:14.613602 25110 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:12:14.709408 22909 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:14.650357 25169 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.700423 25009 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:14.696408 24834 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.657091 25078 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.735468 22759 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:14.736428 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 396851us
W20260709 13:12:14.726063 24761 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.022s	sys 0.044s
I20260709 13:12:14.740351 22907 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:12:14.740485 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.713142 25164 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.721771 24993 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.649292 25158 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.721493 24800 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.745903 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:14.751960 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:14.756542 22758 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.766004 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:14.747658 24761 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.022s	sys 0.044s
I20260709 13:12:14.769006 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:14.768138 24790 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:14.778343 24790 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.778604 25158 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.779246 22906 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:14.779503 25104 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.767082 24800 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.780185 25104 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:14.778755 24790 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.741839 25169 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.753378 24993 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:14.745518 24940 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:14.743172 24834 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:14.792065 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49592us
I20260709 13:12:14.796483 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:14.809453 22907 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.809670 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:14.812037 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73723.9us
I20260709 13:12:14.796949 22766 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:14.741762 25009 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:14.801486 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 265064us
I20260709 13:12:14.817406 25009 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:14.817767 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:14.820654 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.816378 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:14.822105 22766 pending_rounds.cc:85] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 2
I20260709 13:12:14.889348 22766 pending_rounds.cc:107] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:12:14.889773 22766 raft_consensus.cc:2889] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:14.767041 25162 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:12:14.810158 24993 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:14.903514 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.812193 24834 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:14.913929 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 97791.5us
W20260709 13:12:14.918946 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92893.8us
I20260709 13:12:14.814116 24761 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:14.872430 24835 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:14.871663 25078 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:14.928821 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:14.902073 24940 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:14.949863 22766 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:14.951087 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.921077 24833 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:14.904296 25166 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.799716 25084 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:14.932272 24808 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.003s
I20260709 13:12:14.946483 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:14.961956 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 422043us
I20260709 13:12:14.944249 22758 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:14.939075 24944 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.869222 25102 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:14.932068 24835 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:14.971899 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 154216us
I20260709 13:12:14.975519 22757 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:14.987865 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:14.992574 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63909.8us
W20260709 13:12:14.999114 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:14.955269 24761 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.978158 24947 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:15.003396 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.008868 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:15.019058 24927 tablet_copy_client.cc:538] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:15.021533 24875 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:15.022125 24875 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:15.022565 24875 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.022589 23459 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:15.010128 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 95145.2us
I20260709 13:12:15.026269 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:14.916540 25164 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.989717 24835 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:14.989807 24833 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:14.996266 25082 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:15.058029 25082 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.058558 25082 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:15.060781 25082 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.061655 25131 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:15.036985 24957 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:15.039305 25168 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:12:15.041569 25164 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.049593 24940 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:14.993870 25175 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:15.054334 25104 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.011353 25043 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:12:14.990528 24944 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.056028 25073 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:15.057039 25171 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.281s	user 0.002s	sys 0.000s
W20260709 13:12:15.074699 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 305792us
I20260709 13:12:14.974364 25166 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.061335 25082 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:14.981806 24808 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:15.087591 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52929.5us
W20260709 13:12:15.119442 25177 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
I20260709 13:12:15.028287 25174 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:15.065048 25105 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:15.097939 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:15.128365 25165 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.394s	user 0.004s	sys 0.003s
I20260709 13:12:15.113224 25172 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:15.135532 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 373207us
W20260709 13:12:15.154968 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.077289 25110 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:15.028138 25102 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:15.151932 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.156778 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:15.159180 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 131638us
W20260709 13:12:15.156765 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.163993 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.164290 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.172019 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.176545 22914 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:15.170908 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:15.182493 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.158586 24957 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.189288 22914 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:15.133809 24983 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.187031 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.198611 22908 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:12:15.194064 25172 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:15.185307 25040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.002s	sys 0.012s
I20260709 13:12:15.173885 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.173409 25182 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:15.166339 25174 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:15.173418 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.212029 22912 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:12:15.213676 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49777.7us
I20260709 13:12:15.249152 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.130363 24833 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:15.218525 25040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.003s	sys 0.012s
I20260709 13:12:15.189955 24947 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:15.252163 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.268411 22910 raft_consensus.cc:2376] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4. Re-sending same reply.
I20260709 13:12:15.196274 25082 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.203055 25105 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.164551 25110 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.272567 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.263653 22906 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:12:15.264248 24944 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:15.270375 22907 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:15.230269 24859 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:15.262010 22866 heartbeater.cc:646] Failed to heartbeat to 127.21.33.62:44143 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.33.62:44143 timed out after 15.000s (SENT)
I20260709 13:12:15.180274 25189 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:12:15.247574 25188 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.287930 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.281869 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:15.233076 25009 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.258136 24983 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.258394 24808 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:15.305467 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.290282 25102 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.302035 24761 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.293411 24859 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.293573 24944 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:15.303315 25169 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.303453 24790 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.260604 25174 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.293399 23458 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:15.306748 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:15.307905 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 320168us
W20260709 13:12:15.286278 24800 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.007s
I20260709 13:12:15.351643 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:15.337056 25053 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.568s	user 0.004s	sys 0.009s
W20260709 13:12:15.354146 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47493.7us
I20260709 13:12:15.279711 25175 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:15.362947 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.364001 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:15.364356 24872 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.366376 25044 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:15.367444 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:15.369480 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:15.371004 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.374439 25104 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:15.376324 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2056) took 1089 ms. Trace:
I20260709 13:12:15.378064 25053 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:15.379288 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.305248 25073 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:15.343490 25078 raft_consensus.cc:2764] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:12:15.284718 25105 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.297547 25009 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.415740 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.416339 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.419102 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:15.398545 22759 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.650s	user 0.015s	sys 0.000s
I20260709 13:12:15.459710 22759 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:15.414616 24970 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.072s	user 0.012s	sys 0.003s
W20260709 13:12:15.459941 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.292543 24611 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 13:12:15.454784 22909 rpcz_store.cc:276] 0709 13:12:14.287199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.287331 (+   132us) service_pool.cc:224] Handling call
0709 13:12:15.376109 (+1088778us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54003c91d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:15.376237 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471799680}
I20260709 13:12:15.380131 24808 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:15.379141 24947 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.419911 23464 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:15.477665 22915 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 6. Ops: [4.5-4.5]
I20260709 13:12:15.352334 24761 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:12:15.470604 23457 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:15.484614 24875 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.485002 25169 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.485774 25110 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.465371 25043 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:12:15.489532 25175 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:12:15.492040 24835 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.492710 22757 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:15.494027 25191 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:12:15.419327 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:15.496968 25158 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.002s
I20260709 13:12:15.380733 24800 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.295656 25188 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.418479 24790 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:15.455627 25104 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:15.464313 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:15.458560 25139 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:15.417286 24944 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.585665 25189 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:15.568253 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.521947 23458 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:15.418823 24993 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.471987 24808 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:15.566457 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:15.584501 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:15.596804 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 412576us
I20260709 13:12:15.521999 25166 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:15.522320 25131 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.471789 24970 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.601476 23466 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:15.503168 23461 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:15.599946 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.577627 25009 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.655207 23465 raft_consensus.cc:2368] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6.
I20260709 13:12:15.595677 24761 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:15.554741 24834 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
W20260709 13:12:15.605899 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 141681us
I20260709 13:12:15.594544 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.588164 25193 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:15.625856 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 434220us
I20260709 13:12:15.625543 24808 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.639746 25044 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.640533 25169 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.643190 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:15.644681 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:15.645435 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:15.647301 22841 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:15.651700 22833 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:15.713940 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:15.714565 22835 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.715271 24970 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.715862 24970 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:15.716656 24970 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:15.717855 22834 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:15.656106 25110 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:15.652041 22838 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:15.721241 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 215) took 1028 ms. Trace:
I20260709 13:12:15.721727 22915 rpcz_store.cc:276] 0709 13:12:14.692544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.692707 (+   163us) service_pool.cc:224] Handling call
0709 13:12:14.693189 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:15.483264 (+790075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.720920 (+237656us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b540020a3d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:15.721165 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1803367808}
W20260709 13:12:15.654642 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.675206 22839 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:15.682533 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:15.724447 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 372904us
I20260709 13:12:15.724918 22840 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:15.683234 23465 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:15.623603 24859 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:15.685915 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:15.685760 25104 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.687724 24800 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.688935 24835 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.694517 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:15.740386 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:15.602416 22906 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:15.699599 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.659519 24761 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:15.732895 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:15.693969 25043 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.735762 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.635195 25105 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:15.737110 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1226) took 1002 ms. Trace:
W20260709 13:12:15.737262 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 255287us
I20260709 13:12:15.623986 25189 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.657596 25172 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:15.697146 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:15.619510 25139 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:15.741629 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47216us
I20260709 13:12:15.745999 23466 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:12:15.746068 22833 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:15.710999 25175 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:15.700192 25197 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:15.655943 25131 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.678864 25009 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:15.743993 25167 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.730s	user 0.004s	sys 0.000s
I20260709 13:12:15.697886 24834 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.764472 22766 rpcz_store.cc:276] 0709 13:12:14.734636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.968636 (+234000us) service_pool.cc:224] Handling call
0709 13:12:14.983112 (+ 14476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:14.983409 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:15.252413 (+269004us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.736613 (+484200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.736726 (+   113us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:15.736844 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.736924 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.737051 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42446,"prepare.queue_time_us":54925,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":31,"raft.queue_time_us":41369,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":21403,"thread_start_us":712644,"threads_started":2}
W20260709 13:12:15.785282 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.009s	sys 0.016s
W20260709 13:12:15.791680 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 65645.8us
I20260709 13:12:15.791713 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:15.746980 25162 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:15.759660 25104 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.793188 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:15.719319 25044 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:15.791680 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 190734us
I20260709 13:12:15.748591 24993 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.722549 25169 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.775738 24800 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:15.823122 24800 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:15.823559 24800 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:15.823659 25199 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:12:15.824425 25009 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.781438 24859 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:15.828047 25195 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:12:15.770068 25053 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:15.829819 25198 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:12:15.831357 25189 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:15.817824 25207 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:15.832479 25207 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:15.833621 22762 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.806020 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:15.806571 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1227) took 1067 ms. Trace:
W20260709 13:12:15.808632 22805 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:15.835299 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47127.6us
I20260709 13:12:15.836079 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:15.811271 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.010s	sys 0.016s
W20260709 13:12:15.836809 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 630326us
I20260709 13:12:15.817937 22835 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:15.820914 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:15.834646 22758 rpcz_store.cc:276] 0709 13:12:14.739624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.976667 (+237043us) service_pool.cc:224] Handling call
0709 13:12:14.992027 (+ 15360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:14.995113 (+  3086us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:15.471892 (+476779us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.794025 (+322133us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.806078 (+ 12053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.806148 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:15.806209 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.806321 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.806478 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13357,"prepare.queue_time_us":45353,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":19350,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":10737,"thread_start_us":754567,"threads_started":2}
I20260709 13:12:15.838827 22759 raft_consensus.cc:2410] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:12:15.815912 22879 connection.cc:256] RPC call timeout handler was delayed by -1.5904582360000177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.35240000000000005s for a response
I20260709 13:12:15.774215 25043 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.843467 22912 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:15.843925 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 199338us
I20260709 13:12:15.850749 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.851039 25044 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:15.853458 25206 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:15.855265 24875 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:15.858386 22766 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:15.861481 25204 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:15.861982 25204 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:15.862946 25204 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.864261 25204 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:15.867290 25196 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:12:15.845463 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:15.872491 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:15.890614 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:15.894891 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57374.5us
I20260709 13:12:15.823774 25110 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:15.897192 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.895825 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:15.941699 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:15.945127 22836 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:15.945608 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:15.942989 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52480us
I20260709 13:12:15.946380 22758 raft_consensus.cc:2368] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5.
I20260709 13:12:15.947561 22838 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:15.937433 25224 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:15.949863 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:15.950579 25215 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:15.950907 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:15.951993 25218 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:12:15.953328 22906 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:15.949921 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.954356 22910 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:15.833316 25207 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:15.943585 22762 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:15.958312 23462 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:12:15.942183 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 235985us
I20260709 13:12:15.958877 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:15.961406 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:15.960278 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:15.964061 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.964362 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.964697 22911 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.965045 22913 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.896790 25174 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:15.806767 25175 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:15.963050 23431 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.651s (SENT)
I20260709 13:12:15.972958 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:15.973806 22910 raft_consensus.cc:606] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:15.974125 22910 raft_consensus.cc:614] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:15.975383 25202 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:12:15.934073 25110 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:15.974038 25215 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.972437 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:15.895673 25206 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:15.984170 22762 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: [3.6-4.7]
I20260709 13:12:15.984768 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:15.985599 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89862.1us
I20260709 13:12:15.985984 25009 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.986315 24875 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:15.986867 24875 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:15.986583 25009 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.987802 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:15.987908 24875 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.989967 22880 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:15.990576 25158 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:15.991155 25158 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:15.992130 25158 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.939806 25188 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:16.002290 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:16.004070 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:15.959968 25027 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.842234 25162 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:15.993657 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:15.943749 24872 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:15.995834 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.017777 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:15.809864 25197 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:15.842516 24761 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.022293 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:16.022470 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:15.979979 25189 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.982590 24918 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:15.983071 25102 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.982800 25224 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.033372 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.034181 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.034091 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.034888 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.040692 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.041206 22762 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:16.034806 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.041932 22760 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:16.037472 22759 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:16.039253 22765 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:16.018599 22766 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:16.046011 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.046756 22914 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:16.021660 25164 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.936s	user 0.005s	sys 0.012s
I20260709 13:12:16.040011 22764 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:16.049067 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.049615 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.049651 22766 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.050390 22912 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:16.055348 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.019567 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.036924 24918 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.060621 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:16.076068 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.040216 25189 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.074728 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.082268 22760 raft_consensus.cc:2410] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:16.075433 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.076908 22765 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:16.066234 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 82731.1us
I20260709 13:12:16.078668 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:15.895870 25172 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.074498 24761 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.098968 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.044688 25044 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.020265 25027 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.115475 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.116432 22765 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.080556 24918 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.124526 22762 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:12:16.147814 22760 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:16.148773 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:16.124709 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 161717us
I20260709 13:12:16.125785 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.150034 22759 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:16.150315 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.042570 24808 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.152941 22762 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:16.154726 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.150528 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.158241 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.054837 25105 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:12:16.156580 25211 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:16.162052 22836 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 3.
I20260709 13:12:16.163852 22879 leader_election.cc:400] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6.
I20260709 13:12:16.164284 22879 leader_election.cc:403] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:16.054621 25102 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.160838 25043 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.170271 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.182605 22909 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:16.212241 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.214207 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.215233 22762 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:16.216513 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:16.217245 22837 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:16.163645 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:16.220675 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57126.2us
I20260709 13:12:16.223472 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.224129 22760 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 3.
I20260709 13:12:16.225025 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.229636 22762 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:16.227387 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:16.227698 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.158277 24993 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:16.219837 25078 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:12:16.161755 24834 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:16.242242 22766 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:16.206195 24835 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:16.121265 25044 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.220013 24872 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:16.243033 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.249738 22764 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.243630 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.205389 24833 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.231842 25174 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:16.251029 22766 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.251466 22765 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:16.235877 25131 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.257612 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.262152 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.244921 25139 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.246102 25212 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.246620 25169 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:16.037057 25215 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.256032 25174 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.258775 23464 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 2 LEARNER]: Advancing to term 4
I20260709 13:12:16.258586 24834 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6.
I20260709 13:12:16.301538 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.301924 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.305560 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.306885 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.315685 22911 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:16.319271 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.320446 22766 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:16.320739 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 157035us
I20260709 13:12:16.266108 25186 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:16.245081 24993 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.324419 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:16.326417 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.322172 23463 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:16.253528 24835 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:16.332974 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.334875 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:16.056720 25224 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:16.342226 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.345948 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.300592 25131 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.340447 23465 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:16.356564 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:16.300503 25238 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:16.104087 25210 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.400s
I20260709 13:12:16.343426 25043 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:16.305665 25212 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:16.374917 24859 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 13:12:16.375352 24859 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:16.376349 22763 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:16.321025 25174 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.376806 25027 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:16.377774 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.379348 22759 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:16.351475 25186 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:16.359879 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:16.360137 22913 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:16.162213 25182 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.171s
I20260709 13:12:16.318661 25205 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:12:16.364328 25233 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:12:16.397761 22839 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
W20260709 13:12:16.364410 25235 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:12:16.364985 22765 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:12:16.365795 25229 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:12:16.367520 25237 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 13:12:16.368129 24927 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
W20260709 13:12:16.370820 25082 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.094s	user 0.014s	sys 0.006s
I20260709 13:12:16.371512 23461 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:16.372242 24944 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.749s	user 0.010s	sys 0.000s
I20260709 13:12:16.350890 24957 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:12:16.344481 25190 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:16.387117 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:16.389483 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.391337 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.391541 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.442058 22913 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:16.392017 22837 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
W20260709 13:12:16.358543 25216 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
W20260709 13:12:16.372242 25204 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.005s
I20260709 13:12:16.383252 25105 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:16.376363 25212 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.428511 22907 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:16.428927 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 94118.9us
I20260709 13:12:16.425467 24808 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:16.422724 24834 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:16.478287 24834 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:16.426031 25237 consensus_queue.cc:1243] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:16.398026 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 468) took 1363 ms. Trace:
I20260709 13:12:16.420323 25211 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:16.487749 22837 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:12:16.488266 22837 pending_rounds.cc:85] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:12:16.488543 22837 pending_rounds.cc:107] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:12:16.488808 22837 raft_consensus.cc:2889] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:16.454298 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67204.8us
W20260709 13:12:16.460063 23457 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.904s	user 0.013s	sys 0.000s
I20260709 13:12:16.439002 25243 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:16.461004 23464 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEARNER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:16.462282 25215 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:16.462481 25231 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:12:16.463344 24859 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.463863 25084 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.465605 25188 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:16.466125 24918 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.383975 25238 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.467073 25044 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.424562 25189 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.439155 24944 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.433331 25082 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:16.439353 25186 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:16.453457 25104 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.004s
I20260709 13:12:16.472558 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:16.472919 24970 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:16.176564 25206 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.435608 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:16.717298 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.717203 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:16.720122 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:16.494494 22835 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:16.721684 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 249250us
W20260709 13:12:16.614656 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:16.722633 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.520190 22840 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:16.491605 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.729224 22760 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:16.724541 22909 raft_consensus.cc:2376] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6. Re-sending same reply.
W20260709 13:12:16.615324 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.745883 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:16.773712 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:16.747411 22839 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:16.775161 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.781728 22757 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:16.776647 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:16.492141 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.782799 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:16.612519 25239 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.454s	user 0.000s	sys 0.004s
I20260709 13:12:16.805219 22761 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:16.812830 22908 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:16.786638 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:16.792060 23463 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:16.811134 22762 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:16.796514 22760 raft_consensus.cc:2376] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5. Re-sending same reply.
I20260709 13:12:16.816304 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:16.791180 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:12:16.763825 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 43793.3us
I20260709 13:12:16.818542 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:16.820508 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47499.6us
I20260709 13:12:16.722468 24970 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.822489 23466 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:16.726894 25204 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.838419 25204 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.549705 24834 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.840893 24918 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:16.492130 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:16.776888 24957 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.646416 25102 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.571658 25166 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:16.490705 24993 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:16.825909 25158 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:16.827013 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.823824 25182 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:12:16.825937 24835 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.830284 25207 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.832866 25238 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.832624 24833 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:16.778393 25189 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:16.883387 25189 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.883903 25189 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:16.833755 25247 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:12:16.833695 22759 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:12:16.885982 22759 pending_rounds.cc:85] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:12:16.834354 25242 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.886257 25204 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.835232 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.886322 22759 pending_rounds.cc:107] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:12:16.887233 22759 raft_consensus.cc:2889] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:16.887302 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:16.581385 25105 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:16.888767 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:16.558919 25073 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.889034 25105 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:16.795372 25110 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.763s	user 0.007s	sys 0.013s
I20260709 13:12:16.819705 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:16.776996 25243 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.714532 25197 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.791507 25190 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.791836 25131 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:16.813033 25205 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:16.870235 24957 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.747866 23457 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:16.746464 25212 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:16.749929 25224 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.864969 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:16.863755 24834 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.791870 25215 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:16.842629 24970 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:16.876006 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.861636 25237 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:16.747217 25084 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.870930 23458 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.889348 22838 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:16.891655 25073 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.773375 25043 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:16.898948 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79650.8us
I20260709 13:12:16.737844 24859 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.858424 25174 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.893469 24833 consensus_queue.cc:1243] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:16.873880 25166 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.989259 25166 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:16.991063 25243 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.487279 23459 rpcz_store.cc:276] 0709 13:12:15.034169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.056848 (+ 22679us) service_pool.cc:224] Handling call
0709 13:12:15.096213 (+ 39365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:15.636177 (+539964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.655484 (+ 19307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.717964 (+ 62480us) log.cc:844] Serialized 26 byte log entry
0709 13:12:15.892763 (+174799us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:15.895146 (+  2383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.895284 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.358393 (+463109us) raft_consensus.cc:1685] finished
0709 13:12:16.362253 (+  3860us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.394475 (+ 32222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27145,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3649,"thread_start_us":585912,"threads_started":2,"wal-append.queue_time_us":356737}
I20260709 13:12:16.994810 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:16.996716 24835 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:16.997148 24835 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:16.999672 25174 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:17.000008 24835 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:16.963181 24761 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.007s
I20260709 13:12:16.911846 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:17.002204 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 90743.1us
W20260709 13:12:17.002676 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 209) took 1264 ms (client timeout 1623 ms). Trace:
I20260709 13:12:17.003057 22765 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:17.003731 22841 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:12:17.003063 22757 rpcz_store.cc:269] 0709 13:12:15.753449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.768374 (+ 14925us) service_pool.cc:224] Handling call
0709 13:12:17.002460 (+1234086us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:17.002585 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232927616}
I20260709 13:12:17.004549 25175 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:16.916016 25197 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.895577 25110 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.923923 22915 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:16.913096 23457 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 460) took 1546 ms (client timeout 1885 ms). Trace:
I20260709 13:12:16.902462 25131 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:16.993165 23460 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:16.914608 24800 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.088s	user 0.011s	sys 0.004s
I20260709 13:12:16.911151 25205 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:17.001277 24835 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:17.005174 25175 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:17.005236 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:17.005334 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1216) took 1021 ms. Trace:
I20260709 13:12:17.023240 22841 rpcz_store.cc:276] 0709 13:12:15.984301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.985402 (+  1101us) service_pool.cc:224] Handling call
0709 13:12:17.005106 (+1019704us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:17.005271 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296861568}
I20260709 13:12:16.965654 25082 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.027424 23464 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:16.965924 25237 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:16.909358 22764 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.000s
I20260709 13:12:16.961877 25215 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:16.913334 22760 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:16.964607 25212 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:17.016654 22762 raft_consensus.cc:2376] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5. Re-sending same reply.
I20260709 13:12:16.901536 25190 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.158385 22838 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:17.159044 22915 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:17.025869 22757 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:17.030670 25084 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.022687 24800 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.014109 25210 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:12:17.016120 23457 rpcz_store.cc:269] 0709 13:12:15.364826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.367075 (+  2249us) service_pool.cc:224] Handling call
0709 13:12:16.911107 (+1544032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.032066 25188 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:17.032732 25205 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.005554 25224 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.034426 24957 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:17.035531 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:17.198341 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 162912us
I20260709 13:12:17.038930 25082 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.039614 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:17.022889 25175 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:17.041790 25073 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:17.050387 22764 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:17.052456 25237 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:17.019774 25197 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.027634 25053 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:12:17.150307 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:17.146381 25131 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:17.125607 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.147121 25215 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.176887 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2067) took 1279 ms. Trace:
I20260709 13:12:17.217365 22906 rpcz_store.cc:276] 0709 13:12:15.897590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.933671 (+ 36081us) service_pool.cc:224] Handling call
0709 13:12:15.949697 (+ 16026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:15.953787 (+  4090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.176489 (+1222702us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b54003c8cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:17.176794 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7597952}
I20260709 13:12:17.172050 25104 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:12:17.190970 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:17.193218 25084 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:17.193949 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 222) took 1084 ms. Trace:
I20260709 13:12:17.224350 22759 rpcz_store.cc:276] 0709 13:12:16.109633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.380021 (+270388us) service_pool.cc:224] Handling call
0709 13:12:16.380289 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.887789 (+507500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.193611 (+305822us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540002e4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:17.193874 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041855232}
I20260709 13:12:17.194222 22910 pending_rounds.cc:85] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 5
I20260709 13:12:17.206456 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2002) took 1081 ms. Trace:
I20260709 13:12:17.204770 25175 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:17.195403 25188 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:17.214339 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.181654 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:17.230752 22757 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:17.216068 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:17.223395 22760 raft_consensus.cc:2376] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6. Re-sending same reply.
I20260709 13:12:17.225613 22764 rpcz_store.cc:276] 0709 13:12:16.124674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.161138 (+ 36464us) service_pool.cc:224] Handling call
0709 13:12:17.206378 (+1045240us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:17.212993 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62781us
I20260709 13:12:17.234174 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:17.214598 25211 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:17.257376 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:17.258450 22766 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:17.212656 25009 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:17.225472 22910 pending_rounds.cc:107] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:12:17.263906 22910 raft_consensus.cc:2889] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:17.233336 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:17.265720 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:17.267006 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:17.255695 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.268196 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:17.215094 25242 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:17.211856 24927 tablet_bootstrap.cc:700] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent opening tablet: real 0.103s	user 0.001s	sys 0.002s
I20260709 13:12:17.201442 25224 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:17.270179 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53279.2us
I20260709 13:12:17.270341 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:17.279734 22757 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 4: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:17.214802 24875 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:12:17.223181 25073 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:12:17.224115 25182 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:12:17.283612 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:17.284614 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:17.392439 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 158385us
I20260709 13:12:17.394560 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:17.283681 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:17.395937 22766 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:17.395944 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:17.392297 22760 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:17.397411 22838 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:17.398128 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.400079 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:17.406109 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 190136us
I20260709 13:12:17.257416 24761 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:17.388670 23458 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:12:17.392699 25207 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.003s
W20260709 13:12:17.333113 25239 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:12:17.271238 25215 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:17.483008 25215 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.283293 25110 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.283257 25105 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:17.465588 22841 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:17.386668 25082 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:17.229952 25084 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.333801 24859 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.370744 25257 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.469079 24970 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.388696 25256 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.274869 25211 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:17.233705 25212 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:17.492079 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.492563 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:17.499158 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93150.6us
I20260709 13:12:17.504894 22841 pending_rounds.cc:85] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 2
I20260709 13:12:17.494792 22766 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:17.392220 25259 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 13:12:17.485049 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 274) took 1002 ms. Trace:
I20260709 13:12:17.575320 22835 raft_consensus.cc:2368] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
I20260709 13:12:17.573879 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:17.494918 22838 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:17.470145 25009 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.496284 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 224) took 1385 ms. Trace:
I20260709 13:12:17.488197 25210 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.562652 22760 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:17.577610 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:17.538582 25110 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:12:17.507268 25274 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:17.565007 25211 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:17.564215 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.567538 22766 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:17.567925 22841 pending_rounds.cc:107] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:17.568315 25190 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:17.584211 22841 raft_consensus.cc:2889] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:17.483466 25242 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:17.538837 24800 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.554644 24859 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:17.529110 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 121806us
I20260709 13:12:17.528462 25053 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.577212 22759 rpcz_store.cc:276] 0709 13:12:16.110469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.469791 (+1359322us) service_pool.cc:224] Handling call
0709 13:12:17.495347 (+ 25556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.495807 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.495851 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:17.495915 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:17.495963 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.496045 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:17.496205 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":14821}
W20260709 13:12:17.518826 24834 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.007s
I20260709 13:12:17.631695 25215 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.450484 24761 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:17.632962 25263 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:12:17.633838 24761 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.634395 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:17.634600 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1220) took 1485 ms. Trace:
I20260709 13:12:17.634867 24957 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:17.635320 24957 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:17.634990 22833 rpcz_store.cc:276] 0709 13:12:16.149288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.161869 (+ 12581us) service_pool.cc:224] Handling call
0709 13:12:16.235321 (+ 73452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:16.332166 (+ 96845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.332213 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:16.494367 (+162154us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b18003bd4c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a
0709 13:12:17.036456 (+542089us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:17.037266 (+   810us) log.cc:844] Serialized 26 byte log entry
0709 13:12:17.214071 (+176805us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:17.214134 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.214229 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:17.634366 (+420137us) raft_consensus.cc:1685] finished
0709 13:12:17.634388 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:17.634518 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63350,"raft.queue_time_us":76747,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":15841,"spinlock_wait_cycles":29368960,"thread_start_us":176313,"threads_started":1,"wal-append.queue_time_us":268069}
I20260709 13:12:17.636380 24957 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.587090 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2015) took 1117 ms. Trace:
I20260709 13:12:17.578567 22838 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:17.541581 25257 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.574651 23458 rpcz_store.cc:276] 0709 13:12:16.478226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.491776 (+ 13550us) service_pool.cc:224] Handling call
0709 13:12:17.479937 (+988161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.592095 25277 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:17.640311 25257 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:17.642391 25267 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:12:17.644004 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 279) took 1050 ms. Trace:
I20260709 13:12:17.644817 22766 rpcz_store.cc:276] 0709 13:12:16.470097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.566919 (+1096822us) service_pool.cc:224] Handling call
0709 13:12:17.567351 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.583541 (+ 16190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.587007 (+  3466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.555580 25256 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.644434 23459 rpcz_store.cc:276] 0709 13:12:16.593410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.996146 (+402736us) service_pool.cc:224] Handling call
0709 13:12:16.996887 (+   741us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:17.020457 (+ 23570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.021662 (+  1205us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:17.147785 (+126123us) log.cc:844] Serialized 574 byte log entry
0709 13:12:17.388368 (+240583us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:17.388418 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.388492 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:17.643712 (+255220us) raft_consensus.cc:1685] finished
0709 13:12:17.643748 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:17.643933 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":122393,"threads_started":1,"wal-append.queue_time_us":251345}
I20260709 13:12:17.647061 25279 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:17.647838 25256 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.648516 25162 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:17.606385 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.649574 25084 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.624145 25190 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:17.626490 24970 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:17.629217 24835 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.002s
W20260709 13:12:17.651718 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.651556 22764 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:17.586374 25009 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:17.629974 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 88900us
I20260709 13:12:17.630106 25197 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.583035 25210 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.631350 22759 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:17.678054 22760 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:17.584923 25189 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.700s	user 0.007s	sys 0.007s
I20260709 13:12:17.656368 22766 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:12:17.651533 24859 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.664673 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.689052 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:17.689621 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.690186 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.691032 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2016) took 1220 ms. Trace:
I20260709 13:12:17.666756 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.662971 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:17.691596 22759 rpcz_store.cc:276] 0709 13:12:16.470396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.630640 (+1160244us) service_pool.cc:224] Handling call
0709 13:12:17.631013 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.678528 (+ 47515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.690958 (+ 12430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:17.692521 22909 raft_consensus.cc:606] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:17.696444 24859 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:17.692058 22911 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:17.703388 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.704873 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.681468 22906 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:17.687126 22766 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:17.710229 22766 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:17.713593 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.701117 22760 pending_rounds.cc:85] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:12:17.716166 22760 pending_rounds.cc:107] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:12:17.716441 22760 raft_consensus.cc:2889] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:17.717128 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.683148 25162 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.694026 25105 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:17.682106 25190 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.682744 24834 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.720454 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.722710 22911 raft_consensus.cc:2368] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:17.723541 22759 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:17.691111 24957 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:17.727834 22906 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:17.679771 25053 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.748628 25053 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:17.725278 24834 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:17.751555 22908 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
W20260709 13:12:17.741488 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.752404 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.753403 22906 raft_consensus.cc:2368] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:17.745916 25188 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.689702 25189 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.775660 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.777066 22906 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:17.778322 22759 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:17.750771 25175 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.784113 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.788916 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:17.789739 22911 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:17.758841 25110 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:17.793313 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:17.727376 25190 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.808828 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 226) took 1769 ms. Trace:
I20260709 13:12:17.811071 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 276) took 1268 ms. Trace:
I20260709 13:12:17.811652 23464 rpcz_store.cc:276] 0709 13:12:16.542391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.882532 (+340141us) service_pool.cc:224] Handling call
0709 13:12:16.882921 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.028102 (+145181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.810694 (+782592us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002c9554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:17.810984 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331584256}
I20260709 13:12:17.727222 25162 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.809474 22907 rpcz_store.cc:276] 0709 13:12:16.039631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.040336 (+   705us) service_pool.cc:224] Handling call
0709 13:12:16.061761 (+ 21425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.474041 (+412280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.808552 (+1334511us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54001e3cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:17.808758 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842967296}
W20260709 13:12:17.827780 25237 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.000s
I20260709 13:12:17.828354 25237 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.831681 25237 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:17.832930 25280 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.833846 25280 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:17.813755 25234 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.464s	user 0.007s	sys 0.025s
I20260709 13:12:17.852044 25189 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:17.855041 25175 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:17.801066 25105 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:17.844636 25234 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.501s	user 0.007s	sys 0.030s
I20260709 13:12:17.800606 25110 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:17.883117 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:17.886703 22832 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:18.145890 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.147082 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1248) took 1158 ms. Trace:
I20260709 13:12:18.179037 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 478) took 1791 ms. Trace:
I20260709 13:12:18.282974 25175 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.283629 25175 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:18.286302 25175 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:18.179860 23465 rpcz_store.cc:276] 0709 13:12:16.388061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.390348 (+  2287us) service_pool.cc:224] Handling call
0709 13:12:16.422920 (+ 32572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.813840 (+390920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.813873 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.813921 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:16.813956 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.178654 (+1364698us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:18.178733 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.178941 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898486144}
I20260709 13:12:18.287225 25280 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:18.291162 25285 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.397s	user 0.010s	sys 0.003s
I20260709 13:12:18.293372 25158 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:18.282114 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.295665 22837 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:18.296175 22837 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1225) took 1815 ms (client timeout 2247 ms). Trace:
I20260709 13:12:18.296485 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1259) took 1580 ms. Trace:
W20260709 13:12:18.294783 25275 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.715s	user 0.002s	sys 0.007s
I20260709 13:12:18.296868 22910 rpcz_store.cc:276] 0709 13:12:16.716273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.716406 (+   133us) service_pool.cc:224] Handling call
0709 13:12:16.716815 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.303473 (+586658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.303505 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.334352 (+ 30847us) log.cc:844] Serialized 26 byte log entry
0709 13:12:17.727925 (+393573us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:17.727983 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.290333 (+562350us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:18.296185 (+  5852us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:18.296246 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.296291 (+    45us) raft_consensus.cc:1685] finished
0709 13:12:18.296304 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.296428 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120721024,"thread_start_us":312700,"threads_started":1,"wal-append.queue_time_us":412724}
I20260709 13:12:18.297688 24957 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:18.298785 24957 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:18.299150 24957 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:18.299714 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1990) took 2293 ms (client timeout 2231 ms). Trace:
I20260709 13:12:18.299979 24957 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.301667 25190 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:18.238647 25210 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:18.302445 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.263006 24834 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:18.293047 25043 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:18.229952 25188 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:18.303964 25043 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:18.304311 22833 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:12:18.265983 25162 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:18.306417 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.306787 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.307051 23464 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:18.307507 23465 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:18.307297 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:18.308271 23457 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.309899 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.306432 25254 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:18.310689 25254 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:18.311591 25254 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.312826 25254 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:18.312824 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:18.276780 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1226) took 1795 ms. Trace:
I20260709 13:12:18.313447 25224 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:18.313671 23461 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:18.314260 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 277) took 1738 ms. Trace:
W20260709 13:12:18.317435 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2011) took 1848 ms (client timeout 1870 ms). Trace:
I20260709 13:12:18.325073 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1245) took 1336 ms. Trace:
I20260709 13:12:18.301329 24957 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.322880 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:18.359578 22836 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:18.360373 22836 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1223) took 1880 ms (client timeout 2121 ms). Trace:
I20260709 13:12:18.322793 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:18.363027 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:18.329852 23464 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:18.179280 22759 rpcz_store.cc:276] 0709 13:12:16.989128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.884326 (+895198us) service_pool.cc:224] Handling call
0709 13:12:18.146310 (+261984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.146548 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.146588 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.146660 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:18.146712 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.146833 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.147004 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:18.360875 22836 rpcz_store.cc:269] 0709 13:12:16.480185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.549537 (+ 69352us) service_pool.cc:224] Handling call
0709 13:12:18.317775 (+1768238us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:18.360280 (+ 42505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28796800}
W20260709 13:12:18.366310 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68816us
W20260709 13:12:18.367168 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.321751 25189 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:18.300283 22763 rpcz_store.cc:269] 0709 13:12:16.007033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.016532 (+  9499us) service_pool.cc:224] Handling call
0709 13:12:18.299385 (+2282853us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b540024b8d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:18.299587 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769964800}
W20260709 13:12:18.296525 22837 rpcz_store.cc:269] 0709 13:12:16.480857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.573976 (+ 93119us) service_pool.cc:224] Handling call
0709 13:12:18.281871 (+1707895us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:18.296093 (+ 14222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27853568}
I20260709 13:12:18.374372 22836 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:18.322127 22840 rpcz_store.cc:276] 0709 13:12:16.481153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.751972 (+270819us) service_pool.cc:224] Handling call
0709 13:12:16.752316 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.993820 (+241504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.993859 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.993909 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.993941 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.276309 (+1282368us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54000a4e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:18.276396 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.276653 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554585856}
I20260709 13:12:18.354911 22764 rpcz_store.cc:276] 0709 13:12:16.988335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.266716 (+278381us) service_pool.cc:224] Handling call
0709 13:12:18.324875 (+1058159us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54001fa254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:18.325011 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882364800}
W20260709 13:12:18.355736 22762 rpcz_store.cc:269] 0709 13:12:16.468852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.815960 (+347108us) service_pool.cc:224] Handling call
0709 13:12:18.317238 (+1501278us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54002224d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:18.317365 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459124480}
I20260709 13:12:18.327025 22832 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:18.276844 25214 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:18.367918 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:18.336755 22806 connection.cc:256] RPC call timeout handler was delayed by -1.6054362050000464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:12:18.379478 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5325362050000422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 13:12:18.379944 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4749362050000627s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260709 13:12:18.380342 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4452362050000147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 13:12:18.380686 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4335362049999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
W20260709 13:12:18.381487 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2985362050000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:12:18.381940 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2544362050000473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 13:12:18.382316 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0898172949999889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 13:12:18.382658 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0375362050000376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
W20260709 13:12:18.382943 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0295172950000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
W20260709 13:12:18.366812 25266 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.617s	user 0.005s	sys 0.005s
W20260709 13:12:18.385051 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1991) took 2376 ms (client timeout 1554 ms). Trace:
W20260709 13:12:18.385525 22761 rpcz_store.cc:269] 0709 13:12:16.008577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.008938 (+   361us) service_pool.cc:224] Handling call
0709 13:12:18.384830 (+2375892us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b5400221ad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:18.384957 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1759671040}
I20260709 13:12:18.386721 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 233) took 1475 ms. Trace:
I20260709 13:12:18.387120 22760 rpcz_store.cc:276] 0709 13:12:16.910941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.678992 (+768051us) service_pool.cc:224] Handling call
0709 13:12:17.700829 (+ 21837us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:17.753435 (+ 52606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.753481 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:17.754210 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 13:12:17.879792 (+125582us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:17.879880 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.879997 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.386456 (+506459us) raft_consensus.cc:1685] finished
0709 13:12:18.386493 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.386659 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1051,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3221,"thread_start_us":89616,"threads_started":1,"wal-append.queue_time_us":589327}
I20260709 13:12:18.367993 25224 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:17.899712 25110 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:18.341135 23466 rpcz_store.cc:276] 0709 13:12:16.576315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.887458 (+311143us) service_pool.cc:224] Handling call
0709 13:12:16.887779 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.043611 (+155832us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:17.204504 (+160893us) log.cc:844] Serialized 14 byte log entry
0709 13:12:17.285571 (+ 81067us) log.cc:844] Serialized 14 byte log entry
0709 13:12:17.286189 (+   618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:17.286246 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:17.286294 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.313994 (+1027700us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54006707d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:18.314055 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.314195 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4232,"prepare.queue_time_us":577045,"prepare.run_cpu_time_us":4865,"prepare.run_wall_time_us":72841,"raft.queue_time_us":430,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":27853,"spinlock_wait_cycles":357807488,"thread_start_us":124725,"threads_started":2,"wal-append.queue_time_us":165707}
I20260709 13:12:18.396050 25224 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:18.376327 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.377671 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:18.397073 25224 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:18.338008 25226 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.398423 25226 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:18.391932 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1227) took 1905 ms (client timeout 2245 ms). Trace:
I20260709 13:12:18.393126 25254 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:18.401116 22832 rpcz_store.cc:269] 0709 13:12:16.486720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.633910 (+147190us) service_pool.cc:224] Handling call
0709 13:12:17.882884 (+1248974us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:18.391848 (+508964us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47031680}
I20260709 13:12:18.400534 25226 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:18.145258 25283 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.343957 25208 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:12:18.378845 25214 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:18.402325 25212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.009s	sys 0.047s
I20260709 13:12:18.413265 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:18.417096 22840 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:12:18.420377 22840 raft_consensus.cc:606] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Received request to transfer leadership to TS 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:18.420821 22840 raft_consensus.cc:614] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:18.411954 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:18.420822 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:12:18.423203 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.393122 25169 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:18.379566 25189 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:18.417842 25073 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:18.409667 25290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.019s	sys 0.036s
W20260709 13:12:18.422657 25212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.009s	sys 0.048s
I20260709 13:12:18.419049 25254 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:18.427024 25063 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:18.433821 25290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.020s	sys 0.037s
I20260709 13:12:18.444373 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:18.414583 25283 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.431434 25280 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:18.446966 25243 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:18.458181 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.464502 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.465291 23462 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4. Re-sending same reply.
I20260709 13:12:18.459455 25063 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:18.490912 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:18.494104 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:18.499969 25110 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:18.500576 25110 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.434482 25073 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:18.435176 25043 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.506126 23466 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:18.537997 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 215) took 2579 ms. Trace:
I20260709 13:12:18.538790 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 223) took 2428 ms. Trace:
I20260709 13:12:18.539492 22765 rpcz_store.cc:276] 0709 13:12:16.110054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.414824 (+304770us) service_pool.cc:224] Handling call
0709 13:12:17.002709 (+587885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.003430 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.533918 (+1530488us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400220bd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:18.538705 (+  4787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350955904}
I20260709 13:12:18.538637 22758 rpcz_store.cc:276] 0709 13:12:15.958853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.959035 (+   182us) service_pool.cc:224] Handling call
0709 13:12:15.977493 (+ 18458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:16.041969 (+ 64476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.042001 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.158322 (+1116321us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:17.193759 (+ 35437us) log.cc:844] Serialized 26 byte log entry
0709 13:12:17.273181 (+ 79422us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:17.273258 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.537622 (+1264364us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b540035d754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:18.537679 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.537725 (+    46us) raft_consensus.cc:1685] finished
0709 13:12:18.537739 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.537910 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155751040,"thread_start_us":78830,"threads_started":1,"wal-append.queue_time_us":166337,"wal-append.run_cpu_time_us":2283,"wal-append.run_wall_time_us":1115400}
I20260709 13:12:18.462309 23461 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
W20260709 13:12:18.515307 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1229) took 1902 ms (client timeout 2213 ms). Trace:
I20260709 13:12:18.546298 22759 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:18.457278 25280 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.463995 25040 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:18.460908 25243 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:18.570016 22880 connection.cc:256] RPC call timeout handler was delayed by -1.0639410859999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 13:12:18.511498 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.547716 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.552316 25073 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:18.469681 25214 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.544690 25254 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.553632 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.585052 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:18.555068 25110 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:18.557881 25283 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:18.546635 22835 rpcz_store.cc:269] 0709 13:12:16.612784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:16.721686 (+108902us) service_pool.cc:224] Handling call
0709 13:12:18.515020 (+1793334us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54003219d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:18.515227 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754656128}
I20260709 13:12:18.594470 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.566640 25280 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:18.607409 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:18.609814 22832 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:18.614521 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:18.626142 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.541049 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.571130 24834 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.532866 24872 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.670364 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.550364 25212 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.588176 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.685111 22759 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.599448 22757 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:18.544992 25053 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:12:18.695690 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.686299 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:18.696911 22757 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.568626 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.698246 22758 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:18.564210 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:18.583052 25186 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:18.702060 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 137943us
I20260709 13:12:18.678282 22913 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:18.704639 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 340417us
I20260709 13:12:18.676198 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.577816 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.674547 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:18.720849 22760 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.681542 22761 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.688302 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.674804 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:18.709225 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:18.709218 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.749846 22765 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.761446 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:18.762485 22759 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:18.716179 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:18.765838 22907 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:18.717602 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.761924 22836 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:18.767416 22837 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:18.779088 22758 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:12:18.780586 22841 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:18.780794 22836 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:18.694535 25175 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.774379 22765 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:18.791870 22765 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:18.793668 22761 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:18.803052 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.793346 22765 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:18.793833 22760 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:18.835378 22765 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:18.686848 24834 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.782760 25214 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:18.650532 25063 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.713797 25162 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:18.818320 22914 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:18.811940 25271 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:18.827630 22907 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:18.813056 25189 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:18.810374 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 398435us
I20260709 13:12:18.695288 24872 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:18.702008 25212 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:18.711127 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 386523us
I20260709 13:12:18.836333 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:18.702535 25243 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.883734 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:18.763962 25186 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.889665 22832 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:12:18.909420 22832 pending_rounds.cc:85] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:12:18.909937 22832 pending_rounds.cc:107] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:12:18.807797 25175 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:18.869171 25158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.008s	sys 0.026s
I20260709 13:12:18.919414 22757 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:18.866268 25162 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.857795 25271 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:18.934787 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.883694 25189 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.941026 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:12:18.926391 25158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.009s	sys 0.026s
I20260709 13:12:18.911893 22832 raft_consensus.cc:2889] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:18.932678 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:18.947253 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:18.943843 25298 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:12:18.948757 22907 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:18.951313 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:18.952497 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.928187 25243 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.545989 23457 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:18.953223 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:18.934087 25162 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.949163 25189 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:18.957569 25259 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:18.949581 25188 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:18.973603 25242 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.967772 25175 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:18.970026 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:18.935168 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 54934.7us
I20260709 13:12:18.939157 25043 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:18.990720 22763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.024s	sys 0.037s
W20260709 13:12:18.991266 22763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.025s	sys 0.037s
I20260709 13:12:18.923061 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:18.955287 25169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.014s	sys 0.023s
I20260709 13:12:18.983044 24872 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:18.994347 25169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.015s	sys 0.024s
I20260709 13:12:18.999550 25169 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:18.999706 25274 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:12:18.999961 25299 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:19.000212 25274 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:19.000634 25169 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.001991 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:19.002712 23461 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:19.006429 23466 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:19.001195 25274 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:18.985049 25186 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:18.993450 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70480.2us
I20260709 13:12:19.010491 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:19.019551 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1236) took 1601 ms. Trace:
I20260709 13:12:19.024533 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:19.041908 25301 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:12:19.052477 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:19.055049 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:19.056383 22836 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:19.057763 25271 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.059284 25271 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.068745 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:19.069638 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.070492 24872 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.988595 25053 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:18.986756 25242 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:19.023788 22834 rpcz_store.cc:276] 0709 13:12:17.418181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.831259 (+413078us) service_pool.cc:224] Handling call
0709 13:12:17.831649 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.832044 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.832075 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:18.162765 (+330690us) log.cc:844] Serialized 26 byte log entry
0709 13:12:18.711386 (+548621us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:18.711447 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.711575 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.019220 (+307645us) raft_consensus.cc:1685] finished
0709 13:12:19.019256 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.019463 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":159,"thread_start_us":342832,"threads_started":1,"wal-append.queue_time_us":603698}
I20260709 13:12:19.062908 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:19.073900 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49410.1us
I20260709 13:12:19.082800 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:19.086215 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:19.071053 24872 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:19.057385 24957 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.001s
I20260709 13:12:19.020902 25175 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.033699 25188 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:19.091909 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 369315us
I20260709 13:12:19.097077 22912 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:12:19.098016 24957 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:19.069662 25212 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:19.089624 25259 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:12:19.102905 25316 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:19.109582 25040 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.000s
I20260709 13:12:19.111078 25162 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.082180 25303 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:19.129886 25303 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:19.129956 24834 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:19.084263 25186 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:12:19.133409 25303 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:19.136416 25303 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.120039 25188 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:19.136723 25313 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:19.136816 25303 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.125056 25189 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:19.128671 25310 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:19.138154 24927 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260709 13:12:19.127336 25040 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:19.159107 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 316535us
W20260709 13:12:19.160285 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.520s (SENT) [suppressed 19 similar messages]
I20260709 13:12:19.163048 23464 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:12:19.118005 25175 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:19.172525 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 235) took 1407 ms. Trace:
I20260709 13:12:19.183256 22766 rpcz_store.cc:276] 0709 13:12:17.752475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.901732 (+149257us) service_pool.cc:224] Handling call
0709 13:12:18.145440 (+243708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:18.214575 (+ 69135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.214609 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:18.215431 (+   822us) log.cc:844] Serialized 26 byte log entry
0709 13:12:18.586882 (+371451us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:19.158863 (+571981us) log.cc:844] Serialized 14 byte log entry
0709 13:12:19.159192 (+   329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.159313 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.159377 (+    64us) raft_consensus.cc:1685] finished
0709 13:12:19.159395 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.159607 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35578,"raft.queue_time_us":70468,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":840923,"threads_started":2,"wal-append.queue_time_us":721206}
W20260709 13:12:19.186220 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.194976 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:19.133078 25280 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:19.162133 25311 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:19.224565 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:19.227994 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:19.236079 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:19.236683 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:19.145412 25316 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:12:19.265275 25158 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:19.186559 23460 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 272) took 3626 ms (client timeout 1907 ms). Trace:
I20260709 13:12:19.178891 25189 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:12:19.227067 25186 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:19.278847 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 192823us
I20260709 13:12:19.285148 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:19.287213 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:19.227327 24834 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:19.288126 22837 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:19.261615 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:19.188004 25313 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.093s
W20260709 13:12:19.307999 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.293974 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:19.079948 24808 consensus_queue.cc:1175] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "acfda9b350734def9b6c621f66a5f247" responder_term: 6 status { last_received { term: 5 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:19.276080 25311 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:19.333081 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:19.340088 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:19.339571 25053 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:19.277878 25158 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.341861 25158 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.343076 25053 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.288443 25040 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:12:19.367197 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 206931us
I20260709 13:12:19.329689 25310 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:19.328753 25317 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:19.355091 24927 tablet_bootstrap.cc:492] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:12:19.337899 25325 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:19.360607 25259 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:19.338032 23458 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:19.331095 24834 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:19.394122 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 158128us
I20260709 13:12:19.311777 25186 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.329463 25188 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:19.389202 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 152541us
I20260709 13:12:19.414237 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1265) took 1750 ms. Trace:
I20260709 13:12:19.415051 22909 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:19.366664 25040 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:19.280254 23460 rpcz_store.cc:269] 0709 13:12:15.551707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.612521 (+ 60814us) service_pool.cc:224] Handling call
0709 13:12:19.162539 (+3550018us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b5400236254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:19.175979 (+ 13440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220184320}
I20260709 13:12:19.422319 22730 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:19.428920 22730 leader_election.cc:403] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:19.430285 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:19.374369 25310 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:19.421772 22908 rpcz_store.cc:276] 0709 13:12:17.663345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.663503 (+   158us) service_pool.cc:224] Handling call
0709 13:12:19.414022 (+1750519us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b54003c9bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:19.414156 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140254464}
W20260709 13:12:19.417665 25309 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:12:19.408443 25242 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:19.398103 25306 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
I20260709 13:12:19.447342 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:19.398545 24872 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:19.420715 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 338208us
I20260709 13:12:19.382802 25337 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:19.868908 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:19.863878 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 523856us
I20260709 13:12:19.418231 25259 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.765314 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1239) took 1172 ms. Trace:
I20260709 13:12:19.771790 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1241) took 1178 ms. Trace:
I20260709 13:12:19.900640 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:19.902465 22761 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:19.484647 22766 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:19.660768 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:19.710904 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.013s
W20260709 13:12:19.881093 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.003s	sys 0.018s
W20260709 13:12:19.502530 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 83676.3us
I20260709 13:12:19.883092 22765 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:19.904836 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.000s	sys 0.022s
W20260709 13:12:19.904906 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.003s	sys 0.018s
I20260709 13:12:19.882735 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 286) took 1499 ms. Trace:
W20260709 13:12:19.905352 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 809410us
W20260709 13:12:19.850687 25214 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.001s	user 0.006s	sys 0.008s
I20260709 13:12:19.571925 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:19.404220 25206 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:19.897147 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:19.891570 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:19.905321 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 817599us
I20260709 13:12:19.905762 23464 rpcz_store.cc:276] 0709 13:12:18.383017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.405609 (+ 22592us) service_pool.cc:224] Handling call
0709 13:12:19.162707 (+757098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:19.179048 (+ 16341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.882491 (+703443us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b5400236250. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:19.882670 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739911936}
I20260709 13:12:19.816569 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1243) took 1222 ms. Trace:
W20260709 13:12:19.691394 25342 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:12:19.900444 22839 rpcz_store.cc:276] 0709 13:12:18.592543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.592947 (+   404us) service_pool.cc:224] Handling call
0709 13:12:18.593702 (+   755us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:18.951935 (+358233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.951964 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:18.986559 (+ 34595us) log.cc:844] Serialized 574 byte log entry
0709 13:12:19.292300 (+305741us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:19.292339 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.292416 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.765010 (+472594us) raft_consensus.cc:1685] finished
0709 13:12:19.765040 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.765230 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":362420,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":184,"spinlock_wait_cycles":2432,"thread_start_us":660274,"threads_started":2,"wal-append.queue_time_us":440491}
I20260709 13:12:19.571292 25105 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.782255 25280 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:19.417937 25325 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:19.903826 24872 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.485620 24927 ts_tablet_manager.cc:1403] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 3.118s	user 0.056s	sys 0.047s
W20260709 13:12:19.777627 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:19.942930 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.013s	sys 0.013s
I20260709 13:12:19.500913 25317 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:19.500721 25212 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.907524 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:19.922257 23457 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:12:19.777797 25175 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:19.915979 22833 rpcz_store.cc:276] 0709 13:12:18.594014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.594170 (+   156us) service_pool.cc:224] Handling call
0709 13:12:18.675239 (+ 81069us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:19.085611 (+410372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:19.085651 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:19.121039 (+ 35388us) log.cc:844] Serialized 52 byte log entry
0709 13:12:19.338632 (+217593us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:19.338678 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.338747 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.816155 (+477408us) raft_consensus.cc:1685] finished
0709 13:12:19.816191 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.816466 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2321,"raft.queue_time_us":283644,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":571799,"threads_started":2,"wal-append.queue_time_us":682074}
W20260709 13:12:19.503588 25302 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.344s	user 0.001s	sys 0.001s
I20260709 13:12:19.772694 25186 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:19.408979 25311 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.907629 25259 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.932360 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:19.931941 25310 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.740352 25345 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:19.902171 22840 rpcz_store.cc:276] 0709 13:12:18.593280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.593407 (+   127us) service_pool.cc:224] Handling call
0709 13:12:18.593713 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:18.594074 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:19.091651 (+497577us) log.cc:844] Serialized 14 byte log entry
0709 13:12:19.356471 (+264820us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:19.371980 (+ 15509us) log.cc:844] Serialized 26 byte log entry
0709 13:12:19.372455 (+   475us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:19.372529 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.372696 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.771416 (+398720us) raft_consensus.cc:1685] finished
0709 13:12:19.771452 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.771691 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17449,"prepare.queue_time_us":601577,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":385782,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":16637,"spinlock_wait_cycles":16256,"thread_start_us":634600,"threads_started":2,"wal-append.queue_time_us":564790}
I20260709 13:12:19.645434 25158 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:19.568423 25040 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:19.944684 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:19.643810 25335 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.329s
I20260709 13:12:19.934746 25131 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:19.864173 25319 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:19.779457 24834 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:19.933287 25105 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:19.422573 25162 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:19.910681 25271 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:19.944013 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.014s	sys 0.013s
I20260709 13:12:19.952316 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1269) took 2252 ms. Trace:
I20260709 13:12:19.953840 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 240) took 1070 ms. Trace:
W20260709 13:12:20.052987 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 979749us
I20260709 13:12:20.053185 22907 rpcz_store.cc:276] 0709 13:12:18.883277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.883432 (+   155us) service_pool.cc:224] Handling call
0709 13:12:19.953596 (+1070164us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b5400041854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:19.953756 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90589056}
I20260709 13:12:19.922675 25206 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:19.940631 25044 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:19.991061 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:19.991716 25310 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.501775 25009 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:20.000486 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 237) took 2141 ms. Trace:
I20260709 13:12:19.977555 25336 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.079928 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2083) took 2400 ms. Trace:
I20260709 13:12:20.080120 24927 raft_consensus.cc:359] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 NON_PARTICIPANT]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.082532 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:20.083563 22841 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:20.084772 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:20.002373 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70164.3us
I20260709 13:12:19.997632 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.018304 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:19.977531 25175 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.019838 22834 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:20.020548 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.021379 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.091856 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:20.023489 22837 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:19.989753 25345 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.028311 22833 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:20.096969 22833 pending_rounds.cc:85] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 2
I20260709 13:12:20.097301 22833 pending_rounds.cc:107] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:20.097623 22833 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:19.950775 24872 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:19.938755 25053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.018s	sys 0.016s
I20260709 13:12:20.053037 22913 rpcz_store.cc:276] 0709 13:12:17.700191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.711508 (+ 11317us) service_pool.cc:224] Handling call
0709 13:12:17.726195 (+ 14687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.703623 (+977428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.951966 (+1248343us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b54001e2dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:19.952228 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187868928}
W20260709 13:12:20.104127 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.058001 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.058822 22762 rpcz_store.cc:276] 0709 13:12:17.859212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.379223 (+520011us) service_pool.cc:224] Handling call
0709 13:12:18.379786 (+   563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:18.380123 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.380154 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:18.387168 (+  7014us) log.cc:844] Serialized 26 byte log entry
0709 13:12:18.876600 (+489432us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:18.876648 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.876729 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:20.000141 (+1123412us) raft_consensus.cc:1685] finished
0709 13:12:20.000175 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:20.000397 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":489078,"threads_started":1,"wal-append.queue_time_us":702434}
I20260709 13:12:20.056363 25208 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:20.066318 25237 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.067973 25283 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:20.068686 25105 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:20.106807 25053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.018s	sys 0.016s
I20260709 13:12:20.123129 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:20.123485 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.123781 22906 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.019520 25319 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.058591 25325 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.124096 22833 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:20.075708 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 235) took 1390 ms. Trace:
I20260709 13:12:20.019657 25131 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.080639 22909 rpcz_store.cc:276] 0709 13:12:17.679962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.717952 (+ 37990us) service_pool.cc:224] Handling call
0709 13:12:19.414676 (+1696724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:19.422669 (+  7993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:20.079616 (+656947us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54003c9bd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:20.079820 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3901682944}
I20260709 13:12:20.096302 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.106122 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.106410 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.096475 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.112061 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.057236 25206 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.114702 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.116199 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.111933 25208 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:20.112170 25139 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.118852 25335 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:20.104547 25224 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:20.100013 25345 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.071615 25158 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:20.090734 25044 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:20.122362 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.100551 25355 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.127238 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:20.059069 25271 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.130352 22832 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:20.130906 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:20.080291 25336 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.099309 25352 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:20.131876 22834 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:20.128294 22912 rpcz_store.cc:276] 0709 13:12:18.685731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.685955 (+   224us) service_pool.cc:224] Handling call
0709 13:12:20.075420 (+1389465us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:20.075623 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918795520}
I20260709 13:12:20.130488 22838 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:20.130445 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:20.133302 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:20.134614 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:20.154510 22913 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:20.142835 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.140183 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:20.123306 24957 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:20.140419 25303 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.961s	user 0.006s	sys 0.009s
I20260709 13:12:20.096065 25175 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.121482 25237 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.175307 22840 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:20.126814 25105 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:12:20.157853 22836 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:20.123081 25283 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.162163 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.162573 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.182314 22838 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:20.160540 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.182888 22835 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:20.163189 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.183828 22834 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:20.127300 24927 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:12:20.166800 22832 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:20.184626 22906 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:20.167706 25044 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.168326 24872 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:20.138033 25208 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.170068 25175 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:20.171380 25205 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:20.171717 25131 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:20.205792 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:20.161805 24957 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.156836 22909 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:20.138895 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 228900us
W20260709 13:12:20.137362 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 216009us
I20260709 13:12:20.182408 22833 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:20.158776 25139 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.175510 25158 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:20.202914 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.222208 22909 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:12:20.222705 22909 pending_rounds.cc:85] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 4
I20260709 13:12:20.222930 22909 pending_rounds.cc:107] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:12:20.223191 22909 raft_consensus.cc:2889] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:20.166957 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 273518us
I20260709 13:12:20.174068 25303 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.159756 25335 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:20.250695 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 195409us
I20260709 13:12:20.231251 22833 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:20.252465 22833 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:20.231876 22838 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:12:20.253387 22838 pending_rounds.cc:85] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:12:20.253674 22838 pending_rounds.cc:107] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:12:20.253911 22838 raft_consensus.cc:2889] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:20.233351 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:20.254881 22908 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.234922 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:20.236682 24957 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:20.240226 22907 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:20.225996 25158 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:20.245857 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:20.247303 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:20.248744 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.249526 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:20.160705 25355 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.244915 25206 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.214284 25259 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:20.337344 22837 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:20.984505 22832 raft_consensus.cc:2368] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
W20260709 13:12:20.980888 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.289395 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:20.986241 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.986670 22835 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:20.987576 22834 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.386443 22908 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:20.988404 22838 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:12:20.517627 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 380649us
I20260709 13:12:20.257562 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:20.992767 23457 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.993867 25259 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:20.989473 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:20.981853 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 890039us
I20260709 13:12:20.994220 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:20.414341 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 236) took 2661 ms. Trace:
I20260709 13:12:20.259482 22909 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:20.292711 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.026170 22732 leader_election.cc:400] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
W20260709 13:12:21.098704 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 114679us
W20260709 13:12:20.303253 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.162494 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2085) took 1042 ms. Trace:
I20260709 13:12:21.165403 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:20.258635 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.153491 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1238) took 1094 ms. Trace:
I20260709 13:12:21.134814 23457 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:20.392292 22841 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:21.167299 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1241) took 1084 ms. Trace:
W20260709 13:12:20.561415 25351 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:12:21.215000 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:21.102813 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.207687 22732 leader_election.cc:403] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:20.225226 24927 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:21.041723 22840 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 13:12:20.873947 25208 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.004s
I20260709 13:12:21.206794 22833 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.209903 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:21.208257 22907 rpcz_store.cc:276] 0709 13:12:20.120524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.120688 (+   164us) service_pool.cc:224] Handling call
0709 13:12:21.162212 (+1041524us) spinlock_profiling.cc:243] Waited 83.7 ms on lock 0x7b54000fb254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:21.162398 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192623872}
I20260709 13:12:21.206552 22763 rpcz_store.cc:276] 0709 13:12:17.752783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.380856 (+628073us) service_pool.cc:224] Handling call
0709 13:12:18.401475 (+ 20619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:18.401934 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.401977 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:18.402528 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:12:18.991842 (+589314us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:19.193604 (+201762us) log.cc:844] Serialized 14 byte log entry
0709 13:12:19.201159 (+  7555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.201277 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:20.375040 (+1173763us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b240012a328. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea42fb0 00007f68a16b9488 00007f68a16b9be8 00007f68a1691398 00007f68a168c76a 00007f68a168bbb6
0709 13:12:20.413828 (+ 38788us) raft_consensus.cc:1685] finished
0709 13:12:20.413978 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:20.414231 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7276,"prepare.queue_time_us":201996,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":74128,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":13298,"spinlock_wait_cycles":11035008,"thread_start_us":718472,"threads_started":2,"wal-append.queue_time_us":765650}
I20260709 13:12:21.206519 22837 rpcz_store.cc:276] 0709 13:12:20.083063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.133061 (+ 49998us) service_pool.cc:224] Handling call
0709 13:12:21.167002 (+1033941us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:21.167224 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396059008}
I20260709 13:12:20.802464 25360 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:20.535722 25271 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:21.101172 25355 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.212922 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.229112 23457 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:21.229179 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:20.696172 25335 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:20.227427 25175 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:20.622102 25354 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:21.219839 22840 pending_rounds.cc:85] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 2
I20260709 13:12:21.240760 22840 pending_rounds.cc:107] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:12:21.241168 22840 raft_consensus.cc:2889] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:21.242987 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:20.738757 25212 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.006s
W20260709 13:12:21.115105 25310 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.024s	user 0.013s	sys 0.000s
I20260709 13:12:20.372381 25352 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.679123 25205 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.291503 24808 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:21.137218 25139 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.207670 22839 rpcz_store.cc:276] 0709 13:12:20.059340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.114366 (+ 55026us) service_pool.cc:224] Handling call
0709 13:12:21.153151 (+1038785us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b540027f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:21.153393 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341242624}
I20260709 13:12:20.506102 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.141266 25319 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:20.794764 25237 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:20.386969 25274 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.801826 25283 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.097968 25158 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.248509 25175 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.257618 25365 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:21.270725 25365 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.271119 25365 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.271306 22834 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.271871 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1264) took 1025 ms. Trace:
I20260709 13:12:21.272166 22834 rpcz_store.cc:276] 0709 13:12:20.246860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.247014 (+   154us) service_pool.cc:224] Handling call
0709 13:12:21.271804 (+1024790us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:21.211424 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:21.273408 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62078.8us
I20260709 13:12:21.274545 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.275741 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:20.779614 25359 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.000s	sys 0.010s
W20260709 13:12:21.276500 25359 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.000s	sys 0.022s
I20260709 13:12:21.231312 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:21.277794 25319 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:21.278113 25366 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:12:21.223582 22837 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:21.099010 25063 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.123s
I20260709 13:12:20.292055 25009 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.932s
I20260709 13:12:21.204792 25336 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.225261 25360 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:21.238371 25214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.005s	sys 0.020s
I20260709 13:12:20.622313 25358 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 1.012s
W20260709 13:12:21.259330 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 268195us
I20260709 13:12:21.302043 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.259433 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.303881 22766 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:21.236974 24872 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.261318 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:21.252032 25345 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.090s	user 0.011s	sys 0.002s
I20260709 13:12:21.254673 25352 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.227480 25208 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.265926 23458 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.259923 25205 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.262434 25139 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.253245 25212 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:21.239904 25355 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:21.141794 25040 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.413546 22841 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:21.272260 25175 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:21.277972 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46783.6us
I20260709 13:12:21.417276 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:21.280951 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.114835 25211 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.992784 25371 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:21.277104 22833 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.278625 25280 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:21.259717 25162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.008s	sys 0.023s
I20260709 13:12:21.409169 25139 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.300283 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.299930 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.359457 25319 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:12:21.297587 24808 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.309693 24927 ts_tablet_manager.cc:1434] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 1.362s	user 0.012s	sys 0.014s
I20260709 13:12:21.303527 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.332876 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:21.339989 23432 leader_election.cc:336] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.520s (SENT)
I20260709 13:12:21.451421 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.426803 25226 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.422314 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1274) took 1132 ms. Trace:
I20260709 13:12:21.379628 25345 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.425182 22764 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:21.425305 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 144427us
I20260709 13:12:21.454066 22837 rpcz_store.cc:276] 0709 13:12:20.290130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.222622 (+932492us) service_pool.cc:224] Handling call
0709 13:12:21.223164 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.223450 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:21.307249 (+ 83799us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.415420 (+108171us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.421943 (+  6523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:21.422004 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:21.422057 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.422130 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:21.422248 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4076,"prepare.queue_time_us":17094,"raft.queue_time_us":247,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":20136,"spinlock_wait_cycles":8534400,"thread_start_us":74465,"threads_started":1}
I20260709 13:12:21.458674 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.348682 25310 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:21.380928 25206 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:21.441324 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 212274us
I20260709 13:12:21.410511 24872 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:21.443801 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 454279us
I20260709 13:12:21.259883 25313 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.412110 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 290) took 1159 ms. Trace:
I20260709 13:12:21.446390 22761 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:21.296936 25188 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:21.449442 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 345714us
I20260709 13:12:21.444044 25336 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:21.452946 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 113035us
W20260709 13:12:21.298964 25214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.006s	sys 0.020s
W20260709 13:12:21.454381 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 150950us
W20260709 13:12:21.422878 22806 connection.cc:256] RPC call timeout handler was delayed by -3.1090288810000857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25s for a response
W20260709 13:12:21.463145 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 130391us
W20260709 13:12:21.484097 22806 leader_election.cc:336] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.870s (SENT)
I20260709 13:12:21.484572 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:21.441476 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1269) took 1188 ms (client timeout 1533 ms). Trace:
W20260709 13:12:21.485284 22833 rpcz_store.cc:269] 0709 13:12:20.252974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.253127 (+   153us) service_pool.cc:224] Handling call
0709 13:12:21.441398 (+1188271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40064}
I20260709 13:12:21.464214 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.448508 25139 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.411983 25364 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:12:21.467280 22732 leader_election.cc:400] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:21.491331 22732 leader_election.cc:403] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:21.450578 24808 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.442523 25211 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.460444 25242 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:21.456084 25226 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:21.452682 24927 ts_tablet_manager.cc:1467] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Tablet startup took 1507ms
W20260709 13:12:21.493463 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.540475 22806 connection.cc:256] RPC call timeout handler was delayed by -2.1930810249999695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3133s for a response
W20260709 13:12:21.540900 22806 connection.cc:256] RPC call timeout handler was delayed by -1.7881776500000797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 13:12:21.541211 22806 connection.cc:256] RPC call timeout handler was delayed by -1.7458776500000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260709 13:12:21.541482 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5649776500000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
I20260709 13:12:21.471084 25311 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:21.468489 25355 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:21.515316 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54008.9us
I20260709 13:12:21.547065 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:21.445327 25371 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:21.524004 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1270) took 1270 ms. Trace:
I20260709 13:12:21.423964 25360 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:21.526183 25376 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:12:21.526424 25208 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.473740 23458 rpcz_store.cc:276] 0709 13:12:20.250705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.254763 (+  4058us) service_pool.cc:224] Handling call
0709 13:12:21.409685 (+1154922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:21.531792 25303 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.519098 25259 raft_consensus.cc:2764] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:21.535526 25345 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.477873 25313 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.582274 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:21.537585 25211 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.537789 24872 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:21.542094 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5001776500000688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
I20260709 13:12:21.480568 25336 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.488847 25358 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:12:21.521087 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55137.2us
I20260709 13:12:21.589969 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:21.590137 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.540091 25226 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.551192 22832 rpcz_store.cc:276] 0709 13:12:20.253722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.097628 (+843906us) service_pool.cc:224] Handling call
0709 13:12:21.098077 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:21.098526 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:21.286913 (+188387us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.450757 (+163844us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:21.467508 (+ 16751us) log.cc:844] Serialized 26 byte log entry
0709 13:12:21.467969 (+   461us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:21.468018 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.468114 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:21.523673 (+ 55559us) raft_consensus.cc:1685] finished
0709 13:12:21.523707 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:21.523912 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":211327,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":10441,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":29430,"thread_start_us":341191,"threads_started":2,"wal-append.queue_time_us":230369}
I20260709 13:12:21.477733 25325 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:12:21.514657 25224 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:21.555480 25188 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:21.594832 25188 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.556351 25158 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:21.557821 25009 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.558111 25162 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.606639 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.606920 25162 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.607412 22761 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.607662 25371 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.608863 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:21.609124 25162 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:21.559150 25205 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:21.516601 25206 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:12:21.609818 25242 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:21.610127 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:21.559587 22913 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:21.610731 22757 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.529157 24957 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:21.611850 24957 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.614063 24957 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.613967 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.561192 22907 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:21.615417 23459 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:21.590780 25336 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.600678 25188 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.561951 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:21.517916 24808 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.562721 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.598764 25259 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:21.564186 22911 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:21.565253 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:21.618973 22908 raft_consensus.cc:2376] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5. Re-sending same reply.
I20260709 13:12:21.563498 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:21.546499 25377 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:12:21.567739 25214 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:12:21.619964 22909 raft_consensus.cc:2376] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6. Re-sending same reply.
W20260709 13:12:21.586254 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3921776500000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:12:21.620801 25214 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:12:21.621004 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3237776500000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260709 13:12:21.621310 25214 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:21.622426 25214 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.624307 25214 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.540253 25388 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:12:21.573699 25360 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.589361 25358 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.629563 22761 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.630066 25358 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.566934 25343 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:12:21.614851 22758 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 3.
I20260709 13:12:21.633785 25313 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.618167 25131 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.635105 25131 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.556079 25365 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.596660 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:21.646430 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49905.7us
I20260709 13:12:21.647857 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:21.648706 25359 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.649154 25359 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:21.649546 25359 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.650668 24808 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.651952 22906 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:21.653901 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:21.654812 25158 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:21.654855 25139 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.656599 25158 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:21.597405 25271 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:12:21.657364 24957 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.657923 24957 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:21.658337 24957 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.605928 25009 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.609432 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:21.659814 22757 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:21.660187 22766 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.617986 22914 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:21.470968 25310 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.662984 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.663934 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.664966 22837 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.668834 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:21.672999 22757 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.676600 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.681838 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.682356 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.683019 23457 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:21.683955 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.684988 23465 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.686350 25009 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.686689 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.686342 23461 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEARNER]: Advancing to term 5
I20260709 13:12:21.657210 25158 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:21.681474 25352 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.697238 22766 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.698313 25158 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.701527 24834 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:21.539737 24927 ts_tablet_manager.cc:1469] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Trace:
0709 13:12:06.397476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.537135 (+139659us) service_pool.cc:224] Handling call
0709 13:12:07.357373 (+820238us) ts_tablet_manager.cc:934] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
0709 13:12:07.501219 (+143846us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea00caa 00007f689ea00bd2 00007f68a6a65131
0709 13:12:10.441962 (+2940743us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:12:11.119385 (+677423us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:11.123345 (+  3960us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:12:11.751184 (+627839us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:12:11.751547 (+   363us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:11.871350 (+119803us) ts_tablet_manager.cc:989] Replica registered.
0709 13:12:11.892072 (+ 20722us) inbound_call.cc:177] Queueing success response
0709 13:12:16.367486 (+4475414us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:12:19.967007 (+3599521us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:12:20.053430 (+ 86423us) tablet_replica.cc:261] Starting instrumentation
0709 13:12:20.077901 (+ 24471us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":36401,"raft.queue_time_us":4353,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":6134,"spinlock_wait_cycles":3318528,"tablet-copy.queue_time_us":679839,"tablet-download-pool.queue_time_us":36114,"tablet-download-pool.run_cpu_time_us":2843,"tablet-download-pool.run_wall_time_us":2710467,"thread_start_us":698357,"threads_started":2}
I20260709 13:12:21.701422 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.704043 22914 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8670cf017371429ea31529b6f247493e"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:21.704771 22914 raft_consensus.cc:606] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:21.705184 22914 raft_consensus.cc:614] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:21.703526 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.705551 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.709422 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.709478 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.711125 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:21.712210 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.712930 22839 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:21.710585 22766 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.714099 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:21.714569 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:21.718075 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.719157 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.719820 22912 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:21.720662 25224 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:21.720966 22907 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:21.724242 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.725489 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.725929 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.706343 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.726302 25206 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:21.726799 22840 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:21.706420 22837 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:21.727068 25303 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:21.726809 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:21.727770 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.729051 25303 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:21.729130 22840 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:21.729640 25303 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:21.729703 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.730054 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.730954 25208 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:21.730660 25303 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.731029 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:21.731647 25208 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:21.732244 25206 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:21.732666 25208 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.725528 25224 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:12:21.733816 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.734776 25224 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:21.735148 25189 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:21.735998 25189 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:21.736322 25189 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:21.737143 25189 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.725466 22757 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.735155 25224 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.739601 25224 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.738238 22840 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:21.738447 25189 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.740056 22757 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:21.746745 22839 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:21.747968 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.751703 25224 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.754966 25224 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:21.756167 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:21.756856 22915 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:21.759721 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55634.6us
I20260709 13:12:21.760504 22879 leader_election.cc:400] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
I20260709 13:12:21.760785 22879 leader_election.cc:403] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:12:21.764566 23463 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:12:21.767680 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:21.759469 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:21.776284 22915 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:21.778017 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.778146 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.681474 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.815538 22836 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:21.823216 22761 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.806286 25395 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:21.825799 25224 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:21.834767 24808 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:21.757236 24834 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:21.840669 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.803000 25335 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
W20260709 13:12:21.856910 25277 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.016s	sys 0.007s
W20260709 13:12:21.880111 25277 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.016s	sys 0.007s
W20260709 13:12:21.879426 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 119061us
I20260709 13:12:21.803207 23458 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:21.877170 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1272) took 1588 ms. Trace:
I20260709 13:12:21.838110 25009 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.882143 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.882004 22835 rpcz_store.cc:276] 0709 13:12:20.289213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.101479 (+812266us) service_pool.cc:224] Handling call
0709 13:12:21.101968 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.103577 (+  1609us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:21.463289 (+359712us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.876710 (+413421us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:21.876787 (+    77us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:21.876852 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.876924 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:21.877084 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":206615,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":13255,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"thread_start_us":749102,"threads_started":3,"wal-append.queue_time_us":178140}
I20260709 13:12:21.841583 25139 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:21.889197 25009 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.889695 22907 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:21.891378 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 473) took 2024 ms. Trace:
I20260709 13:12:21.919171 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.920464 22838 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:21.796868 25358 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.867743 25395 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:21.929819 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.930950 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.931736 22835 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:21.932220 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.934003 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.888015 25343 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:21.884517 25169 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.945274 22839 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.891306 24834 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:21.936400 25335 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:12:21.947755 25343 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.975821 25188 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.960233 23461 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:21.976634 25343 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.976979 23461 pending_rounds.cc:85] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 2
I20260709 13:12:21.977319 23461 pending_rounds.cc:107] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:12:21.866149 25224 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:21.977609 23461 raft_consensus.cc:2889] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:21.978122 25009 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:21.915859 23459 rpcz_store.cc:276] 0709 13:12:19.867456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:19.880974 (+ 13518us) service_pool.cc:224] Handling call
0709 13:12:21.891102 (+2010128us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540062f554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:21.891304 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3728258816}
I20260709 13:12:21.954758 25400 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:21.941614 25211 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:21.950520 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:21.968585 22837 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:21.936252 25063 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:12:21.961804 24957 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:21.965168 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:21.980306 22913 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:12:21.949671 22908 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:12:21.998416 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:22.000169 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:22.000470 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:22.001662 23463 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:22.002260 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.987999 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1273) took 1698 ms. Trace:
I20260709 13:12:22.003293 22841 rpcz_store.cc:276] 0709 13:12:20.289772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.219555 (+929783us) service_pool.cc:224] Handling call
0709 13:12:21.236064 (+ 16509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.414055 (+177991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.987616 (+573561us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000c62d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:21.987910 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407673216}
I20260709 13:12:21.943239 25271 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:22.005443 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:21.957850 25383 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:22.007122 22835 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:21.955317 25352 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:22.011608 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61168us
I20260709 13:12:22.014005 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:22.016044 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:21.996486 25009 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.020627 22914 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:22.017352 23463 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:21.995045 25188 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:22.006444 25271 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:22.037942 23432 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.194s (SENT)
W20260709 13:12:22.058071 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.075815 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:22.077684 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:21.992122 25360 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:12:22.080989 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:22.012418 25211 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:21.964679 25359 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:22.091012 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53160.2us
I20260709 13:12:22.091228 22841 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:22.008003 25336 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:22.089910 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 69989.3us
I20260709 13:12:22.019541 25383 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:22.093322 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:22.014772 25226 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:22.079901 25271 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.082592 25224 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:22.087317 25188 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.029753 25009 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:22.140386 25402 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:22.090566 25242 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:22.092041 23461 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:12:22.154778 25409 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:22.083282 25365 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:22.142041 23462 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEARNER]: Advancing to term 6
I20260709 13:12:22.142853 22909 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:22.155529 25009 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:22.155966 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:22.156601 22911 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:22.157364 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:22.158792 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-0ad66b5c9842437d9320507b92b78fd5 received from {username='slave'} at 127.0.0.1:51142
W20260709 13:12:22.159380 25406 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:12:22.025190 25395 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:22.143414 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:22.160597 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59010.5us
I20260709 13:12:22.162878 25398 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:22.164089 25404 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:22.164284 25063 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:22.164722 25404 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:22.165032 25063 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:22.165742 25063 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.166074 25404 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:22.150923 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:22.007689 25400 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:22.152594 25188 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:22.159505 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:22.083309 25277 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:22.137158 24834 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:22.064678 25214 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:12:22.174708 22832 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:22.183326 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:22.188203 22909 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:22.188421 22915 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.159399 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-0ad66b5c9842437d9320507b92b78fd5 on tablet 0ad66b5c9842437d9320507b92b78fd5 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:22.180565 25277 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.179625 25335 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:22.178860 24834 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.203778 22915 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:22.182210 25404 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:22.187435 25211 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:22.207034 25405 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:12:22.203217 25335 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.204773 25277 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:22.217334 24834 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.213620 25358 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:22.207094 24957 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:22.229202 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 745954us
W20260709 13:12:22.229900 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 762759us
W20260709 13:12:22.229903 23432 leader_election.cc:336] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.138s (SENT)
I20260709 13:12:22.238766 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:22.220963 25158 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:22.238317 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:22.238224 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:22.241225 25360 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:22.227272 25335 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:22.241745 25410 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:22.263032 22835 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:12:22.264147 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:22.267082 25407 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:12:22.267853 25271 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:22.180958 25383 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:22.276845 25271 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:22.277540 25188 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:22.278283 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:22.265138 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:22.257684 25158 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:22.292793 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:22.295051 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:22.346752 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:22.324350 23432 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.286s (SENT)
I20260709 13:12:22.593483 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:22.607151 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 423884us
I20260709 13:12:22.622617 22837 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:22.605830 22910 raft_consensus.cc:2376] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7. Re-sending same reply.
W20260709 13:12:22.623569 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 299266us
I20260709 13:12:22.624243 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:22.624648 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:22.556748 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 393534us
I20260709 13:12:22.383159 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:22.626585 22915 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:22.626410 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:22.542580 25412 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:22.627228 25386 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:22.627661 25412 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:22.410502 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.628108 25386 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.554260 25310 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:22.628813 25386 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:22.509408 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:22.631428 22907 raft_consensus.cc:2376] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7. Re-sending same reply.
W20260709 13:12:22.536571 25413 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.309s	user 0.000s	sys 0.002s
I20260709 13:12:22.481761 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:22.653483 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:22.403817 22909 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:22.401683 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2094) took 2112 ms. Trace:
I20260709 13:12:22.656586 22906 rpcz_store.cc:276] 0709 13:12:20.289270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.292268 (+  2998us) service_pool.cc:224] Handling call
0709 13:12:21.545045 (+1252777us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:21.545290 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.652364 (+107074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.401284 (+748920us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54001b9ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:22.401580 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247589504}
I20260709 13:12:22.648483 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:22.485251 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:22.644743 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.661965 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:22.662935 22840 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:22.665755 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:22.670603 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:22.684863 22839 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:22.685447 22839 pending_rounds.cc:85] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 3
I20260709 13:12:22.685804 22839 pending_rounds.cc:107] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:22.686141 22839 raft_consensus.cc:2889] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:22.589319 25131 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:22.601432 25169 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:22.640841 25158 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:22.294363 25336 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:22.324923 25359 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:22.694842 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.696738 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:22.639922 25226 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:22.705470 25169 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.677179 25405 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 13:12:22.745013 25423 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:12:22.587625 24834 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:22.739892 25188 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:22.742698 25224 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:22.754457 23458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.035s	sys 0.028s
I20260709 13:12:22.739208 25175 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:12:22.735323 25364 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:22.875943 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:22.879006 22913 raft_consensus.cc:2368] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7.
I20260709 13:12:22.754895 24957 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.116s
W20260709 13:12:22.805985 25404 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.001s
I20260709 13:12:22.771872 25359 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:22.851140 23458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.035s	sys 0.028s
I20260709 13:12:22.679157 25063 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:22.797725 25418 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 13:12:22.829468 25188 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:22.830241 25224 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.908612 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:22.908923 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:22.828660 25211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.000s	sys 0.013s
W20260709 13:12:22.875408 25394 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:12:22.928977 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1275) took 1321 ms. Trace:
I20260709 13:12:22.887992 23459 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:22.864766 25411 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:22.816956 25226 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.091s
W20260709 13:12:22.937776 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 699650us
I20260709 13:12:22.927424 25400 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:22.929035 25421 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.296s	user 0.000s	sys 0.002s
I20260709 13:12:22.931072 25373 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:22.936187 25335 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.003s
I20260709 13:12:22.980271 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:22.980784 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:22.937170 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 698932us
I20260709 13:12:22.998762 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:22.967303 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:22.999975 22837 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:22.970641 25389 raft_consensus.cc:2764] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:22.991978 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:22.970742 25224 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.012939 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:22.970191 24872 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:23.013191 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 246) took 1061 ms. Trace:
I20260709 13:12:22.968168 25188 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.978910 25411 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:23.023571 22913 rpcz_store.cc:276] 0709 13:12:21.951843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.481357 (+529514us) service_pool.cc:224] Handling call
0709 13:12:23.012961 (+531604us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:23.013123 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515327360}
I20260709 13:12:22.989162 25237 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:23.033262 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 407832us
I20260709 13:12:23.030293 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:23.026840 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 378495us
I20260709 13:12:23.049240 22730 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:23.049729 22730 leader_election.cc:403] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:12:23.054914 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.025120 24872 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:23.056062 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 431480us
I20260709 13:12:23.026409 25355 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:12:23.055645 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 319771us
W20260709 13:12:23.065693 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 667711us
W20260709 13:12:23.063428 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 323444us
I20260709 13:12:23.049427 25283 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.071434 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:23.007166 25335 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.049079 25364 raft_consensus.cc:2764] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:23.052842 25389 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:23.004544 25373 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.077432 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:23.072285 25415 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 13:12:23.057734 25313 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.058787 25387 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:23.061528 25325 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:23.069281 24872 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.069976 25355 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.107836 25355 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:23.082090 25389 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.109028 25389 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.058305 25371 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:12:23.086791 25404 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:23.102270 25416 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.333s	user 0.001s	sys 0.001s
I20260709 13:12:23.093931 25400 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:12:23.057052 25237 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.128679 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 490) took 1120 ms. Trace:
I20260709 13:12:23.087041 25398 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:12:23.103986 25387 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.084753 25283 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:23.116160 25343 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.139s	user 0.018s	sys 0.000s
I20260709 13:12:22.937443 25259 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:12:23.105386 24834 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.092929 25280 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:12:23.121058 25404 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.136121 25387 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.087080 25411 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.147578 25404 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:23.155463 25277 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.916s	user 0.009s	sys 0.005s
I20260709 13:12:23.117118 25435 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:12:23.139443 25343 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.163803 25435 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:23.164266 25435 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:23.168291 25411 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:23.171772 25426 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:12:23.165186 25435 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.171617 25373 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.176107 25435 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.175379 25412 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.165408 25343 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:23.178026 25412 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:23.183538 25425 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.133s	user 0.003s	sys 0.000s
I20260709 13:12:23.166374 25433 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:23.194419 25433 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:23.195704 25433 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.192094 25389 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.200434 25389 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:23.200714 25427 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
I20260709 13:12:23.203024 25433 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:23.243579 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.258222 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 293) took 1034 ms. Trace:
I20260709 13:12:23.259117 22836 rpcz_store.cc:276] 0709 13:12:22.223700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.397757 (+174057us) service_pool.cc:224] Handling call
0709 13:12:22.398214 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:22.697925 (+299711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.697958 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.698434 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:12:22.963211 (+264777us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:22.963277 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.965265 (+  1988us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.187956 (+222691us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:23.256936 (+ 68980us) raft_consensus.cc:1685] finished
0709 13:12:23.257001 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.257805 (+   804us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":299136,"raft.queue_time_us":340725,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":13817,"spinlock_wait_cycles":4381952,"thread_start_us":264405,"threads_started":1,"wal-append.queue_time_us":378841}
I20260709 13:12:23.265192 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 287) took 1273 ms. Trace:
W20260709 13:12:23.265729 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 186829us
I20260709 13:12:23.265887 22841 rpcz_store.cc:276] 0709 13:12:21.992117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.137986 (+145869us) service_pool.cc:224] Handling call
0709 13:12:22.138385 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:22.138873 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.138902 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.139556 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 13:12:22.604693 (+465137us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:22.604743 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.604827 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.264428 (+659601us) raft_consensus.cc:1685] finished
0709 13:12:23.264483 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.264904 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":47764,"threads_started":1,"wal-append.queue_time_us":665411}
I20260709 13:12:23.270841 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:23.275089 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.279840 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.281558 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:23.122859 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51125.7us
I20260709 13:12:23.282708 22761 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
W20260709 13:12:23.283846 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 201198us
I20260709 13:12:23.284354 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:23.287649 23432 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.967s (SENT)
W20260709 13:12:23.288718 23432 leader_election.cc:336] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.164s (SENT)
I20260709 13:12:23.129416 23458 rpcz_store.cc:276] 0709 13:12:22.008229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.010312 (+  2083us) service_pool.cc:224] Handling call
0709 13:12:22.093376 (+ 83064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:22.166057 (+ 72681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.166098 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.166953 (+   855us) log.cc:844] Serialized 26 byte log entry
0709 13:12:22.968908 (+801955us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:22.972155 (+  3247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.991033 (+ 18878us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.128314 (+137281us) raft_consensus.cc:1685] finished
0709 13:12:23.128355 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.128573 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20790,"raft.queue_time_us":25241,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5654,"thread_start_us":767604,"threads_started":1,"wal-append.queue_time_us":896319}
I20260709 13:12:23.293881 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:23.243579 22660 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 2 to 4, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:23.301764 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 59009.8us
I20260709 13:12:23.302794 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 257) took 1219 ms. Trace:
I20260709 13:12:22.971570 22759 rpcz_store.cc:276] 0709 13:12:21.607710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.614963 (+  7253us) service_pool.cc:224] Handling call
0709 13:12:21.615305 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.615522 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:21.615857 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:12:22.928628 (+1312771us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:22.928677 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:22.928722 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.928789 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.928916 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1162789,"prepare.queue_time_us":17309,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":1172177,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7313,"thread_start_us":149607,"threads_started":1,"wal-append.queue_time_us":261596,"wal-append.run_cpu_time_us":1709,"wal-append.run_wall_time_us":1009316}
I20260709 13:12:23.305333 24808 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:23.305855 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.306185 24834 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.285207 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:23.289291 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.380216 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.387789 22761 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:23.398423 22762 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.009s	sys 0.013s
W20260709 13:12:23.399204 22762 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.010s	sys 0.013s
I20260709 13:12:23.402448 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.404547 25009 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:23.411679 25063 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:23.412086 22664 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 243) took 23444 ms (client timeout 15000 ms). Trace:
I20260709 13:12:23.428772 25371 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:23.430111 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119134us
I20260709 13:12:23.430544 25371 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.431238 25352 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:23.432356 25352 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.405162 25009 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:23.441744 25175 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.442291 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:23.319856 22911 rpcz_store.cc:276] 0709 13:12:22.082719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.483578 (+400859us) service_pool.cc:224] Handling call
0709 13:12:22.484089 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:22.484440 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:22.637812 (+153372us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.081820 (+444008us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.082379 (+   559us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.187142 (+104763us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b180008d8e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:23.298503 (+111361us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b180008d8e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:23.301745 (+  3242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:23.301864 (+   119us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:23.302013 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.302163 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.302470 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22158,"prepare.queue_time_us":165998,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":7047,"raft.queue_time_us":244716,"raft.run_cpu_time_us":642,"raft.run_wall_time_us":11886,"spinlock_wait_cycles":34438400,"thread_start_us":546916,"threads_started":2,"wal-append.queue_time_us":518586}
I20260709 13:12:23.464406 22763 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:23.464617 25364 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:23.465301 25358 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:23.465955 25358 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:23.465823 25364 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.466703 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.468181 22763 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:23.467869 25358 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:23.473826 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 188911us
W20260709 13:12:23.473917 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 316918us
I20260709 13:12:23.499037 22761 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:23.470774 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1277) took 1862 ms. Trace:
I20260709 13:12:23.469491 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.478185 25438 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:23.501996 25358 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:23.479518 25445 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:12:23.472035 25386 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.822s	user 0.004s	sys 0.014s
I20260709 13:12:23.500753 22763 raft_consensus.cc:2376] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6. Re-sending same reply.
I20260709 13:12:23.478385 25373 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:23.479321 25418 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.478363 24808 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:23.516968 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.469327 25364 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.517560 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:23.518594 25373 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:23.519095 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 138044us
W20260709 13:12:23.519604 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.519814 25434 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.548139 25434 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.551137 24808 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.551949 25352 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.477703 25310 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.553180 25310 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.506080 25438 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.553968 25459 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.554337 25438 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.523439 25442 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:23.524446 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.526548 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:23.477941 25325 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:12:23.589915 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52652.8us
I20260709 13:12:23.590281 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:23.591431 23431 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.967s (SENT)
I20260709 13:12:23.591981 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:23.593981 25271 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:23.594858 25271 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:23.596577 25271 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.528743 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.528677 25436 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:23.530871 22909 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:23.472816 22660 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 2, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:23.532390 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.541736 25418 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.543977 25404 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.515264 25175 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:23.545866 25345 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:23.620088 25313 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.005s
I20260709 13:12:23.498296 25395 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:12:23.620687 25313 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:23.445567 25277 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.622066 25175 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.623772 25277 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:23.541808 22914 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:23.499742 22758 rpcz_store.cc:276] 0709 13:12:21.608258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.631974 (+ 23716us) service_pool.cc:224] Handling call
0709 13:12:21.632464 (+   490us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:22.905681 (+1273217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.905726 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:22.931961 (+ 26235us) log.cc:844] Serialized 52 byte log entry
0709 13:12:23.073101 (+141140us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:23.196556 (+123455us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:23.367854 (+171298us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.469689 (+101835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.469859 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.469957 (+    98us) raft_consensus.cc:1685] finished
0709 13:12:23.470078 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.470418 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1370427,"prepare.queue_time_us":422535,"raft.queue_time_us":1387596,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":177,"spinlock_wait_cycles":77719680,"thread_start_us":205366,"threads_started":2,"wal-append.queue_time_us":207728}
I20260709 13:12:23.625564 22660 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:23.627082 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.627784 22758 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:23.632326 25418 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:23.633107 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 246) took 2370 ms. Trace:
I20260709 13:12:23.633808 25389 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:23.634712 25404 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.634541 25389 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.633571 22762 rpcz_store.cc:276] 0709 13:12:21.262227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.262361 (+   134us) service_pool.cc:224] Handling call
0709 13:12:21.281441 (+ 19080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:21.424122 (+142681us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:21.489752 (+ 65630us) log.cc:844] Serialized 14 byte log entry
0709 13:12:22.734431 (+1244679us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.744307 (+  9876us) log.cc:844] Serialized 26 byte log entry
0709 13:12:22.744544 (+   237us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:23.628762 (+884218us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180018c700. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf
0709 13:12:23.628965 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.629359 (+   394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.629465 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.632748 (+  3283us) raft_consensus.cc:1685] finished
0709 13:12:23.632764 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.632983 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1586640,"prepare.queue_time_us":475922,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":103,"raft.queue_time_us":1209174,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":74735,"spinlock_wait_cycles":24774400,"thread_start_us":868537,"threads_started":4,"wal-append.queue_time_us":282506}
I20260709 13:12:23.461899 25009 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.559890 25226 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.569326 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.645581 22759 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:23.576431 25310 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.556734 25442 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:23.578114 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 289415us
I20260709 13:12:23.579298 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1271) took 1972 ms. Trace:
I20260709 13:12:23.659360 22760 rpcz_store.cc:276] 0709 13:12:21.606479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.608467 (+  1988us) service_pool.cc:224] Handling call
0709 13:12:21.608835 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:22.762855 (+1154020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.762906 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.874853 (+111947us) log.cc:844] Serialized 26 byte log entry
0709 13:12:23.516870 (+642017us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:23.541061 (+ 24191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.541176 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.578813 (+ 37637us) raft_consensus.cc:1685] finished
0709 13:12:23.578842 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.579055 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1279954,"raft.queue_time_us":1160987,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":27179,"thread_start_us":493436,"threads_started":1,"wal-append.queue_time_us":646371}
I20260709 13:12:23.660681 25398 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:23.579792 22910 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:23.580087 25283 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.662782 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.663676 22910 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:23.664057 25404 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:23.665107 25237 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.586004 25386 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:23.670327 23432 leader_election.cc:336] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.529s (SENT)
I20260709 13:12:23.670971 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.672273 25386 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:23.586616 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.554s (SENT)
W20260709 13:12:23.674526 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 88028.9us
I20260709 13:12:23.674927 25009 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:23.675405 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.612s (SENT)
I20260709 13:12:23.587294 22913 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:23.588310 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62031.4us
I20260709 13:12:23.677932 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:23.527962 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.679179 25418 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:23.679637 25418 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:23.680039 25418 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:23.597981 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73773.8us
I20260709 13:12:23.598289 25214 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:23.683153 25214 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:23.673300 25386 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.610803 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.617043 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.687426 22906 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:23.619009 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.687958 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:23.578132 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:23.689260 25395 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:23.689724 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.690263 25439 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:23.629204 22660 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 2 to 3, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:23.690620 25395 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:23.690867 25439 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.691139 25395 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:23.691382 25439 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.692302 25395 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.681248 22911 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:23.648038 25271 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.648895 25226 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.650352 25373 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:23.412631 22664 rpcz_store.cc:269] 0709 13:11:59.967468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.967705 (+   237us) service_pool.cc:224] Handling call
0709 13:12:23.411756 (+23444051us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:15.132164 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:15.133043 (+   879us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:15.133096 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:12:15.133113 (+    17us) tablet.cc:662] Decoding operations
0709 13:12:15.297927 (+164814us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:15.298004 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:15.298039 (+    35us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:12:15.299622 (+  1583us) tablet.cc:701] Row locks acquired
0709 13:12:15.299671 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:12:15.299908 (+   237us) write_op.cc:270] Start()
0709 13:12:15.300079 (+   171us) write_op.cc:276] Timestamp: P: 1783602735299825 usec, L: 0
0709 13:12:15.300103 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:12:15.301130 (+  1027us) log.cc:844] Serialized 6927 byte log entry
0709 13:12:16.244053 (+942923us) op_driver.cc:464] REPLICATION: finished
0709 13:12:17.747195 (+1503142us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a5527ef0
0709 13:12:17.828146 (+ 80951us) write_op.cc:301] APPLY: starting
0709 13:12:17.828244 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:17.832186 (+  3942us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:17.832231 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:17.835112 (+  2881us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:17.835607 (+   495us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:12:17.835688 (+    81us) write_op.cc:312] APPLY: finished
0709 13:12:17.836330 (+   642us) log.cc:844] Serialized 119 byte log entry
0709 13:12:18.464322 (+627992us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:18.488560 (+ 24238us) write_op.cc:454] Released schema lock
0709 13:12:18.495451 (+  6891us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:18.496019 (+   568us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:18.496064 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1552369,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":1578003,"apply.run_cpu_time_us":60154,"apply.run_wall_time_us":667823,"num_ops":17,"prepare.queue_time_us":440747,"prepare.run_cpu_time_us":23297,"prepare.run_wall_time_us":587750,"raft.queue_time_us":236594,"raft.run_cpu_time_us":38999,"raft.run_wall_time_us":1728814,"replication_time_us":919872,"spinlock_wait_cycles":18931456,"thread_start_us":2911134,"threads_started":5,"wal-append.queue_time_us":1444479,"wal-append.run_cpu_time_us":22354,"wal-append.run_wall_time_us":2397015}]]}
I20260709 13:12:23.552093 25434 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.661134 25335 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.470345 24834 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.665340 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.665668 25404 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:23.665861 25442 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:23.673478 25277 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:23.598865 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 156712us
I20260709 13:12:23.689845 25310 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:23.628827 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.620309 25345 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.690621 22763 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:23.652683 25371 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:23.705035 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.652395 25188 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.005s
I20260709 13:12:23.692739 22660 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 4, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:23.708292 22910 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:23.712179 22761 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:23.712934 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:23.712896 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.659s (SENT)
I20260709 13:12:23.636749 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.709448 25448 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.727649 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:23.730000 22760 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:23.712699 25404 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:23.738077 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.743124 24834 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.744159 25226 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.744704 25404 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.743422 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:23.737278 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:23.749958 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.752367 25435 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:23.753458 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.730511 25434 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:23.757884 25169 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.940s	user 0.010s	sys 0.010s
I20260709 13:12:23.758602 25389 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.758366 25434 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.758569 25169 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.760181 22660 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:23.761734 22758 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:23.761929 22909 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:12:23.762969 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.763898 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.763690 22660 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:23.765076 25169 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.765702 22759 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:23.764894 22912 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:23.768311 25345 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.770208 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.771499 22910 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:23.763692 22758 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.777762 25371 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:23.778254 25371 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:23.778698 25371 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:23.779377 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.781227 25389 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:23.739602 25188 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.782394 25345 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:23.757979 25311 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:23.782789 22914 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:23.783299 25311 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:23.785226 25455 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.118s	user 0.003s	sys 0.001s
I20260709 13:12:23.784641 25311 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.761248 25434 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.773214 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:23.789122 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:23.802639 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1285) took 1955 ms. Trace:
I20260709 13:12:23.809203 22766 rpcz_store.cc:276] 0709 13:12:21.657488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.723655 (+ 66167us) service_pool.cc:224] Handling call
0709 13:12:21.724070 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:22.928618 (+1204548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.928648 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:22.929283 (+   635us) log.cc:844] Serialized 52 byte log entry
0709 13:12:23.254434 (+325151us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:23.371738 (+117304us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.507950 (+136212us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.508256 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.515876 (+  7620us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.521242 (+  5366us) raft_consensus.cc:1685] finished
0709 13:12:23.522713 (+  1471us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.612686 (+ 89973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1361629,"prepare.queue_time_us":336197,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":111,"raft.queue_time_us":1317827,"raft.run_cpu_time_us":556,"raft.run_wall_time_us":38888,"thread_start_us":383018,"threads_started":2,"wal-append.queue_time_us":188338}
I20260709 13:12:23.800019 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.803747 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:23.804946 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 152773us
W20260709 13:12:23.788285 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52830.2us
I20260709 13:12:23.789311 25434 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:12:23.837246 23463 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:12:23.837764 23463 pending_rounds.cc:85] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 5
I20260709 13:12:23.837999 23463 pending_rounds.cc:107] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:12:23.838259 23463 raft_consensus.cc:2889] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:23.840554 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.793603 25448 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.844130 25435 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:23.844672 25435 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.845081 25435 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:23.847052 25224 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.818s	user 0.012s	sys 0.008s
I20260709 13:12:23.847631 25224 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:23.849340 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 139625us
I20260709 13:12:23.826153 25310 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.851203 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:23.852082 23461 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:23.852411 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:23.852824 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1270) took 2250 ms. Trace:
I20260709 13:12:23.857317 22913 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:23.853912 24872 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.542s	user 0.021s	sys 0.000s
I20260709 13:12:23.863075 25448 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:23.785466 25188 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:23.797645 25387 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.583s	user 0.004s	sys 0.010s
I20260709 13:12:23.787186 25311 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.788070 25395 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.874222 25162 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:23.863442 24872 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:23.874486 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:23.874719 25433 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.008s
I20260709 13:12:23.876912 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.878106 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.879944 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:23.903280 22834 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:23.880697 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:23.881805 25345 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:23.921908 25345 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.922547 25345 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:23.922602 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57235.8us
I20260709 13:12:23.886266 22760 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:23.925117 25360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.003s	sys 0.019s
W20260709 13:12:23.925676 25360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.004s	sys 0.019s
I20260709 13:12:23.891358 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:23.894865 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:23.926505 22761 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:23.872632 25398 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:12:23.927739 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.928333 25398 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:23.928766 25398 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:23.897118 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:23.932384 22759 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:23.908675 25404 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.936604 25224 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.909349 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:23.890563 25310 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:23.938251 25310 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.938763 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:23.939556 22841 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.939698 22914 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:23.910315 22838 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.942642 25435 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:23.945839 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46684.2us
I20260709 13:12:23.950049 22912 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:23.888347 22915 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:23.910094 25387 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:23.917989 22909 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:23.922034 25355 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.813s	user 0.008s	sys 0.011s
W20260709 13:12:23.917771 23432 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.950s (SENT)
I20260709 13:12:23.960966 22841 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:23.992718 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:23.989019 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:24.017844 22913 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:23.853231 22765 rpcz_store.cc:276] 0709 13:12:21.602447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.602605 (+   158us) service_pool.cc:224] Handling call
0709 13:12:21.603046 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:22.905259 (+1302213us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:23.028818 (+123559us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.707034 (+678216us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.718472 (+ 11438us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:23.784294 (+ 65822us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:23.850638 (+ 66344us) log.cc:844] Serialized 26 byte log entry
0709 13:12:23.852151 (+  1513us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:23.852196 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.852289 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.852420 (+   131us) raft_consensus.cc:1685] finished
0709 13:12:23.852436 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.852615 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1198703,"prepare.queue_time_us":600312,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":4854,"raft.queue_time_us":1337711,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":82132,"spinlock_wait_cycles":60566400,"thread_start_us":585650,"threads_started":3,"wal-append.queue_time_us":604289}
I20260709 13:12:23.948813 25435 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:23.929875 25398 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.026120 22757 raft_consensus.cc:2368] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7.
I20260709 13:12:24.085407 25418 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.157291 22763 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:24.156524 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:24.038890 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50003us
I20260709 13:12:24.173795 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.076437 25459 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.029968 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:24.210789 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.211731 22907 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:24.213438 22909 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:24.213910 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.216063 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:24.054946 25434 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:24.217603 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:24.230336 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:24.222164 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 327393us
I20260709 13:12:24.232370 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:24.216109 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.225632 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:24.222663 22765 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:24.244192 22761 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:24.240726 22906 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
W20260709 13:12:24.246928 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.004423 25224 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.208s
W20260709 13:12:24.255200 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.249258 22763 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:24.250272 22758 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:24.253057 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:24.254119 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67872.1us
I20260709 13:12:24.256878 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1297) took 1002 ms. Trace:
I20260709 13:12:24.257349 22758 rpcz_store.cc:276] 0709 13:12:23.254795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.201981 (+947186us) service_pool.cc:224] Handling call
0709 13:12:24.256793 (+ 54812us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:24.260044 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 357649us
W20260709 13:12:24.239169 25442 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.001s
I20260709 13:12:24.273798 22766 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:24.278288 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:24.278963 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.279903 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:24.285277 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.034114 25355 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.288003 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:24.289103 23461 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.296085 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:24.236150 25434 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.301745 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.302886 22761 raft_consensus.cc:2410] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:24.309727 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.262316 25310 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.315481 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:24.297273 22806 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.681s (SENT) [suppressed 56 similar messages]
I20260709 13:12:24.309219 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:24.322717 22766 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:24.308000 25373 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.322257 22838 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:24.331947 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.222280 25459 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.350497 22765 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:24.337891 23463 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:24.334929 22759 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:24.324306 22763 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:24.360911 25464 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.294261 25435 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:24.373009 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.311254 25434 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:24.317243 25398 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:24.294008 25169 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.320387 25433 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:24.284394 25442 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.237360 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:24.392297 22758 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:12:24.396282 22806 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.860s (SENT)
W20260709 13:12:24.407768 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.411762 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:24.412662 23462 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:24.412868 25438 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.858s	user 0.009s	sys 0.009s
I20260709 13:12:24.421909 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2032) took 1112 ms. Trace:
I20260709 13:12:24.422329 22758 rpcz_store.cc:276] 0709 13:12:23.309289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.327881 (+1018592us) service_pool.cc:224] Handling call
0709 13:12:24.421841 (+ 93960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.424695 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:24.402191 25371 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.615s	user 0.020s	sys 0.000s
I20260709 13:12:24.428210 25389 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:24.428830 25158 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.534s	user 0.010s	sys 0.025s
I20260709 13:12:24.423668 25009 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:12:24.384222 25313 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.380259 25310 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.434371 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2021) took 1171 ms. Trace:
I20260709 13:12:24.440362 22759 pending_rounds.cc:85] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 2
I20260709 13:12:24.372107 25345 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.396004 25259 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51126 (request call id 6) took 35789 ms. Trace:
I20260709 13:12:24.448721 22915 raft_consensus.cc:606] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:24.449378 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:24.443444 25465 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.227s	user 0.003s	sys 0.006s
I20260709 13:12:24.455468 22759 pending_rounds.cc:107] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 2.3
W20260709 13:12:24.456389 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.457182 25158 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.448501 25373 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.453605 22757 rpcz_store.cc:276] 0709 13:12:23.247368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.249695 (+  2327us) service_pool.cc:224] Handling call
0709 13:12:24.406124 (+1156429us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54001fa254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:24.418221 (+ 12097us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703672192}
I20260709 13:12:24.433605 25418 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.438824 24872 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.446558 25387 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.472569 22908 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:24.425987 25169 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.479820 22759 raft_consensus.cc:2889] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:24.469621 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.448256 25389 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.413409 25438 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.489048 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:24.490137 22759 pending_rounds.cc:107] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:12:24.492328 22759 raft_consensus.cc:2889] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:24.493809 23462 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:24.495692 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:24.445787 25464 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.498595 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2051) took 1054 ms. Trace:
I20260709 13:12:24.508083 22764 rpcz_store.cc:276] 0709 13:12:23.444241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.264082 (+819841us) service_pool.cc:224] Handling call
0709 13:12:24.498506 (+234424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.451704 25433 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.473683 25158 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.510007 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:24.504106 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:24.511300 22757 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:24.503734 22763 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:24.513597 25373 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:24.513850 25395 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.005s
I20260709 13:12:24.517858 25283 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:24.516655 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1295) took 1263 ms. Trace:
W20260709 13:12:24.519068 25439 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.008s
I20260709 13:12:24.520210 22911 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:24.518718 25283 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:24.522101 25009 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:24.519614 22763 rpcz_store.cc:276] 0709 13:12:23.253676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.269245 (+1015569us) service_pool.cc:224] Handling call
0709 13:12:24.516389 (+247144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.462293 25259 rpcz_store.cc:276] 0709 13:11:48.606372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.606515 (+   143us) service_pool.cc:224] Handling call
0709 13:11:48.714460 (+107945us) log.cc:844] Serialized 492 byte log entry
0709 13:11:48.930691 (+216231us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b180003dcc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1a4ab0b 00007f68a1acfb63 00007f68a1af7752 00007f68a6ab7d29
0709 13:12:24.385899 (+35455208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17275,"raft.queue_time_us":76929,"raft.run_cpu_time_us":2718,"raft.run_wall_time_us":79898,"spinlock_wait_cycles":4247808,"thread_start_us":249165,"threads_started":2,"wal-append.queue_time_us":179362,"wal-append.run_cpu_time_us":2283,"wal-append.run_wall_time_us":1024375}
I20260709 13:12:24.524778 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:24.463323 25313 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.526620 22763 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:24.453759 25371 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.466002 25310 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.533206 25283 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.497848 25466 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:24.545288 25466 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:24.546386 23462 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:24.560988 22760 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 3.
I20260709 13:12:24.526846 25459 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.521226 25438 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:24.528445 25387 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.570530 22762 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:24.528939 25355 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:24.567194 25271 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.873s	user 0.004s	sys 0.012s
I20260709 13:12:24.519694 25439 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.605726 25169 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.609452 24872 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.662045 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:24.665684 23462 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:24.690580 25371 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.691000 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:24.702344 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af39123de9434cc0b451f4b03d305310"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43260
I20260709 13:12:24.703464 22915 raft_consensus.cc:606] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:12:24.703930 22915 raft_consensus.cc:614] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:24.705061 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:24.706009 22913 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:24.691800 25476 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.072s	user 0.000s	sys 0.002s
I20260709 13:12:24.721438 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:24.569010 25466 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.620683 25387 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.664489 25442 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:24.731459 25473 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:12:24.734565 22908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 248) took 2782 ms (client timeout 3079 ms). Trace:
I20260709 13:12:24.734036 22766 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:24.683543 25169 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:24.736264 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2138) took 1029 ms. Trace:
W20260709 13:12:24.741283 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 245820us
W20260709 13:12:24.742997 23432 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.622s (SENT)
I20260709 13:12:24.668411 25438 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.754267 22911 rpcz_store.cc:276] 0709 13:12:23.706954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.710918 (+  3964us) service_pool.cc:224] Handling call
0709 13:12:24.736001 (+1025083us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b540010efd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:24.736188 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802187648}
W20260709 13:12:24.755014 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:24.749702 22908 rpcz_store.cc:269] 0709 13:12:21.952513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:22.484899 (+532386us) service_pool.cc:224] Handling call
0709 13:12:24.734251 (+2249352us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54002116d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:24.734465 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4167472512}
W20260709 13:12:24.764149 25404 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.003s
I20260709 13:12:24.758715 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:24.769369 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.683947 24872 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.747303 25345 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.781919 25271 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.812585 25271 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:24.831027 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.753661 25371 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.833479 22913 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:24.823818 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 97124.5us
I20260709 13:12:24.774935 25169 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:24.846179 23431 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.622s (SENT)
I20260709 13:12:24.850026 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:24.851265 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.823138 25389 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.801342 25438 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:24.838551 25483 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:24.860965 23461 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.571s	user 0.017s	sys 0.000s
I20260709 13:12:24.855365 25435 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:24.892305 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:24.874333 25311 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.978s	user 0.009s	sys 0.009s
I20260709 13:12:24.868252 25404 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:24.868641 25433 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.901613 22834 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:24.886322 25310 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.860975 25371 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.900187 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:24.911209 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:24.891409 25464 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.897012 25439 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:24.928663 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 538813us
I20260709 13:12:24.908432 25486 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:24.913169 25271 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.932668 23461 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:12:24.953296 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107466us
I20260709 13:12:24.953789 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:24.966988 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.932s (SENT)
I20260709 13:12:24.967636 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:24.949440 25404 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:24.919919 25483 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.981328 22835 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:24.960039 25448 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.989879 25360 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:24.989244 25448 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.991622 25483 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.992342 25458 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:24.993362 25458 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.934511 25169 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:24.998555 25371 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:24.994971 22832 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 4.
I20260709 13:12:24.999570 25458 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:24.999605 22839 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:25.001212 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:24.995090 25439 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:24.982712 25435 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:25.003399 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 56833.5us
W20260709 13:12:25.004179 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.052s (SENT)
I20260709 13:12:25.004823 25404 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:25.010381 25438 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:25.011049 22839 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:25.012300 22840 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:25.012997 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:25.013599 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 498) took 1055 ms. Trace:
I20260709 13:12:24.972764 25387 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:25.017382 23432 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.750s (SENT)
I20260709 13:12:24.984714 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:25.020319 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:25.048164 22833 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 4: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:25.046404 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:25.058496 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:25.059321 22833 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:25.026021 24872 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:25.064689 25398 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.002s
I20260709 13:12:25.018077 23458 rpcz_store.cc:276] 0709 13:12:23.849876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.853110 (+  3234us) service_pool.cc:224] Handling call
0709 13:12:23.853596 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:23.882223 (+ 28627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.882270 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:23.883011 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 13:12:24.259737 (+376726us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:24.265662 (+  5925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.283181 (+ 17519us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:24.889320 (+606139us) raft_consensus.cc:1685] finished
0709 13:12:24.893235 (+  3915us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:24.905028 (+ 11793us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8911,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":227,"thread_start_us":343051,"threads_started":1,"wal-append.queue_time_us":662592}
W20260709 13:12:25.065267 25480 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
I20260709 13:12:25.053959 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:25.070587 22835 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:25.072319 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:25.073148 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:25.074064 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1290) took 1823 ms. Trace:
I20260709 13:12:25.075042 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:25.076045 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1291) took 1824 ms. Trace:
I20260709 13:12:25.076324 22834 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:12:25.076484 22758 rpcz_store.cc:276] 0709 13:12:23.251511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.422957 (+1171446us) service_pool.cc:224] Handling call
0709 13:12:24.448621 (+ 25664us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:24.848620 (+399999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.848660 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:24.893344 (+ 44684us) log.cc:844] Serialized 544 byte log entry
0709 13:12:24.940023 (+ 46679us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:24.940072 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.940158 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:25.075775 (+135617us) raft_consensus.cc:1685] finished
0709 13:12:25.075808 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.075965 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71920,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":4994,"thread_start_us":409258,"threads_started":2,"wal-append.queue_time_us":172584}
I20260709 13:12:25.078653 22836 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:25.072319 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:25.080459 22840 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:25.074541 22759 rpcz_store.cc:276] 0709 13:12:23.250919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.405040 (+1154121us) service_pool.cc:224] Handling call
0709 13:12:24.439722 (+ 34682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:24.511141 (+ 71419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.511175 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:24.518092 (+  6917us) log.cc:844] Serialized 26 byte log entry
0709 13:12:24.605605 (+ 87513us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:24.605674 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.605833 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:25.064435 (+458602us) spinlock_profiling.cc:243] Waited 86.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:25.073652 (+  9217us) raft_consensus.cc:1685] finished
0709 13:12:25.073675 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.073987 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198715136,"thread_start_us":85068,"threads_started":1,"wal-append.queue_time_us":220075}
W20260709 13:12:25.091871 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94128.6us
I20260709 13:12:24.962936 25486 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:25.121023 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:25.126801 22758 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:25.140687 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:25.144420 22839 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:25.147583 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.002s	sys 0.033s
W20260709 13:12:25.151244 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.002s	sys 0.034s
I20260709 13:12:25.129269 25494 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:12:25.139109 25496 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:25.211750 25486 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:25.226496 25490 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:25.227386 25490 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.227900 25490 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:25.185846 25458 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:12:25.253765 22833 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:25.270145 23431 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.719s (SENT)
W20260709 13:12:26.027196 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 757201us
I20260709 13:12:25.404968 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:26.028323 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 849981us
I20260709 13:12:25.377182 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2023) took 2128 ms. Trace:
I20260709 13:12:25.404567 22758 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:26.029469 22766 rpcz_store.cc:276] 0709 13:12:23.248512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.339951 (+1091439us) service_pool.cc:224] Handling call
0709 13:12:24.733789 (+393838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.755711 (+ 21922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.376830 (+621119us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54003403d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:25.377075 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803399552}
I20260709 13:12:25.572494 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:25.326508 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.353030 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:25.698213 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.032998 22915 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:25.268810 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:25.545053 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.024827 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.034204 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:25.303719 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:25.450543 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.041817 22835 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.041982 22764 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.088683 22914 raft_consensus.cc:2368] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5.
W20260709 13:12:25.253890 25434 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.811s	user 0.010s	sys 0.007s
W20260709 13:12:25.339828 25283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.020s	sys 0.040s
W20260709 13:12:26.105756 25283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.325s	user 0.021s	sys 0.077s
I20260709 13:12:26.106799 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:25.321959 25436 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:26.107793 25436 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:25.456121 25474 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.760s	user 0.000s	sys 0.003s
I20260709 13:12:26.109431 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.110409 25436 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:26.107754 22908 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:26.109925 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:25.270712 25458 consensus_queue.cc:1243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:12:26.100917 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1266) took 1145 ms. Trace:
I20260709 13:12:25.267642 25313 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.114077 22835 rpcz_store.cc:276] 0709 13:12:24.945554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.113031 (+167477us) service_pool.cc:224] Handling call
0709 13:12:26.090775 (+977744us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8448}
W20260709 13:12:26.101740 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.750s (SENT)
I20260709 13:12:26.126430 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.132063 22914 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:26.134258 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.135272 22761 raft_consensus.cc:2376] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
I20260709 13:12:26.096665 25459 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.136916 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1304) took 1121 ms. Trace:
I20260709 13:12:26.139583 22766 rpcz_store.cc:276] 0709 13:12:25.015422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.100419 (+1084997us) service_pool.cc:224] Handling call
0709 13:12:26.136811 (+ 36392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.118330 25313 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:26.152945 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.041s	sys 0.030s
I20260709 13:12:26.143397 22906 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
W20260709 13:12:26.159210 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.043s	sys 0.031s
I20260709 13:12:26.160192 23463 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:26.160099 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.164323 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.165077 22761 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:12:26.162772 25313 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.167667 22757 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:26.169137 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2026) took 2859 ms (client timeout 1932 ms). Trace:
I20260709 13:12:26.169445 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2068) took 1013 ms. Trace:
W20260709 13:12:26.169704 22762 rpcz_store.cc:269] 0709 13:12:23.306866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.636243 (+329377us) service_pool.cc:224] Handling call
0709 13:12:26.165886 (+2529643us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400002dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:26.166051 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749827328}
I20260709 13:12:26.169898 22766 rpcz_store.cc:276] 0709 13:12:25.153461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.140972 (+987511us) service_pool.cc:224] Handling call
0709 13:12:26.167019 (+ 26047us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.171274 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.172178 22914 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:26.122443 25434 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.165343 22835 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:12:26.166409 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.178834 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.179663 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.144996 25459 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.181419 22762 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:26.190587 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2071) took 1029 ms. Trace:
I20260709 13:12:26.190409 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.191054 22762 rpcz_store.cc:276] 0709 13:12:25.161232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.171094 (+1009862us) service_pool.cc:224] Handling call
0709 13:12:26.190508 (+ 19414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.191301 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:26.195921 22766 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.202062 25506 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:26.202337 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2056) took 2589 ms (client timeout 3294 ms). Trace:
W20260709 13:12:26.202795 22760 rpcz_store.cc:269] 0709 13:12:23.637770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.277308 (+639538us) service_pool.cc:224] Handling call
0709 13:12:26.202109 (+1924801us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54003408d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:26.202262 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576198912}
I20260709 13:12:26.202956 25506 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.204363 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.205119 22760 raft_consensus.cc:2376] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5. Re-sending same reply.
I20260709 13:12:26.205701 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2073) took 1043 ms. Trace:
I20260709 13:12:26.206033 22760 rpcz_store.cc:276] 0709 13:12:25.161973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.204028 (+1042055us) service_pool.cc:224] Handling call
0709 13:12:26.205635 (+  1607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.203087 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.208180 22762 raft_consensus.cc:2376] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
I20260709 13:12:26.209360 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2072) took 1047 ms. Trace:
I20260709 13:12:26.210610 25508 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.211324 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.212602 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2074) took 1050 ms. Trace:
I20260709 13:12:26.213063 22760 rpcz_store.cc:276] 0709 13:12:25.162280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.206749 (+1044469us) service_pool.cc:224] Handling call
0709 13:12:26.212482 (+  5733us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.211248 25508 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.209888 22762 rpcz_store.cc:276] 0709 13:12:25.161629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.191902 (+1030273us) service_pool.cc:224] Handling call
0709 13:12:26.209284 (+ 17382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.218668 22761 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:26.219628 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1306) took 1153 ms. Trace:
I20260709 13:12:26.223378 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.223698 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.193080 23461 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:26.220141 22761 rpcz_store.cc:276] 0709 13:12:25.066526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.143187 (+1076661us) service_pool.cc:224] Handling call
0709 13:12:26.219563 (+ 76376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.267072 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.243839 22760 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6.
I20260709 13:12:26.230201 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.268888 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2070) took 1108 ms. Trace:
I20260709 13:12:26.269454 22762 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.269407 22760 rpcz_store.cc:276] 0709 13:12:25.160821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.214080 (+1053259us) service_pool.cc:224] Handling call
0709 13:12:26.268790 (+ 54710us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:26.267892 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.271477 22914 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:26.272714 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.273586 22764 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:26.274247 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1313) took 1205 ms (client timeout 1584 ms). Trace:
W20260709 13:12:26.275024 22764 rpcz_store.cc:269] 0709 13:12:25.068386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.544593 (+476207us) service_pool.cc:224] Handling call
0709 13:12:26.274178 (+729585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.275580 22761 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 7: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.276283 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.277194 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2065) took 1125 ms. Trace:
I20260709 13:12:26.277657 22764 rpcz_store.cc:276] 0709 13:12:25.152252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.276006 (+1123754us) service_pool.cc:224] Handling call
0709 13:12:26.277133 (+  1127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.282254 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1312) took 1214 ms. Trace:
I20260709 13:12:26.281566 22906 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.282840 22761 rpcz_store.cc:276] 0709 13:12:25.068135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.266745 (+1198610us) service_pool.cc:224] Handling call
0709 13:12:26.282184 (+ 15439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.281448 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.284314 22908 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:26.285322 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.247529 25506 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.287096 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.315253 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.316144 25459 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.318089 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.318835 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.319046 22836 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:26.320111 22764 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.322072 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.323051 22840 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:26.324507 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:26.325656 25475 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.585s	user 0.022s	sys 0.009s
I20260709 13:12:26.325212 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1312) took 1397 ms. Trace:
I20260709 13:12:26.326108 22840 rpcz_store.cc:276] 0709 13:12:24.928116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.375047 (+446931us) service_pool.cc:224] Handling call
0709 13:12:26.317451 (+942404us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:26.319242 (+  1791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:26.324933 (+  5691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.325151 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13197696}
I20260709 13:12:26.328039 25508 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.328789 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1316) took 1328 ms. Trace:
I20260709 13:12:26.320984 25434 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.329211 22840 rpcz_store.cc:276] 0709 13:12:25.000886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.327771 (+1326885us) service_pool.cc:224] Handling call
0709 13:12:26.328107 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.328355 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.328438 (+    83us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.328515 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:26.328554 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.328612 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.328715 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.324519 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.331281 22761 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.332690 22912 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:12:26.332933 25466 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:26.333370 25466 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:26.334493 23431 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.904s (SENT)
I20260709 13:12:26.335440 22759 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.339120 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.340322 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1265) took 1399 ms. Trace:
I20260709 13:12:26.340792 22836 rpcz_store.cc:276] 0709 13:12:24.938123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.113023 (+174900us) service_pool.cc:224] Handling call
0709 13:12:26.317662 (+1204639us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9cce9c 00007f689e9cca5b 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:26.337691 (+ 20029us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17675904}
I20260709 13:12:26.329301 25508 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.318260 25506 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:26.337844 25490 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.109s	user 0.007s	sys 0.012s
I20260709 13:12:26.344252 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.346028 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.346513 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.348847 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.354202 22761 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:26.350996 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.354135 22759 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.353888 22764 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:26.356851 25487 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.455s	user 0.006s	sys 0.012s
I20260709 13:12:26.357939 25471 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:26.358181 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.359519 22841 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:12:26.359764 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.931s (SENT)
I20260709 13:12:26.361363 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.362004 22912 raft_consensus.cc:2368] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:26.362502 22759 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:26.363623 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1272) took 1293 ms. Trace:
I20260709 13:12:26.364214 22841 rpcz_store.cc:276] 0709 13:12:25.070137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.258261 (+188124us) service_pool.cc:224] Handling call
0709 13:12:26.357956 (+1099695us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:26.363564 (+  5608us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12771072}
I20260709 13:12:26.366053 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1292) took 3114 ms. Trace:
I20260709 13:12:26.369150 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1293) took 3116 ms. Trace:
I20260709 13:12:26.370487 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1294) took 3117 ms. Trace:
I20260709 13:12:26.369623 22765 rpcz_store.cc:276] 0709 13:12:23.252585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.732931 (+1480346us) service_pool.cc:224] Handling call
0709 13:12:26.206832 (+1473901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.210491 (+  3659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.210535 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.210605 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:26.210664 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.210765 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.368955 (+158190us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54003408d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:26.369083 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3388770560}
I20260709 13:12:26.371006 22757 rpcz_store.cc:276] 0709 13:12:23.253148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.993554 (+1740406us) service_pool.cc:224] Handling call
0709 13:12:26.167458 (+1173904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.169186 (+  1728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.370287 (+201101us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400002dd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:26.370427 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696191360}
I20260709 13:12:26.372193 22908 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:26.373884 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.373996 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:26.374636 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1321) took 1372 ms (client timeout 1584 ms). Trace:
I20260709 13:12:26.374891 22833 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:26.375396 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1269) took 1393 ms. Trace:
W20260709 13:12:26.375825 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 311) took 1604 ms (client timeout 1899 ms). Trace:
I20260709 13:12:26.374897 22834 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:26.341827 25434 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:26.376451 22832 rpcz_store.cc:269] 0709 13:12:24.771040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.771233 (+   193us) service_pool.cc:224] Handling call
0709 13:12:26.375636 (+1604403us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54000a4e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:26.375748 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167512832}
I20260709 13:12:26.379140 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:26.375099 22839 rpcz_store.cc:269] 0709 13:12:25.002591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.012647 (+ 10056us) service_pool.cc:224] Handling call
0709 13:12:26.374469 (+1361822us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:26.374574 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148838144}
I20260709 13:12:26.379206 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.379359 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1317) took 1378 ms. Trace:
I20260709 13:12:26.375941 22833 rpcz_store.cc:276] 0709 13:12:24.986531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.317986 (+331455us) service_pool.cc:224] Handling call
0709 13:12:26.373833 (+1055847us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:26.375332 (+  1499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14437120}
I20260709 13:12:26.381769 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1318) took 1380 ms. Trace:
I20260709 13:12:26.366412 22763 rpcz_store.cc:276] 0709 13:12:23.252007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.534702 (+1282695us) service_pool.cc:224] Handling call
0709 13:12:24.535508 (+   806us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:24.536603 (+  1095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.536664 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:24.538287 (+  1623us) log.cc:844] Serialized 52 byte log entry
0709 13:12:24.748571 (+210284us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:24.906295 (+157724us) log.cc:844] Serialized 14 byte log entry
0709 13:12:24.906588 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.906679 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.356086 (+1449407us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:26.359096 (+  3010us) raft_consensus.cc:1685] finished
0709 13:12:26.365814 (+  6718us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.365994 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8312,"prepare.queue_time_us":228019,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":23534,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":25016,"spinlock_wait_cycles":90373376,"thread_start_us":357548,"threads_started":2,"wal-append.queue_time_us":309660}
I20260709 13:12:26.381577 22758 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.348250 25313 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.383189 22834 rpcz_store.cc:276] 0709 13:12:25.001588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.077381 (+ 75793us) service_pool.cc:224] Handling call
0709 13:12:26.373680 (+1296299us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:26.381706 (+  8026us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13430144}
W20260709 13:12:26.377928 23431 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.393s (SENT)
I20260709 13:12:26.389912 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.384631 22839 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:26.391459 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1270) took 1374 ms. Trace:
I20260709 13:12:26.391646 22757 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:26.391844 22839 rpcz_store.cc:276] 0709 13:12:25.017398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.381452 (+1364054us) service_pool.cc:224] Handling call
0709 13:12:26.384316 (+  2864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:26.391179 (+  6863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.391400 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.392787 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.393528 25313 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:26.393805 22837 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:26.384729 22758 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:26.394448 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1262) took 1510 ms. Trace:
I20260709 13:12:26.394934 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2040) took 2960 ms. Trace:
I20260709 13:12:26.385113 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1298) took 3125 ms. Trace:
I20260709 13:12:26.381738 22836 rpcz_store.cc:276] 0709 13:12:25.001220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.341987 (+1340767us) service_pool.cc:224] Handling call
0709 13:12:26.363428 (+ 21441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.378994 (+ 15566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.379022 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.379079 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:26.379114 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.379183 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.379293 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.388594 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.393718 22763 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:26.386523 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1322) took 1383 ms. Trace:
I20260709 13:12:26.396027 22759 rpcz_store.cc:276] 0709 13:12:23.255301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.378509 (+3123208us) service_pool.cc:224] Handling call
0709 13:12:26.378862 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.379958 (+  1096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.379975 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.380001 (+    26us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:26.380023 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.380061 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.380132 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.396545 22841 rpcz_store.cc:276] 0709 13:12:25.002956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.365032 (+1362076us) service_pool.cc:224] Handling call
0709 13:12:26.385477 (+ 20445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.386078 (+   601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.386134 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.386185 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:26.386248 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.386314 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.386441 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.395780 22758 rpcz_store.cc:276] 0709 13:12:23.435049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.384181 (+2949132us) service_pool.cc:224] Handling call
0709 13:12:26.384538 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.394679 (+ 10141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.394875 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.401767 22760 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:26.402105 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2036) took 2992 ms. Trace:
I20260709 13:12:26.394878 22837 rpcz_store.cc:276] 0709 13:12:24.884332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.922715 (+ 38383us) service_pool.cc:224] Handling call
0709 13:12:26.032484 (+1109769us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:26.373989 (+341505us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9cce9c 00007f689e9cca5b 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:26.394380 (+ 20391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134943360}
I20260709 13:12:26.402467 22765 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:26.402629 22757 rpcz_store.cc:276] 0709 13:12:23.400294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.377591 (+2977297us) service_pool.cc:224] Handling call
0709 13:12:26.391502 (+ 13911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.392202 (+   700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.392341 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.403597 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2034) took 3004 ms. Trace:
I20260709 13:12:26.403990 22761 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:26.404050 22765 rpcz_store.cc:276] 0709 13:12:23.399338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.377841 (+2978503us) service_pool.cc:224] Handling call
0709 13:12:26.395735 (+ 17894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.403391 (+  7656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.403551 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.358791 25471 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.407918 22759 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:26.403389 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:26.406462 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2053) took 2951 ms. Trace:
I20260709 13:12:26.347141 25508 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.405886 22765 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:26.360941 25490 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.425810 25450 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:26.426070 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48178.3us
I20260709 13:12:26.417184 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.408732 23461 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:12:26.340715 25466 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.417826 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.426925 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.438249 22766 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:26.439450 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2069) took 1279 ms. Trace:
I20260709 13:12:26.439611 22764 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.439812 22766 rpcz_store.cc:276] 0709 13:12:25.160064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.172153 (+1012089us) service_pool.cc:224] Handling call
0709 13:12:26.439381 (+267228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.417791 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.427788 25471 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.408656 22763 rpcz_store.cc:276] 0709 13:12:23.445348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.392991 (+2947643us) service_pool.cc:224] Handling call
0709 13:12:26.393311 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.396377 (+  3066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.396598 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.428323 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2057) took 2706 ms. Trace:
I20260709 13:12:26.447180 22759 rpcz_store.cc:276] 0709 13:12:23.721720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.407500 (+2685780us) service_pool.cc:224] Handling call
0709 13:12:26.407785 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.428051 (+ 20266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.428262 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.449061 22759 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:26.408634 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1296) took 3148 ms. Trace:
I20260709 13:12:26.354449 25506 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.437391 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1301) took 2148 ms. Trace:
I20260709 13:12:26.437999 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1307) took 1371 ms. Trace:
I20260709 13:12:26.418074 22757 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:26.441864 22908 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:26.449980 22760 rpcz_store.cc:276] 0709 13:12:23.254242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.375782 (+3121540us) service_pool.cc:224] Handling call
0709 13:12:26.392077 (+ 16295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.402298 (+ 10221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.402536 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.450932 22758 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:26.456105 22765 rpcz_store.cc:276] 0709 13:12:25.066823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.405058 (+1338235us) service_pool.cc:224] Handling call
0709 13:12:26.405389 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.437729 (+ 32340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.437938 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.457103 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2059) took 2662 ms. Trace:
I20260709 13:12:26.457679 22758 rpcz_store.cc:276] 0709 13:12:23.794454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.406659 (+2612205us) service_pool.cc:224] Handling call
0709 13:12:26.450436 (+ 43777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.456796 (+  6360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.457033 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.452020 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1314) took 1366 ms. Trace:
I20260709 13:12:26.459066 22759 rpcz_store.cc:276] 0709 13:12:25.085935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.448365 (+1362430us) service_pool.cc:224] Handling call
0709 13:12:26.448813 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.451417 (+  2604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.451917 (+   500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.455521 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1311) took 1379 ms. Trace:
I20260709 13:12:26.460274 22763 rpcz_store.cc:276] 0709 13:12:25.067884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.445931 (+1378047us) service_pool.cc:224] Handling call
0709 13:12:26.446293 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.446537 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.446579 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.446641 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:26.446696 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.446775 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.446921 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.463500 22761 rpcz_store.cc:276] 0709 13:12:24.288694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.404836 (+2116142us) service_pool.cc:224] Handling call
0709 13:12:26.405237 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.405622 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.405665 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.437017 (+ 31352us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:26.437078 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.437162 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.437322 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.466784 22762 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:26.467334 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.467697 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.456209 22757 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:26.469340 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.475548 22908 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.471275 22764 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:26.385668 25434 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.433799 25511 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:26.475032 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2066) took 1322 ms. Trace:
I20260709 13:12:26.477262 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1309) took 1409 ms. Trace:
I20260709 13:12:26.483356 22764 rpcz_store.cc:276] 0709 13:12:25.067344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.440280 (+1372936us) service_pool.cc:224] Handling call
0709 13:12:26.469102 (+ 28822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.476857 (+  7755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.477181 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.485782 22912 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:26.483290 22762 rpcz_store.cc:276] 0709 13:12:25.152835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.219581 (+1066746us) service_pool.cc:224] Handling call
0709 13:12:26.474940 (+255359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.484707 22757 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.488161 22757 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:26.488703 25506 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.468879 25466 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.505301 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.539439 22758 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.540508 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.438676 25313 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.570688 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:26.553064 22757 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
W20260709 13:12:26.565739 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.865s (SENT)
I20260709 13:12:26.485317 25511 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.576261 22757 pending_rounds.cc:85] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 2
I20260709 13:12:26.586508 22757 pending_rounds.cc:107] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 5.3
W20260709 13:12:26.508611 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2711821789999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24430000000000002s for a response
I20260709 13:12:26.587080 22757 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:26.589793 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:26.590767 23462 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:26.586815 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:26.592195 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:26.598677 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 274246us
W20260709 13:12:26.606972 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 352512us
W20260709 13:12:26.612139 23432 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.931s (SENT)
I20260709 13:12:26.617159 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:26.620177 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 149662us
I20260709 13:12:26.572046 25517 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.573884 25496 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:26.573664 25512 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:26.627490 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2061) took 2824 ms. Trace:
I20260709 13:12:26.635619 22757 rpcz_store.cc:276] 0709 13:12:23.795112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.408534 (+2613422us) service_pool.cc:224] Handling call
0709 13:12:26.417914 (+  9380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.619069 (+201155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.619365 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:26.637362 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.638799 22757 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.624498 23462 raft_consensus.cc:2410] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:12:26.622593 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:26.598021 25511 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.648463 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.604372 25436 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:26.642445 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.651387 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:26.670434 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46576.8us
I20260709 13:12:26.671993 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.675774 22758 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:26.604002 25188 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:26.615604 25352 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.667867 22908 raft_consensus.cc:1217] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-5.3]   Dedup: 5.3->[]
I20260709 13:12:26.682180 25188 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.683108 22908 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: [2.2-5.3]
W20260709 13:12:26.687382 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 121711us
W20260709 13:12:26.688809 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 3.079s (SENT)
I20260709 13:12:26.625957 25506 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.690547 25506 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.668618 22757 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:26.618558 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:26.694674 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 75888.2us
I20260709 13:12:26.695230 23458 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:26.681025 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.699409 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.710800 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:26.631661 25496 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:26.711995 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 78138.8us
I20260709 13:12:26.720777 25506 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.728195 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:26.732301 25518 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:12:26.733593 23459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.097s	user 0.005s	sys 0.016s
I20260709 13:12:26.734015 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.682390 25352 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.734252 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:26.735174 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:26.735702 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.672227 25450 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:26.740581 22757 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:26.749923 22911 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:26.727794 25520 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:26.753892 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86831.1us
I20260709 13:12:26.779158 22758 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:26.755115 22913 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:26.755566 25508 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:26.782673 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 165470us
I20260709 13:12:26.686022 25516 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:26.758873 22759 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:26.747884 25483 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:26.764003 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:26.686445 25448 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.752986 25436 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.784686 23464 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:26.667354 25512 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:26.789829 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 94066.9us
I20260709 13:12:26.790527 22912 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:26.782065 23459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.326s	user 0.005s	sys 0.017s
I20260709 13:12:26.642009 25517 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:26.795061 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66982.3us
I20260709 13:12:26.797192 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.801710 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.807919 22913 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.808053 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.808933 22911 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:26.808990 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:26.810034 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46121.5us
I20260709 13:12:26.777268 25434 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.811204 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:26.812600 23465 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:26.785122 25516 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:26.818794 22906 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.819170 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.820605 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:26.819264 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 307) took 2347 ms. Trace:
I20260709 13:12:26.823474 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:26.863790 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.879329 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.797672 25188 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:26.824178 22837 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
W20260709 13:12:26.880187 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.882834 22839 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:26.884040 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1315) took 1618 ms. Trace:
I20260709 13:12:26.881059 25540 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:26.882782 22833 raft_consensus.cc:1217] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Deduplicated request from leader. Original: 2.2->[5.3-5.3]   Dedup: 5.3->[]
I20260709 13:12:26.885298 22833 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:26.822319 22838 rpcz_store.cc:276] 0709 13:12:24.467030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.478707 (+ 11677us) service_pool.cc:224] Handling call
0709 13:12:24.479211 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.590640 (+111429us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:26.213718 (+1623078us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18003ee240. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf
0709 13:12:26.218734 (+  5016us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.632982 (+414248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.633041 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:26.633092 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.814264 (+181172us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:26.814329 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.814491 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":257338,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":4133,"raft.queue_time_us":113763,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":287,"spinlock_wait_cycles":206212864,"thread_start_us":639085,"threads_started":2,"wal-append.queue_time_us":403328}
I20260709 13:12:26.881618 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.893388 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:26.895242 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71778.5us
I20260709 13:12:26.888180 22765 rpcz_store.cc:276] 0709 13:12:25.265338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.485126 (+1219788us) service_pool.cc:224] Handling call
0709 13:12:26.572527 (+ 87401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.572784 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:26.677189 (+104405us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.883675 (+206486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.883735 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:26.883788 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.883861 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.883983 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7364,"prepare.queue_time_us":181139,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":110206,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":263407,"threads_started":3}
I20260709 13:12:26.892470 23466 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:26.899065 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:26.899199 25525 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 13:12:26.899753 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.900548 22839 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:26.901530 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.901603 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1323) took 1891 ms. Trace:
W20260709 13:12:26.902199 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.901773 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1310) took 1834 ms. Trace:
I20260709 13:12:26.905694 22839 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:12:26.904408 22766 rpcz_store.cc:276] 0709 13:12:25.067609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.440608 (+1372999us) service_pool.cc:224] Handling call
0709 13:12:26.441167 (+   559us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:26.473839 (+ 32672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.473884 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:26.493923 (+ 20039us) log.cc:844] Serialized 52 byte log entry
0709 13:12:26.632474 (+138551us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:26.632526 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.632631 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.901435 (+268804us) raft_consensus.cc:1685] finished
0709 13:12:26.901459 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.901707 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34573,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":147,"thread_start_us":138026,"threads_started":1,"wal-append.queue_time_us":241428}
I20260709 13:12:26.897459 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.907948 22914 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.902194 22832 rpcz_store.cc:276] 0709 13:12:25.003298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.380829 (+1377531us) service_pool.cc:224] Handling call
0709 13:12:26.405706 (+ 24877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.434360 (+ 28654us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:26.588124 (+153764us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.894373 (+306249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.894429 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:26.894476 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.894550 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.894679 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70688,"thread_start_us":335923,"threads_started":2}
I20260709 13:12:26.909900 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:26.910784 22838 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.912533 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.914491 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.920017 22906 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.914816 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.923267 25541 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:26.923993 25541 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:26.914222 25313 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.926733 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.927402 22832 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:26.928334 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:26.928541 25512 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:26.886570 25506 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:12:26.929050 22832 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:26.929590 25506 consensus_queue.cc:1243] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:26.929713 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 310) took 2273 ms. Trace:
I20260709 13:12:26.930197 23459 rpcz_store.cc:276] 0709 13:12:24.656512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.732033 (+ 75521us) service_pool.cc:224] Handling call
0709 13:12:24.732686 (+   653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:24.733284 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:24.733329 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:25.454572 (+721243us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b2c00040ad8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:25.455348 (+   776us) log.cc:844] Serialized 26 byte log entry
0709 13:12:26.197744 (+742396us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b1800354340. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:26.814915 (+617171us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:26.818956 (+  4041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.879673 (+ 60717us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.929419 (+ 49746us) raft_consensus.cc:1685] finished
0709 13:12:26.929447 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.929630 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3883,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":54890112,"thread_start_us":1347287,"threads_started":1,"wal-append.queue_time_us":1469378}
I20260709 13:12:26.925000 25541 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.929081 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.934760 22833 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:26.935644 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.935958 25517 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.936301 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:26.936923 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:26.936976 22841 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:26.939412 22833 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:26.933792 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:26.941884 22911 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:26.920449 22837 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:26.944077 22837 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:26.793466 25520 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:26.946134 25515 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.946771 25515 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:26.947669 25520 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.949093 25515 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:26.955358 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:26.955775 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.956161 23465 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:26.956532 23461 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:26.957695 25313 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:26.958093 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.958344 25313 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:26.958741 23464 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
I20260709 13:12:26.959296 25436 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:26.958793 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.960067 23466 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:26.960440 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.826707 25516 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:26.926909 25526 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:12:26.886053 25483 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:26.974634 22833 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:26.887152 25521 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:27.004892 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.930227 25506 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:26.997328 22915 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:26.933642 22712 auto_rebalancer.cc:893] Removing replace marker failed: Replica 6545453e74344bb2957dd3729c5b3870 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:27.009610 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.915246 25490 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.023454 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:26.913174 25434 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:27.009893 22712 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:12:27.027573 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:27.029299 22836 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:12:27.031162 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.030776 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.042212 22914 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:27.033921 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.039680 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.043707 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:27.051693 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 275542us
I20260709 13:12:27.038880 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1319) took 2037 ms. Trace:
W20260709 13:12:27.028720 22712 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:12:27.059232 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.050661 22908 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:27.062130 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:27.063071 22833 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.063186 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.008731 25506 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:27.046723 23465 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:27.070969 22838 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:27.028403 25434 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.074350 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1315) took 2074 ms. Trace:
W20260709 13:12:27.074594 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43996.3us
I20260709 13:12:27.053352 22840 rpcz_store.cc:276] 0709 13:12:25.001890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.342368 (+1340478us) service_pool.cc:224] Handling call
0709 13:12:26.355012 (+ 12644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:26.382378 (+ 27366us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:12:26.382415 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:26.391398 (+  8983us) log.cc:844] Serialized 26 byte log entry
0709 13:12:26.618081 (+226683us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:26.618147 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.618239 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:27.038550 (+420311us) raft_consensus.cc:1685] finished
0709 13:12:27.038580 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:27.038794 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52817,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":119485,"threads_started":1,"wal-append.queue_time_us":279400}
I20260709 13:12:27.083662 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.078308 22906 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:27.084285 22835 rpcz_store.cc:276] 0709 13:12:25.000420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.146466 (+1146046us) service_pool.cc:224] Handling call
0709 13:12:26.164842 (+ 18376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:26.165127 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:26.277821 (+112694us) log.cc:844] Serialized 14 byte log entry
0709 13:12:27.066950 (+789129us) log.cc:844] Serialized 14 byte log entry
0709 13:12:27.069253 (+  2303us) log.cc:844] Serialized 14 byte log entry
0709 13:12:27.072896 (+  3643us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:27.072979 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:27.073049 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.073165 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:27.074257 (+  1092us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3322,"prepare.queue_time_us":286646,"raft.queue_time_us":113756,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":687511,"threads_started":3,"wal-append.queue_time_us":415394}
I20260709 13:12:27.031347 25531 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:27.029816 25537 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.083914 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.111064 22833 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:27.111759 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.119220 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:27.065279 25515 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:27.113392 23461 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:27.122759 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.032328 25512 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:27.126188 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:27.127136 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 192026us
W20260709 13:12:27.128724 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 69642.3us
I20260709 13:12:27.098879 25434 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:27.132622 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48834.2us
I20260709 13:12:27.098567 25313 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.133042 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:26.929168 25496 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.135313 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:27.132989 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0579848820000279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
I20260709 13:12:27.146878 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:27.147168 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0525848820001329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
I20260709 13:12:27.110932 25537 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:27.147552 22833 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.149930 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.131740 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:27.159107 22833 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:27.162781 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:27.164206 25313 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:27.134586 25542 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:27.162616 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.168252 25313 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:27.169181 25313 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.169250 25490 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.169737 25313 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.169931 25490 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:27.170389 25490 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.173722 25259 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.168242 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:27.175984 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:27.177141 25448 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.177627 25448 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:27.177819 25550 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:27.178049 25448 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.178318 25550 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.178857 25550 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:27.164558 25506 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.183126 25313 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.186936 25448 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.181185 25466 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:12:27.185907 25313 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:27.189281 25448 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.193085 25506 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.144563 25515 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:27.194902 25515 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.133118 25436 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:12:27.204387 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81762.9us
W20260709 13:12:27.206027 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 306936us
W20260709 13:12:27.208045 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 45364.9us
I20260709 13:12:27.216457 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:27.219869 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73814.2us
I20260709 13:12:27.225988 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.227475 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.231078 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:27.212682 25188 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:27.214915 25531 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:27.234673 25531 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:27.235126 25531 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.162498 25537 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:27.253966 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 76464.8us
I20260709 13:12:27.255211 25491 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:27.212785 25521 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.257793 25521 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:27.258273 25521 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:27.259632 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.219161 25352 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:27.248010 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.249532 23461 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:12:27.212364 25483 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:27.266971 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.270344 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:27.270344 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:27.271190 23465 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:27.271322 22763 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:27.274240 22833 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:27.274868 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:27.275442 23461 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.269717 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:27.276064 22757 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.277344 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:27.281868 25554 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:27.282164 25483 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.284746 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:27.285647 22766 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:27.286608 25541 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:27.289665 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 63834.2us
I20260709 13:12:27.290602 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.291586 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:27.274350 23432 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 3.079s (SENT)
I20260709 13:12:27.298707 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.292277 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:27.305605 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:27.316582 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.322026 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:27.323626 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:27.343302 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.319078 22840 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:27.323081 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:27.318125 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.340713 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.347708 22833 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:27.316787 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.271874 25313 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:12:27.354305 22911 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:27.266996 25188 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.274158 25520 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.358230 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.317073 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.311060 25477 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.282200 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:27.361119 22764 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:27.358551 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.357780 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.359020 22833 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:27.362723 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.357707 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.364522 22832 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:27.361024 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68822.7us
W20260709 13:12:27.357424 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 142710us
I20260709 13:12:27.370555 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.371673 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.376917 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.353652 25259 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.382897 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.384352 22839 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:27.388521 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.389827 22838 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:27.391825 25542 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:27.392472 25542 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:27.394017 25542 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.392203 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:27.398942 25511 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.730s	user 0.007s	sys 0.008s
I20260709 13:12:27.399713 25511 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.398963 22832 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:27.423187 23461 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:27.390864 22757 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:27.425177 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:27.358732 25490 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:27.458716 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:27.441524 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 167166us
I20260709 13:12:27.427080 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:27.487644 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57258.4us
I20260709 13:12:27.444161 25516 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.432883 25545 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.440912 25543 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:27.485394 25563 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:27.501793 22660 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 132) took 21247 ms (client timeout 15000 ms). Trace:
I20260709 13:12:27.502032 25511 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.528226 25506 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:12:27.529116 22660 rpcz_store.cc:269] 0709 13:12:06.254572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.265659 (+ 11087us) service_pool.cc:224] Handling call
0709 13:12:27.501701 (+21236042us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:24.546455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:24.776251 (+229796us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:24.808351 (+ 32100us) write_op.cc:435] Acquired schema lock
0709 13:12:24.818248 (+  9897us) tablet.cc:662] Decoding operations
0709 13:12:24.992469 (+174221us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:24.992723 (+   254us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:24.992760 (+    37us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:12:24.993627 (+   867us) tablet.cc:701] Row locks acquired
0709 13:12:24.993676 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:12:24.993984 (+   308us) write_op.cc:270] Start()
0709 13:12:24.994154 (+   170us) write_op.cc:276] Timestamp: P: 1783602744993908 usec, L: 0
0709 13:12:24.994180 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:12:25.104414 (+110234us) log.cc:844] Serialized 2854 byte log entry
0709 13:12:26.198628 (+1094214us) op_driver.cc:464] REPLICATION: finished
0709 13:12:26.392417 (+193789us) write_op.cc:301] APPLY: starting
0709 13:12:26.403893 (+ 11476us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:26.404197 (+   304us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:26.404352 (+   155us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:26.406892 (+  2540us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:26.408771 (+  1879us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:12:26.408853 (+    82us) write_op.cc:312] APPLY: finished
0709 13:12:26.409287 (+   434us) log.cc:844] Serialized 59 byte log entry
0709 13:12:26.411059 (+  1772us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:26.411211 (+   152us) write_op.cc:454] Released schema lock
0709 13:12:26.411286 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:26.411789 (+   503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:26.411831 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4679375,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":148126,"apply.run_cpu_time_us":4123,"apply.run_wall_time_us":38469,"num_ops":7,"prepare.queue_time_us":706280,"prepare.run_cpu_time_us":34526,"prepare.run_wall_time_us":826086,"raft.queue_time_us":102246,"raft.run_cpu_time_us":49562,"raft.run_wall_time_us":85308,"replication_time_us":1204099,"thread_start_us":429071,"threads_started":4,"wal-append.queue_time_us":978622,"wal-append.run_cpu_time_us":17777,"wal-append.run_wall_time_us":1341704}]]}
I20260709 13:12:27.504097 25434 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.381832 25313 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:27.508478 25537 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.510090 25517 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:12:27.514146 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 221051us
I20260709 13:12:27.534463 25517 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:27.535197 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:27.427815 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 45405.6us
W20260709 13:12:27.527647 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 100688us
I20260709 13:12:27.527872 22841 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:12:27.535903 25537 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.536340 25434 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:27.535887 25517 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.537449 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.539790 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.548642 25259 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:27.549258 25259 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:27.550330 25259 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.538358 25517 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:27.558125 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133060us
I20260709 13:12:27.559374 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.541818 25530 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:27.562094 25530 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:27.562538 25530 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.563383 25516 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.546613 25543 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:27.566507 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 207646us
I20260709 13:12:27.570001 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.572559 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:27.573369 22834 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.566730 25520 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.574188 25520 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.541817 25313 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:27.569243 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.547290 25506 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:27.584734 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.591195 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:27.592424 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.593676 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.591305 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.594750 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:27.593263 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:27.596191 22911 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:27.597136 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.599385 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.600963 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.609419 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.611512 25313 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:27.588024 25511 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:27.590432 25516 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:27.617641 25521 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.540205 25545 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.621637 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.590312 25541 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.627820 22834 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
W20260709 13:12:27.649042 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77926.6us
I20260709 13:12:27.619510 25563 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.629488 25436 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:27.661373 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52099.1us
I20260709 13:12:27.630548 25541 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:27.714080 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99566.3us
W20260709 13:12:27.706815 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54987.9us
I20260709 13:12:27.684307 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.655508 25434 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:27.721524 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.652747 25562 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:12:27.722716 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 156616us
I20260709 13:12:27.664662 25542 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.725200 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:27.654839 25448 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 13:12:27.729959 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.727588 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:27.678673 25541 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.728997 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:27.683068 25188 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.733911 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:27.720925 25436 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:27.715867 25570 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:27.688463 25515 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:12:27.713805 25313 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.757747 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:27.763288 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.763804 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:27.784663 22838 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:12:27.731002 25542 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:27.771577 25550 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.005s
W20260709 13:12:27.754388 25561 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:12:27.733701 25571 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:27.761076 25531 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
I20260709 13:12:27.747730 25516 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.800179 22911 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:27.774993 25570 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:27.802011 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.755144 25471 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:27.816521 25471 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:27.817873 25471 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:27.805775 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48378.1us
I20260709 13:12:27.746740 25490 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.820854 25531 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.821957 25572 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:12:27.823799 25531 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:27.812691 25516 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:27.826188 25516 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.821475 25521 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:27.656126 25543 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.804407 25542 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.805003 25571 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.769737 25574 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:27.829797 25554 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:27.826972 25570 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.841025 25572 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:27.835599 25571 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.827787 25436 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:27.805442 25550 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.869484 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.887480 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:27.862110 25554 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:27.862638 25572 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:27.877918 25573 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:27.899147 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 171091us
I20260709 13:12:27.866974 25562 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:27.894474 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66933.3us
I20260709 13:12:27.907305 25439 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.908691 25439 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:27.910499 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.912478 25439 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:27.892833 25521 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:27.915052 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:27.923524 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:27.898844 25572 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.901146 25531 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:27.928541 25521 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:27.932298 25537 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.951020 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:27.904381 25573 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.952798 25537 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:27.977635 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:27.961890 25531 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:27.986752 25531 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:27.978952 25537 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.947721 25581 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:12:27.959575 25259 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:12:27.994035 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 265110us
I20260709 13:12:27.997805 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:27.992472 25531 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:27.998392 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:27.998824 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:27.999827 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.986846 25434 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:27.992799 25521 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.005963 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.013664 25579 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:12:28.013114 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:28.031152 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55999us
I20260709 13:12:28.031821 22913 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:28.045120 23461 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:28.045898 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73178.9us
I20260709 13:12:27.971454 25530 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.039068 22915 raft_consensus.cc:2368] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8.
I20260709 13:12:28.046733 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:28.042970 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:27.983451 25575 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:12:28.053267 22911 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:12:28.071940 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 74247.9us
I20260709 13:12:28.078226 22914 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:28.040202 25434 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:28.062165 25580 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:12:28.088893 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:28.054936 25584 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:12:28.089761 22915 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:28.062950 25550 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:28.101137 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50112.7us
I20260709 13:12:28.102912 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:28.106086 23462 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.103305 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:28.053910 25585 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:28.070041 25563 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:28.106209 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:28.053841 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:28.105321 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:28.106129 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:28.110913 22841 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:28.112303 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:28.080063 25533 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:28.121476 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:28.122221 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54439.7us
I20260709 13:12:28.122407 22911 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:28.125298 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:28.076495 25573 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:28.127907 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.130435 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.098235 25515 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:28.068068 25541 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.059010 25570 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.138656 22806 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6.
W20260709 13:12:28.115758 25506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.024s	sys 0.005s
W20260709 13:12:28.123198 25578 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 13:12:28.150701 22806 leader_election.cc:403] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:28.148008 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:28.147675 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:28.144122 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.154750 22914 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:28.126369 23459 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:28.153435 22837 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:28.161008 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.161136 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.162669 22832 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:28.166105 22841 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:28.152348 25506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.025s	sys 0.005s
I20260709 13:12:28.136767 25515 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:28.170760 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:28.171384 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.102191 25511 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:12:28.149480 25313 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:28.146274 25585 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:28.190235 25563 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:28.149343 25545 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:28.195816 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 92727.3us
I20260709 13:12:28.154848 25483 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6.
I20260709 13:12:28.200971 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.209332 22836 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:28.211730 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:28.213445 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:28.214037 23463 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 9. Ops: []
I20260709 13:12:28.214146 23462 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:28.158982 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:28.216058 22841 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:12:28.175634 25583 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 13:12:28.217325 22879 connection.cc:256] RPC call timeout handler was delayed by -2.4500623349998705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 13:12:28.219653 22879 connection.cc:256] RPC call timeout handler was delayed by -2.3834623349998765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:12:28.220043 22879 connection.cc:256] RPC call timeout handler was delayed by -2.07746233499995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:12:28.220320 22879 connection.cc:256] RPC call timeout handler was delayed by -2.0306623349998745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
I20260709 13:12:28.217896 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:28.220600 23462 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:12:28.221131 22911 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:28.221328 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:28.222091 22906 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:28.219545 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:28.220602 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9586623349998717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 13:12:28.223495 22879 connection.cc:256] RPC call timeout handler was delayed by -1.4267623349999212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2647s for a response
I20260709 13:12:28.226094 25579 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:28.226867 25511 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:28.227313 25511 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:28.172917 25506 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:28.231372 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:28.235776 25545 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.236481 25545 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:28.240039 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 93839.5us
I20260709 13:12:28.240682 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:28.241016 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.243139 22914 raft_consensus.cc:1217] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-6.7]   Dedup: 3.6->[6.7-6.7]
I20260709 13:12:28.250233 22914 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: [3.6-6.7]
I20260709 13:12:28.255522 25589 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.256973 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:28.255833 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:28.262068 22911 raft_consensus.cc:2368] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8.
I20260709 13:12:28.228826 25434 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.265758 25483 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:28.266392 25483 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:28.267607 25483 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.232903 25574 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:28.269797 25574 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.270191 25574 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:28.273959 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.274662 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:28.277631 25259 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:28.258455 25496 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:28.284327 25506 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.284940 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:28.229980 23458 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:28.269995 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:28.290011 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:28.286859 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.300601 22837 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:28.301220 22906 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:28.301719 22841 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:28.195236 25516 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.302919 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:28.314049 22766 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:28.314473 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.314769 22761 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:28.315137 22758 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:28.316102 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:28.303385 23464 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:28.293650 22759 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.254236 23466 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEARNER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:12:28.242822 25588 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.319350 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.320132 22759 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:28.307421 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:28.321153 22914 raft_consensus.cc:2368] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8.
W20260709 13:12:28.311436 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55695.2us
I20260709 13:12:28.320153 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.323676 22762 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.312258 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.324734 22760 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.326218 22832 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:28.329638 22757 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:12:28.336086 25575 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.336656 25575 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.339475 25575 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:28.341903 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.271202 25585 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.333560 25530 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:28.308049 25313 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:28.256188 25589 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.319252 25588 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:28.392590 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.363075 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:28.396027 22759 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:12:28.416713 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:28.417728 22841 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:28.427248 25588 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:28.428277 25541 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:28.429989 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:28.430173 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:28.430824 22836 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.433460 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:28.192283 25563 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.349058 25545 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:28.288775 25511 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.452581 22834 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:27.954340 25554 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:27.940366 25562 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.398651 25259 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:12:28.393960 25313 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.473244 25588 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:28.527267 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 93890.7us
I20260709 13:12:28.556463 22841 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:28.564062 22765 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:28.558276 25571 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.003s
I20260709 13:12:28.524287 25570 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:28.546089 25511 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:28.509198 25574 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.559648 25520 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:28.571426 25563 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:28.585139 25572 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.000s
I20260709 13:12:28.598131 22762 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:28.589109 25542 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.744s	user 0.009s	sys 0.005s
I20260709 13:12:28.574175 25603 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:28.606076 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 394480us
I20260709 13:12:28.585193 25571 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.638834 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:28.640097 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:28.641240 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:28.634665 25531 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
I20260709 13:12:28.609534 23458 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:28.658684 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.661852 23462 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:28.662106 22762 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.663393 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:28.664124 22762 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:28.659689 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:28.601670 25563 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.666851 23459 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:28.670408 25434 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:28.671139 25434 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:28.672364 25434 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.673722 25602 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:28.669193 25563 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:28.665391 22762 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:28.604274 25542 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.628854 25259 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:12:28.602375 25572 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.630242 25436 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 13:12:28.691983 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.634431 25575 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:12:28.687618 25585 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:28.663724 25589 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:28.689857 25537 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.004s
W20260709 13:12:28.691479 25604 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:12:28.683566 25531 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:28.800781 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:28.809760 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 186407us
I20260709 13:12:28.645100 25520 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:12:28.826541 22763 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:28.644353 25603 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:12:28.838192 22762 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:28.688766 25611 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.141s
I20260709 13:12:28.863248 25477 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:28.864607 25530 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.865257 25601 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.866046 25483 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.837387 25585 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.837684 25259 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:12:28.874889 25605 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.288s	user 0.000s	sys 0.007s
I20260709 13:12:28.874250 25562 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:28.878006 25608 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.264s	user 0.000s	sys 0.007s
I20260709 13:12:28.874513 25313 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:28.882808 25612 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.213s	user 0.000s	sys 0.007s
I20260709 13:12:28.868397 25537 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:28.878265 25477 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:28.878002 25571 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:28.884533 25434 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:29.013803 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 372601us
I20260709 13:12:29.015204 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:28.889308 25574 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:28.897928 25585 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:28.995776 25607 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.250s	user 0.005s	sys 0.002s
I20260709 13:12:28.965566 25562 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:28.888912 25530 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.042685 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:29.040421 25542 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.027765 25603 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:29.070379 25624 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:12:29.061591 25616 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:12:29.071720 25626 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.073149 25630 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:29.167536 25630 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.168044 25630 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:29.061057 25606 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.326s	user 0.000s	sys 0.008s
I20260709 13:12:29.064733 25477 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.175527 25477 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:28.896253 25483 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.177701 25626 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.178366 25483 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:29.076534 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.180205 25626 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.052356 25618 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.185s
W20260709 13:12:29.181923 25496 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.900s	user 0.009s	sys 0.012s
I20260709 13:12:29.105469 25603 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:29.183209 25603 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.183714 25630 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.184605 25603 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.185846 25572 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.186175 25630 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.055751 25562 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.186702 25629 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:29.187227 25629 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.187604 25629 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.189749 25550 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.190295 25550 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:28.892015 25601 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:29.098780 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 225485us
I20260709 13:12:29.149660 25188 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:29.039033 25434 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.199189 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:29.110921 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68323.6us
I20260709 13:12:29.160657 25542 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.198179 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.201275 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:29.202050 22911 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5. Re-sending same reply.
I20260709 13:12:29.202154 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.197901 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.163487 25448 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.202879 22906 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:29.204131 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.204468 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.205772 22908 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:29.207226 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.209105 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.210765 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.214735 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.212105 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.199343 25188 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:29.219406 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.217923 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.216642 22909 raft_consensus.cc:2376] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7. Re-sending same reply.
I20260709 13:12:29.220510 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:29.221943 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.223879 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.225600 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.226363 22906 raft_consensus.cc:2368] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:29.227005 22910 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 7.
I20260709 13:12:29.232700 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.236515 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.237388 22906 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.238330 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.221932 25188 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.227113 25483 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:29.248031 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.213275 25542 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.229652 25537 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.214159 25448 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.284168 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.239212 25531 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.285260 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.233971 25635 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:29.299544 25448 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.242687 25477 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:12:29.318125 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:29.291414 25621 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:12:29.330806 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:29.331449 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76196.3us
I20260709 13:12:29.323289 22910 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.335925 25530 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.336397 25601 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.336745 25515 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.337450 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.337868 25515 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.338363 22910 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:29.342119 25434 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.284188 25572 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:12:29.343906 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.345708 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.346432 22909 raft_consensus.cc:2368] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:29.350122 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.342816 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.351598 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.354388 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.355113 22906 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.353235 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:29.287070 25483 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.358385 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.353976 25439 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:29.359503 25620 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:12:29.360900 25542 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.349673 25636 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:29.368682 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.372160 25642 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:29.375926 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.397680 25642 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.397933 22915 raft_consensus.cc:2376] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7. Re-sending same reply.
I20260709 13:12:29.398157 25642 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:29.404696 25576 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:29.405201 25576 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:29.376458 25610 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:29.380894 25581 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:29.384939 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:29.422288 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.385381 25636 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.387815 25562 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:29.389267 25511 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.804s	user 0.008s	sys 0.004s
I20260709 13:12:29.442651 25550 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.443372 25483 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.411576 25610 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:29.394615 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:29.445081 22911 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:29.445250 22764 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.444912 25610 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.446552 25610 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.392683 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:29.447521 22757 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:29.374485 25439 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.396522 25629 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.437892 25636 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.438846 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.454239 22841 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:29.439643 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.455258 22834 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:29.441046 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.422477 25581 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:29.443831 25557 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:29.457542 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:29.427381 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.450992 22912 raft_consensus.cc:2368] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:29.454972 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.458702 22911 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:29.458810 25450 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:29.460335 25450 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.461910 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:29.462530 22758 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.468907 22764 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:29.474972 22906 raft_consensus.cc:2368] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:29.453835 25575 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.505837 22762 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:29.473057 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.520040 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:29.488678 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1327) took 1183 ms (client timeout 1554 ms). Trace:
I20260709 13:12:29.520860 22841 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:29.521193 22765 rpcz_store.cc:269] 0709 13:12:28.305697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.326982 (+ 21285us) service_pool.cc:224] Handling call
0709 13:12:29.488481 (+1161499us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54001b9c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:29.488608 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509242752}
I20260709 13:12:29.489493 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1321) took 1206 ms. Trace:
I20260709 13:12:29.522188 22763 rpcz_store.cc:276] 0709 13:12:28.283389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.286138 (+  2749us) service_pool.cc:224] Handling call
0709 13:12:28.286429 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:28.869993 (+583564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:29.489225 (+619232us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b540024b8d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:29.489433 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990157696}
I20260709 13:12:29.529019 22763 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:29.530123 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:29.531905 22764 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:29.534787 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:29.536124 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:29.488152 25586 raft_consensus.cc:1064] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:12:29.514230 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:29.549728 22915 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:29.551033 22758 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:29.551373 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:29.551512 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.463949 25562 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.496443 25557 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:29.486536 25439 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.556141 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:29.567293 22765 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:29.519656 25636 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.518219 25542 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:29.553364 25313 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:29.566119 22911 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:29.552173 25545 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:29.495805 25188 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.570497 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.571519 22761 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:29.571298 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.585716 22915 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:29.587182 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.588162 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.588871 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.590261 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:29.590821 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.555418 25550 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:29.530218 25575 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.597481 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:29.567107 25585 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.507s	user 0.000s	sys 0.011s
I20260709 13:12:29.561228 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:29.594185 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.602208 22841 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:12:29.602259 22840 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:29.598845 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.603924 22833 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:12:29.604022 22805 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.551s (SENT) [suppressed 55 similar messages]
I20260709 13:12:29.605006 22833 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:29.599112 22836 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:29.599149 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.475351 25576 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.616669 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.551005 25571 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:12:29.569576 25589 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:12:29.613993 22914 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.614226 22833 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:29.594990 25550 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.615738 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:29.621178 22833 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:29.619774 22836 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:29.599905 25602 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:29.612972 22911 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:29.623185 22907 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.624101 22840 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:29.628458 22835 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:29.577564 25542 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.573122 25636 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.626173 22915 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.631381 25564 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:29.634208 22836 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:12:29.634812 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.635648 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:29.663313 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:29.665553 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:29.666368 23466 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:29.641654 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.643702 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.648135 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 322) took 1425 ms. Trace:
W20260709 13:12:29.650637 25650 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:12:29.671958 23462 rpcz_store.cc:276] 0709 13:12:28.222376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.227562 (+  5186us) service_pool.cc:224] Handling call
0709 13:12:28.661654 (+434092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:28.662233 (+   579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:29.647856 (+985623us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54006707d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:29.648054 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992534912}
I20260709 13:12:29.658376 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.673686 22841 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:29.617756 25313 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:29.662153 22907 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:29.661618 22836 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 2
I20260709 13:12:29.705364 22836 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:29.705664 22836 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:29.706023 22836 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:29.706511 22836 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:29.559438 25581 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.698854 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:29.619920 25562 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.710449 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.623548 25550 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:29.606236 25585 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.699787 22911 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:29.702858 22833 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:12:29.694828 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 90845us
W20260709 13:12:29.694072 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 139425us
I20260709 13:12:29.721452 23466 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:29.711143 22915 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:29.707145 25579 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.652503 25575 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.696483 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:29.714371 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:29.719178 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:29.731595 23462 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:29.661571 25564 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.738797 25564 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:29.709935 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:29.745813 23459 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:29.746520 25585 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.724478 22911 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:29.748659 25586 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 2, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.705785 25515 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.751228 25636 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.751470 25515 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.720398 25562 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.755069 25562 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.756969 22910 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:29.759977 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:29.761406 22915 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 13:12:29.761905 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1309) took 1472 ms. Trace:
I20260709 13:12:29.762336 22832 rpcz_store.cc:276] 0709 13:12:28.289382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.289630 (+   248us) service_pool.cc:224] Handling call
0709 13:12:28.322962 (+ 33332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:28.326595 (+  3633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:29.761685 (+1435090us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b54000ec4d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:29.761835 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15651200}
I20260709 13:12:29.766134 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:29.725668 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:29.709242 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:29.768577 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61937.3us
I20260709 13:12:29.770264 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:29.728418 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:29.771421 22907 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:29.771433 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:29.732658 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:29.772660 22841 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:29.773231 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:29.729267 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.708379 25439 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:12:29.733655 22836 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:29.791740 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:29.736636 25516 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:29.737421 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.767978 22913 raft_consensus.cc:2368] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:29.710449 25434 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:12:29.720175 25624 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:29.792572 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:29.716112 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:29.817890 25545 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:29.828802 22766 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:29.829315 25506 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:29.830152 25573 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:29.835232 25576 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:29.838894 22914 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:29.841192 22909 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:29.767014 25634 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:29.794332 25636 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:29.912019 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:29.696745 25557 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.949062 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:29.953003 22841 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:29.897347 25313 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:29.844769 25450 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.699666 25602 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:29.970659 25643 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:12:29.971596 25602 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.906420 25626 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.972400 25626 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.973601 25659 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:12:29.768986 25579 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.975016 25626 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.906376 25635 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:29.976382 25579 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.822322 25571 consensus_queue.cc:1243] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 6.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:12:29.897692 25656 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:29.959069 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47144.3us
I20260709 13:12:29.908815 25530 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:30.026643 25530 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:30.027667 25530 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.028983 25530 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.031718 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:29.846086 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 131843us
I20260709 13:12:29.902638 25506 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.032881 25506 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:29.908347 25573 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.036384 25573 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:30.042160 25542 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:30.043366 25313 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.047053 25506 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:29.964758 25439 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:30.051877 25439 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.052276 25439 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:30.053063 25506 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:29.954124 25554 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:30.010077 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.023545 25564 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.049731 25188 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.978611 25545 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:29.822836 25601 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:29.823021 25466 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.163s
W20260709 13:12:30.056674 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 290636us
W20260709 13:12:30.049422 25642 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.003s
I20260709 13:12:29.962638 25636 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:30.057173 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.965323 25515 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:29.973278 25637 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:30.085287 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:30.090523 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.050937 25573 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.098212 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:30.131980 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.098863 25652 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.142575 22906 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:30.145097 22908 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:30.099327 23459 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:30.090289 25642 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.105784 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:30.214748 22915 raft_consensus.cc:606] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:30.215133 22915 raft_consensus.cc:614] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:30.081658 25554 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.090353 25188 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:12:30.108527 25564 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:30.110738 25610 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.000s
I20260709 13:12:30.111514 25636 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.113440 25576 raft_consensus.cc:1064] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:12:30.230125 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1333) took 1006 ms. Trace:
I20260709 13:12:30.230599 22912 rpcz_store.cc:276] 0709 13:12:29.223549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:29.224823 (+  1274us) service_pool.cc:224] Handling call
0709 13:12:30.229886 (+1005063us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540010e0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:30.230054 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496320384}
I20260709 13:12:30.231002 25576 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.137970 25550 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.232262 25629 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:30.090600 25637 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:30.119405 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.233697 25550 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:30.233853 25439 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.124333 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1336) took 1515 ms. Trace:
I20260709 13:12:30.234810 22766 rpcz_store.cc:276] 0709 13:12:28.609272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.609424 (+   152us) service_pool.cc:224] Handling call
0709 13:12:28.632652 (+ 23228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:29.903153 (+1270501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.124086 (+220933us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002dafd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:30.124271 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750293248}
I20260709 13:12:30.233788 25637 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.236394 25439 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:30.236645 25450 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.216706 25466 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.237128 25450 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.083165 25601 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.238642 25601 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.239241 25450 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:29.953768 25634 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:30.241643 25627 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.102s	user 0.001s	sys 0.003s
I20260709 13:12:30.242344 25634 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.243513 25634 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.148334 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:30.152761 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.246227 22911 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:30.212536 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72585.1us
I20260709 13:12:30.216199 25448 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.255455 22915 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:30.279920 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:30.280781 22912 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:30.281298 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 133041us
W20260709 13:12:30.291327 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 327) took 2377 ms (client timeout 1899 ms). Trace:
W20260709 13:12:30.291836 22838 rpcz_store.cc:269] 0709 13:12:27.914025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:27.914461 (+   436us) service_pool.cc:224] Handling call
0709 13:12:30.291039 (+2376578us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:30.291211 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137405312}
I20260709 13:12:30.288630 25637 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.295691 25637 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:30.297185 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:30.298048 22915 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.302779 25496 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:30.303356 25496 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:30.306988 25483 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:30.308727 25483 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:30.309695 25511 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:30.310936 25589 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:30.311187 25511 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:30.212235 25626 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 13:12:30.314456 23432 leader_election.cc:400] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8.
I20260709 13:12:30.319254 23432 leader_election.cc:403] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:12:30.311383 25589 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:30.315213 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:30.315716 22915 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:30.315904 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:30.322757 23461 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:30.251749 25537 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:30.332535 23466 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.345101 23462 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.286473 25490 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:30.331907 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.021s	sys 0.048s
I20260709 13:12:30.334077 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:30.467454 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 133467us
I20260709 13:12:30.282595 25515 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:30.386242 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.325641 25629 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:12:30.331960 25533 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:30.483018 25533 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:30.483954 25533 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.485160 25533 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:30.410681 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 127721us
I20260709 13:12:30.486133 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.286369 25516 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 13:12:30.489979 25668 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:12:30.287750 25188 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:12:30.415194 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:30.491714 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76595.2us
I20260709 13:12:30.312459 25575 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 13:12:30.417610 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 570206us
I20260709 13:12:30.498697 25554 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:30.318042 25624 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:12:30.329154 25531 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:12:30.504607 22757 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:30.505638 25531 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:30.505995 25573 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:30.506175 25531 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:30.506695 22764 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:30.509070 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:30.509917 22766 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.514731 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:30.515674 22759 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:30.517762 22762 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:30.318730 25610 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:12:30.318470 25564 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:12:30.328914 25434 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:12:30.522176 25610 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:30.333550 25602 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:12:30.335421 25658 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:12:30.523391 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:30.523772 25658 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:30.523979 22761 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.524183 25658 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:30.525117 25658 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.525871 25642 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:30.526177 25629 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:30.415165 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:30.528817 23432 leader_election.cc:400] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8.
I20260709 13:12:30.286571 25611 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:30.469270 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.023s	sys 0.048s
W20260709 13:12:30.469750 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 120912us
I20260709 13:12:30.347853 25537 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.472733 23462 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:30.416925 25471 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8.
I20260709 13:12:30.384852 25626 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:12:30.513255 25601 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.482120 25579 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:12:30.477430 25483 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:30.587530 25483 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:30.588583 25483 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.590144 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:30.590729 25622 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:30.591145 25622 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:30.592324 25573 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:30.592015 25622 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.592741 25573 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:30.593639 25573 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.595099 25573 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.593374 25622 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.519479 22912 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:30.520529 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:30.520620 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.601120 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:30.601372 22759 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.602471 23460 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:30.522727 25610 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:30.604115 25610 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.605716 25610 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:30.607646 25601 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:30.328472 25450 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
W20260709 13:12:30.465557 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.681s (SENT)
I20260709 13:12:30.528873 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:30.612818 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 147364us
I20260709 13:12:30.554912 25611 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:12:30.619684 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.620023 25611 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:30.622184 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:30.624336 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:30.623293 23460 raft_consensus.cc:2376] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5. Re-sending same reply.
I20260709 13:12:30.539628 22761 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:30.637658 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.507583 25531 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.526587 25629 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:30.551330 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:30.648147 23464 raft_consensus.cc:2368] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:30.551690 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:30.542968 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.553048 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:30.551004 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:30.474861 25515 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:30.518208 25516 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:30.540732 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 768154us
I20260709 13:12:30.549089 25643 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:30.561560 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:30.669898 22765 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:30.618510 22661 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 3, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.562034 25652 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.562696 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:30.563715 25626 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:30.557260 23463 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:30.565730 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:30.567241 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:30.568053 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:30.699159 22834 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:30.569576 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:30.645205 25618 raft_consensus.cc:2764] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:12:30.573619 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:30.574252 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:30.573689 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:30.705324 22841 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:30.522233 25550 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:30.634654 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 114164us
I20260709 13:12:30.635869 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:30.636580 22759 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:30.709349 22836 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.538439 23466 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:30.651016 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 108193us
I20260709 13:12:30.710636 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.711102 23466 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:30.712173 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.714049 25313 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:30.657248 25545 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.685218 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:30.714730 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:30.696594 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:30.697811 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:30.702940 22838 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:30.715997 23464 raft_consensus.cc:2376] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8. Re-sending same reply.
I20260709 13:12:30.704126 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:30.527446 25658 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:30.715691 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.715421 23463 raft_consensus.cc:2376] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
I20260709 13:12:30.714686 25313 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:30.706856 25550 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:30.722074 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:30.723675 22879 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:30.669670 25643 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:30.698669 25626 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:30.741638 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:30.730516 22661 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 4, now has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:30.744784 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:30.745424 22838 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:12:30.746784 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.747514 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.734402 25550 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:30.727069 25576 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:12:30.737882 25634 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:30.772042 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:30.749078 25664 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:30.762558 23458 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:12:30.770435 22661 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 6, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.783578 22732 leader_election.cc:400] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:30.784008 22732 leader_election.cc:403] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:30.785559 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:30.786084 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72225.4us
I20260709 13:12:30.786234 22832 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.789436 22838 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:30.745352 25601 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:12:30.811058 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:30.812048 22834 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:30.780179 25313 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.734396 25629 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.811980 22661 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.821732 22732 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 7.
I20260709 13:12:30.823526 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:30.831888 22806 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.135s (SENT)
I20260709 13:12:30.832477 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:30.839422 25626 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.840763 25652 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:30.800391 25664 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:30.841369 25652 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:30.842422 25664 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.799181 25545 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:30.846447 22805 leader_election.cc:400] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:30.846825 22805 leader_election.cc:403] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:30.785094 25643 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:30.847472 25643 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.842394 25652 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.853221 22836 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:30.853778 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:30.870962 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45553.5us
I20260709 13:12:30.740340 25515 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:30.867496 22833 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:30.876160 22765 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:30.879405 22832 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:30.879951 22834 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:30.879173 22661 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:30.925263 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.939739 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.879866 25483 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:31.045939 22906 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:12:31.029662 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 197796us
W20260709 13:12:31.054951 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.056208 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.879173 25662 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:31.029927 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:31.058199 22911 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:31.057418 22806 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.149s (SENT)
I20260709 13:12:31.059931 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:31.027562 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.061738 22912 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:31.064106 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:31.065621 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 244084us
I20260709 13:12:31.066541 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:31.068251 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 246175us
I20260709 13:12:31.069528 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.070086 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:31.070410 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.073369 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:31.027954 25530 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:31.083222 22661 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 3, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.052996 25611 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:31.095202 22661 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:31.052780 25483 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:31.101274 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.109814 22661 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 5, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.111660 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.060590 25662 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.088419 25652 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:31.073010 25636 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:31.080108 25658 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:31.085258 25634 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.087016 25259 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.099934 25545 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:31.129455 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.119155 25662 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.135928 25531 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:31.137938 25686 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:31.138816 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.311s	user 0.035s	sys 0.017s
I20260709 13:12:31.138521 25686 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.140398 25259 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.111827 25439 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:31.145987 25439 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.146698 25188 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.147292 25188 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.111690 25602 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:31.118806 25313 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:31.149670 25313 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:31.164633 22661 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:31.087803 25667 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
W20260709 13:12:31.168515 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111139us
W20260709 13:12:31.178896 25681 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:12:31.151324 25516 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:31.158475 25626 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:31.152254 25652 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:31.197783 22911 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:31.215981 22661 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.210521 25439 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:31.232007 22661 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 6, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:31.256366 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.215322 25691 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.224893 25530 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:31.277174 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 170190us
I20260709 13:12:31.223450 25601 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:31.225802 25521 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:31.279193 25691 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.284686 25188 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:31.285281 25259 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:31.280069 25530 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.268229 25516 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:31.290099 25692 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:12:31.294428 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:31.293051 22661 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:31.296921 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.469s	user 0.036s	sys 0.017s
W20260709 13:12:31.307940 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.4804e+06us
I20260709 13:12:31.282290 25686 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:31.310117 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.311743 22661 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 5 to 6, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.300230 25629 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:31.303444 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 176188us
I20260709 13:12:31.315812 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.316902 25545 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:31.321956 22661 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.323854 25697 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:31.326824 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47008.1us
I20260709 13:12:31.341293 25697 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.342082 25697 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:31.327016 25545 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:31.344707 22833 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:31.344700 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:31.346285 25581 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.355132 22912 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:12:31.332115 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:31.334458 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:31.338047 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:31.362460 23465 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:31.337121 25664 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:31.349931 22911 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:31.350358 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.327431 25537 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.274916 25634 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.291633 25601 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:31.362375 25550 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.000s
I20260709 13:12:31.295698 25521 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.371197 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:31.368016 22914 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:31.374540 22841 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:31.375633 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.353049 25533 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.367861 22661 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.356072 25530 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:31.307605 25516 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.365303 25611 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.395771 22661 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index 4 to 2, term changed from 3 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from VOTER to NON_VOTER, now has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:31.349318 25483 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.408044 25483 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.414852 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 325) took 1100 ms. Trace:
I20260709 13:12:31.415505 23462 rpcz_store.cc:276] 0709 13:12:30.314393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.314934 (+   541us) service_pool.cc:224] Handling call
0709 13:12:31.414618 (+1099684us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54000b6654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:31.414742 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31552896}
I20260709 13:12:31.297425 25652 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.404480 22661 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.422154 25601 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.423171 25635 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.423311 25667 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:31.427347 25521 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:31.428342 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.429762 25691 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:31.430229 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.430742 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.431368 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.431999 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 702759us
I20260709 13:12:31.434010 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.435278 22833 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:31.436136 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:31.433452 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:31.445359 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.420351 22661 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 4, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:31.450654 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.458446 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:31.459336 23462 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:31.307670 25589 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.471875 23465 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:31.424062 25667 consensus_queue.cc:1243] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:31.481316 25662 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.465307 22661 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:31.360404 25496 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.485937 25496 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:31.483978 25662 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.485432 22661 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.489833 22661 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:31.490171 25610 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:31.468936 25618 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:12:31.382929 25550 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.490754 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:31.492348 22805 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.551s (SENT)
I20260709 13:12:31.539168 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:31.457242 25665 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:12:31.369074 25664 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.542661 22661 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 4 to 5, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:31.535847 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 220103us
I20260709 13:12:31.546167 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:31.543030 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50729.2us
W20260709 13:12:31.548592 22805 leader_election.cc:336] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.578s (SENT)
I20260709 13:12:31.549860 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:31.550675 22836 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.564376 25624 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:31.568490 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 324) took 1254 ms. Trace:
I20260709 13:12:31.568982 23458 rpcz_store.cc:276] 0709 13:12:30.314032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.318154 (+  4122us) service_pool.cc:224] Handling call
0709 13:12:31.568254 (+1250100us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400740254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:31.568412 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430118656}
I20260709 13:12:31.569584 25664 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.570955 25618 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:31.571346 25516 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:31.571585 25618 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.495074 25576 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:31.572466 25576 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.573655 25576 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:31.574098 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83438.8us
I20260709 13:12:31.576545 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.580646 25550 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.581928 23466 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:31.397051 25533 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.571938 23462 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:31.592482 23462 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:31.593719 22836 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:31.587978 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:31.470240 25659 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:31.470048 25601 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:31.624202 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.007s	sys 0.036s
I20260709 13:12:31.591250 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:31.622678 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76590.4us
I20260709 13:12:31.593438 23462 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.462445 25676 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 13:12:31.457401 25611 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.577343 25658 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:31.540441 25682 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:12:31.636368 23462 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.564365 25575 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:31.572832 25618 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:31.634510 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.008s	sys 0.036s
I20260709 13:12:31.599182 25579 raft_consensus.cc:2764] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:31.636382 22732 leader_election.cc:400] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:31.460537 25635 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.581789 25483 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:31.654475 22732 leader_election.cc:403] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:31.658318 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.633035 25636 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:31.661401 22832 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:12:31.616096 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.636943 25686 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:31.665616 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.666304 22760 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
W20260709 13:12:31.668013 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76943.9us
I20260709 13:12:31.648927 25601 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:31.669621 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2198) took 1349 ms. Trace:
I20260709 13:12:31.672820 23462 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:12:31.678769 23462 pending_rounds.cc:85] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 6
I20260709 13:12:31.679111 23462 pending_rounds.cc:107] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:12:31.648270 25575 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.679443 23462 raft_consensus.cc:2889] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:31.675835 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:31.624168 25521 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:12:31.687829 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:31.688509 22835 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:31.633416 25259 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:12:31.599829 25664 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:12:31.659256 25610 consensus_queue.cc:1243] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:31.676748 22910 rpcz_store.cc:276] 0709 13:12:30.316507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.333329 (+ 16822us) service_pool.cc:224] Handling call
0709 13:12:30.385492 (+ 52163us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:30.535195 (+149703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:30.535224 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:30.555430 (+ 20206us) log.cc:844] Serialized 52 byte log entry
0709 13:12:30.791520 (+236090us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:30.878098 (+ 86578us) log.cc:844] Serialized 14 byte log entry
0709 13:12:30.882930 (+  4832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.883041 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:31.288380 (+405339us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b1800380320. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:31.338507 (+ 50127us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b1800380320. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1
0709 13:12:31.339387 (+   880us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b1800380320. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:31.665540 (+326153us) raft_consensus.cc:1685] finished
0709 13:12:31.665564 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:31.665709 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162214,"prepare.queue_time_us":95272,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":69437,"raft.queue_time_us":159862,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":29832,"spinlock_wait_cycles":156570880,"thread_start_us":208475,"threads_started":2,"wal-append.queue_time_us":251226}
I20260709 13:12:31.698232 22835 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:31.701149 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.676051 25618 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:31.666427 25659 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:31.696779 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:31.707955 22760 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:31.683391 25691 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:31.696941 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:31.709044 22765 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:31.694643 25564 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:12:31.689843 25483 raft_consensus.cc:2764] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:12:31.705490 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:31.685774 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 97900.7us
I20260709 13:12:31.706837 22835 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:12:31.690676 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 54410.2us
I20260709 13:12:31.726512 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:31.721105 22765 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:31.695752 25636 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:31.731472 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.699736 25686 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.686131 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:31.720956 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:31.738063 22833 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:31.734891 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.735270 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:31.714550 25610 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:31.742240 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.742890 22765 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:31.746074 22765 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:31.748224 22765 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:31.750065 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:31.750844 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.716888 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:31.756078 22765 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:31.759224 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:31.719007 25707 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:31.758841 22836 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:31.757925 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:31.767957 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 51197.7us
I20260709 13:12:31.766366 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.768740 22766 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:31.769469 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.754484 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.734530 25664 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:31.770934 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:31.755515 25626 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.545s	user 0.005s	sys 0.006s
I20260709 13:12:31.779414 25701 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:31.779969 25701 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:31.780836 25701 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.782688 23462 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:31.785429 22912 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:12:31.770951 25634 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.778715 25516 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:31.786175 25634 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.775734 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.788532 25634 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:31.788951 25516 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:31.728938 25697 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.790803 25611 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.791147 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:31.791843 25697 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.736251 25691 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.718089 25659 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:31.815171 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 358567us
W20260709 13:12:31.827523 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.794634 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.828694 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.793594 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:31.768795 25643 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.921s	user 0.011s	sys 0.007s
I20260709 13:12:31.770469 25550 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:31.769353 25707 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:31.843427 25707 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.843654 25634 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.844193 25634 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.766880 25622 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:31.767283 25610 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:31.842047 25533 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:31.863559 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:31.845450 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69805us
W20260709 13:12:31.849658 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.772168 25709 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:31.861644 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.830258 25695 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.862443 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:31.862959 22906 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:31.894666 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:31.865382 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:31.861958 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.774181 25664 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:31.888187 22911 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:31.902611 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:31.868396 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:31.904026 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:31.867370 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:31.906735 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.908416 22915 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:31.764546 25576 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:31.869069 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:31.942463 22763 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:31.827816 25629 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:31.944216 22913 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:31.944216 25629 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:31.895658 22909 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 5: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:31.945464 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.946117 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.946664 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.948248 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.945339 25629 consensus_queue.cc:1243] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:31.950587 25629 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:31.758535 25602 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:12:31.897223 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.961874 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:31.925011 22915 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:31.935158 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61910.3us
I20260709 13:12:31.936314 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:31.939663 22906 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.860050 25626 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.870883 25550 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:31.965683 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 323) took 1652 ms. Trace:
I20260709 13:12:31.957258 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:31.906279 25664 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:31.769765 25601 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.970400 22834 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:31.926321 25622 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:31.933123 25658 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:31.979872 22909 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:31.979930 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 185103us
I20260709 13:12:31.933854 25667 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.932885 25533 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:31.925081 25705 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.983723 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:31.892076 25521 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:12:31.892011 25188 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:12:31.960278 25635 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:32.024943 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:31.986732 25531 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.850s	user 0.017s	sys 0.002s
W20260709 13:12:31.988322 25703 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
W20260709 13:12:32.033696 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 172508us
I20260709 13:12:32.034739 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:32.007100 25622 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:32.060657 22760 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:31.975335 25626 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.024752 25533 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:32.065346 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:31.925968 25686 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:31.982033 25636 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:12:32.036934 25635 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:31.934284 25695 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:31.962540 25691 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:32.070225 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 204948us
I20260709 13:12:32.050657 25709 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:32.064270 25622 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:32.069478 25188 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.089744 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:32.093402 22911 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.069289 25533 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.075376 25701 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:32.073706 25635 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:32.072223 25576 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:32.074910 25700 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:32.034353 25667 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:32.095371 25706 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.322s	user 0.001s	sys 0.001s
I20260709 13:12:32.103441 25496 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.122673 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:32.123456 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81944.4us
I20260709 13:12:32.032598 25610 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.033999 25705 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.106843 25634 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:32.140266 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:32.069768 25550 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.138434 22911 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:32.135964 22906 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:32.133001 25700 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.139052 23460 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:12:32.142658 25700 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.108850 25188 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.133147 25635 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.146910 25635 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:32.147497 25188 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:32.137286 25659 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:32.150362 25602 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:32.156464 25515 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:32.099071 25731 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 13:12:32.164842 25721 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 13:12:32.167133 25725 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:12:32.133971 25714 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
I20260709 13:12:32.125567 25727 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.174226 22909 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:32.196493 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:32.199057 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76476.8us
I20260709 13:12:32.199949 22732 leader_election.cc:400] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:32.200060 22835 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:32.200273 22732 leader_election.cc:403] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:32.202441 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:32.167213 25693 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.889s	user 0.005s	sys 0.000s
W20260709 13:12:32.169485 25719 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:12:32.223315 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:32.232272 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.206420 25723 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:12:32.238000 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 213174us
I20260709 13:12:32.243913 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:32.226230 25724 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:12:32.204316 25635 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.188558 25736 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:32.140920 25496 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.283012 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2203) took 1254 ms. Trace:
I20260709 13:12:32.225869 25586 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:32.285247 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.224608 25573 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.203835 25701 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:32.285959 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 86156.7us
W20260709 13:12:32.291563 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47749.5us
W20260709 13:12:32.296154 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 391025us
I20260709 13:12:32.240722 25188 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:32.299546 22732 leader_election.cc:400] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
I20260709 13:12:32.301393 22732 leader_election.cc:403] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:32.301966 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 59889.6us
I20260709 13:12:32.289443 22907 rpcz_store.cc:276] 0709 13:12:31.028955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.029106 (+   151us) service_pool.cc:224] Handling call
0709 13:12:31.063019 (+ 33913us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.232649 (+169630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.232704 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.267686 (+ 34982us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.590395 (+322709us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:31.590437 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.590518 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.252821 (+662303us) raft_consensus.cc:1685] finished
0709 13:12:32.282216 (+ 29395us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.282919 (+   703us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100589,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":38977,"thread_start_us":466767,"threads_started":2,"wal-append.queue_time_us":454489}
W20260709 13:12:32.300141 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72095.9us
I20260709 13:12:32.227458 25634 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 13:12:32.314127 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 79448.2us
I20260709 13:12:32.288652 25545 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
W20260709 13:12:32.298032 25681 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:12:32.317801 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:32.297812 25586 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.297868 25537 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:32.347114 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:32.349537 22835 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:32.346220 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46813us
I20260709 13:12:32.351446 22732 leader_election.cc:400] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:32.351384 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:32.351843 22732 leader_election.cc:403] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:32.354367 22838 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:32.300952 25188 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.315410 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:32.388882 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73603.6us
I20260709 13:12:32.353338 25521 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.293893 25635 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.250298 25576 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.308156 25738 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:12:32.394651 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.395328 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44050.6us
W20260709 13:12:32.404536 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.395534 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.396623 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45317.1us
I20260709 13:12:32.208145 25727 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.406412 22763 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:32.407094 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:32.407285 25727 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:32.407886 22758 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:32.407881 25727 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:32.408692 25727 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.410380 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:32.410718 25727 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:32.305064 25573 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.304687 25701 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.410673 25736 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:32.411533 25727 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:32.356467 25586 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.412910 25573 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:32.413033 25701 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:32.413341 25573 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:32.413698 25573 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:32.413939 25573 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:32.353446 25626 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:32.418751 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.358076 23460 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:32.419488 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.419555 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.387966 25188 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:32.421123 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:32.418883 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.411099 22765 raft_consensus.cc:2368] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7.
I20260709 13:12:32.422242 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:32.423345 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:32.425489 22766 raft_consensus.cc:2368] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7.
I20260709 13:12:32.424862 22759 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:32.426450 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.319265 25313 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:12:32.419957 25573 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.390069 25537 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:32.428795 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:32.431842 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:32.430179 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:32.432785 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.398239 25636 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:32.434373 22909 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:32.437579 22913 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:32.438614 22766 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:32.438906 25715 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
W20260709 13:12:32.440136 25726 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 13:12:32.434319 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.433486 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.443895 22763 raft_consensus.cc:2410] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:32.284705 25709 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:32.402200 25521 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:32.446343 22913 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:32.780236 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1370) took 1348 ms. Trace:
I20260709 13:12:32.780691 22841 rpcz_store.cc:276] 0709 13:12:31.432133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.462350 (+ 30217us) service_pool.cc:224] Handling call
0709 13:12:31.480916 (+ 18566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.569751 (+ 88835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.569790 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:31.569836 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:31.569872 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.779944 (+1210072us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b54001ff4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:32.779997 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.780169 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203723136}
I20260709 13:12:32.480857 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:32.658831 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.796707 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.748417 22833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.001s	sys 0.025s
W20260709 13:12:32.798413 22833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.001s	sys 0.027s
W20260709 13:12:32.788689 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 381644us
I20260709 13:12:32.796350 22759 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:32.454341 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.446228 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1371) took 1013 ms. Trace:
I20260709 13:12:32.804885 22837 rpcz_store.cc:276] 0709 13:12:31.432453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.433069 (+   616us) service_pool.cc:224] Handling call
0709 13:12:31.433444 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.433878 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.433907 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.434396 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.566162 (+131766us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b4800130158. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:12:31.663566 (+ 97404us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:31.663609 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.663677 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.440562 (+776885us) raft_consensus.cc:1685] finished
0709 13:12:32.440584 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.446161 (+  5577us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2531,"raft.run_cpu_time_us":2059,"raft.run_wall_time_us":110363,"spinlock_wait_cycles":12150656,"thread_start_us":228777,"threads_started":1,"wal-append.queue_time_us":334554}
I20260709 13:12:31.966274 23459 rpcz_store.cc:276] 0709 13:12:30.313380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.317955 (+  4575us) service_pool.cc:224] Handling call
0709 13:12:31.351036 (+1033081us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:31.351321 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.351621 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:31.694254 (+342633us) log.cc:844] Serialized 14 byte log entry
0709 13:12:31.929464 (+235210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:31.941490 (+ 12026us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:31.956762 (+ 15272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.965396 (+  8634us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:31.965595 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23842,"prepare.queue_time_us":388645,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":4144,"raft.queue_time_us":65445,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3928,"spinlock_wait_cycles":61860736,"thread_start_us":465822,"threads_started":2}
W20260709 13:12:32.781399 25664 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.787s	user 0.010s	sys 0.002s
W20260709 13:12:32.654974 25743 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 13:12:32.447310 25733 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:12:32.450044 25313 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:32.634155 25749 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:12:33.358598 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:32.454109 25695 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:32.533200 25729 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.401s	user 0.001s	sys 0.000s
W20260709 13:12:32.448071 25737 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:12:33.326175 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.001s	sys 0.011s
W20260709 13:12:33.246413 25700 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.103s	user 0.015s	sys 0.000s
I20260709 13:12:33.392644 25700 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:33.396178 25700 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:32.455274 25732 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:32.857540 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 464380us
I20260709 13:12:33.405069 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:33.367667 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.004s	sys 0.012s
W20260709 13:12:33.405929 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.004s	sys 0.012s
W20260709 13:12:33.406255 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 745728us
I20260709 13:12:33.358685 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:32.992079 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:32.451227 25496 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:32.808434 25530 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.778452 25748 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:33.421142 25748 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.421705 25748 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:32.446448 25750 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6.
W20260709 13:12:32.822216 25515 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.002s
W20260709 13:12:33.125252 25601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.022s	sys 0.050s
I20260709 13:12:33.079447 25757 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:33.426265 25750 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:33.078883 25581 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.010s	sys 0.057s
W20260709 13:12:33.427009 25581 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.010s	sys 0.064s
I20260709 13:12:33.426800 25750 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.132529 25635 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:33.427716 25760 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 13:12:33.383509 22759 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
W20260709 13:12:33.353515 25610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.000s	sys 0.021s
W20260709 13:12:33.428395 25610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.001s	sys 0.021s
I20260709 13:12:33.428345 22759 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:33.429124 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1387) took 1232 ms. Trace:
I20260709 13:12:33.429783 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:33.429677 22833 rpcz_store.cc:276] 0709 13:12:32.196300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.197637 (+  1337us) service_pool.cc:224] Handling call
0709 13:12:32.198403 (+   766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.198805 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.198829 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.217975 (+ 19146us) log.cc:844] Serialized 548 byte log entry
0709 13:12:32.798965 (+580990us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:32.799026 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.799125 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.428868 (+629743us) raft_consensus.cc:1685] finished
0709 13:12:33.428889 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.429029 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":580553,"threads_started":1,"wal-append.queue_time_us":862102}
W20260709 13:12:33.431164 25746 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.030s	user 0.001s	sys 0.013s
I20260709 13:12:33.431073 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:33.426481 25601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.024s	sys 0.052s
I20260709 13:12:33.432276 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.433305 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:33.433904 22759 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:33.386075 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.002s	sys 0.011s
I20260709 13:12:33.367244 25313 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:32.986178 25618 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.262s	user 0.013s	sys 0.000s
I20260709 13:12:33.390722 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:33.445863 22907 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:32.918705 25664 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.407327 22765 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:33.039125 25634 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:33.416286 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.419395 25496 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.454241 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:33.454720 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.455497 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:33.456280 22907 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:33.420086 25530 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:33.459082 25530 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:33.136615 25573 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:32.723728 25731 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.461468 25731 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:33.462060 25573 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:33.462147 25750 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:33.462749 25573 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:33.463697 25731 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:33.454306 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.464937 22914 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:33.465171 22765 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:33.426442 25759 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:12:33.453423 25634 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.454512 25496 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.483429 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:33.426806 25757 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:12:33.457312 22907 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:32.453979 25755 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.963s
I20260709 13:12:32.454200 25537 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:33.409261 25521 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:33.472339 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.657104 25575 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:33.474253 22910 raft_consensus.cc:1217] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-5.3]   Dedup: 3.2->[5.3-5.3]
I20260709 13:12:33.500170 22910 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
W20260709 13:12:33.446516 25622 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.350s	user 0.015s	sys 0.000s
I20260709 13:12:33.459091 25664 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:33.487255 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.487484 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.509393 22761 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:33.509714 22765 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:33.492518 22759 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:33.510977 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:33.512356 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.513026 22759 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:33.513895 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.514365 22759 raft_consensus.cc:2376] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6. Re-sending same reply.
I20260709 13:12:33.514787 22906 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:12:33.515158 22906 pending_rounds.cc:85] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:12:33.515108 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:33.515518 22906 pending_rounds.cc:107] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:12:33.515761 22759 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:33.493578 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.495481 25530 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:33.516757 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.463721 25573 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.519670 25732 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:33.520241 25732 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.520752 25622 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:33.522513 25732 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:33.496759 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1324) took 1562 ms. Trace:
I20260709 13:12:33.522903 25622 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:33.523085 22836 rpcz_store.cc:276] 0709 13:12:31.933941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.934094 (+   153us) service_pool.cc:224] Handling call
0709 13:12:31.934522 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.971666 (+ 37144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.971718 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.026095 (+ 54377us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.230414 (+204319us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:32.230457 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.230525 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.496522 (+1265997us) raft_consensus.cc:1685] finished
0709 13:12:33.496547 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.496685 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36646,"raft.queue_time_us":37800,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":18001,"thread_start_us":165962,"threads_started":1,"wal-append.queue_time_us":350122}
I20260709 13:12:33.525345 22833 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:33.516971 25530 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:33.504766 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:33.506544 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1374) took 1754 ms. Trace:
W20260709 13:12:33.525463 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.529951 22910 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:33.547053 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:33.530108 22757 rpcz_store.cc:276] 0709 13:12:31.752566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.752743 (+   177us) service_pool.cc:224] Handling call
0709 13:12:31.753138 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.762886 (+  9748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.762919 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.770418 (+  7499us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.069588 (+299170us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:32.069643 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.069727 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.506266 (+1436539us) raft_consensus.cc:1685] finished
0709 13:12:33.506299 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.506465 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9917,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6264,"thread_start_us":205925,"threads_started":1,"wal-append.queue_time_us":350088}
I20260709 13:12:33.549958 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.515774 22906 raft_consensus.cc:2889] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:33.531867 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 532) took 1663 ms. Trace:
I20260709 13:12:33.552049 23462 rpcz_store.cc:276] 0709 13:12:31.868709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.868850 (+   141us) service_pool.cc:224] Handling call
0709 13:12:31.869133 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.872851 (+  3718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.872893 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.897326 (+ 24433us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.083139 (+185813us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:32.083169 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.083226 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.531636 (+1448410us) raft_consensus.cc:1685] finished
0709 13:12:33.531655 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.531803 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4972,"raft.queue_time_us":22990,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":28652,"thread_start_us":111865,"threads_started":1,"wal-append.queue_time_us":290341}
I20260709 13:12:33.532871 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:33.554592 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2093) took 1038 ms. Trace:
I20260709 13:12:33.554867 22841 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:33.554960 22757 rpcz_store.cc:276] 0709 13:12:32.516497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.549671 (+1033174us) service_pool.cc:224] Handling call
0709 13:12:33.554531 (+  4860us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:33.556536 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.532377 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1328) took 1218 ms. Trace:
I20260709 13:12:33.557538 22840 rpcz_store.cc:276] 0709 13:12:32.313539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.313677 (+   138us) service_pool.cc:224] Handling call
0709 13:12:32.314081 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.314625 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.314659 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.345185 (+ 30526us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.789062 (+443877us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:32.789126 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.789236 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:33.532142 (+742906us) raft_consensus.cc:1685] finished
0709 13:12:33.532167 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.532312 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":28039,"spinlock_wait_cycles":741760,"thread_start_us":384111,"threads_started":1,"wal-append.queue_time_us":652194}
I20260709 13:12:33.557694 22757 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:33.565138 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:34.320442 22757 raft_consensus.cc:2376] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
W20260709 13:12:34.322497 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2121) took 1520 ms (client timeout 1911 ms). Trace:
W20260709 13:12:34.323372 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.000s	sys 0.010s
W20260709 13:12:34.323899 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.000s	sys 0.010s
I20260709 13:12:33.542564 22759 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:34.325336 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2099) took 1806 ms (client timeout 2056 ms). Trace:
W20260709 13:12:34.326121 22764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.004s	sys 0.021s
W20260709 13:12:34.322645 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:34.326683 22764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.004s	sys 0.021s
I20260709 13:12:33.595604 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:34.326309 22759 rpcz_store.cc:269] 0709 13:12:32.518476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.514897 (+996421us) service_pool.cc:224] Handling call
0709 13:12:34.325112 (+810215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.329348 22765 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:34.331714 22879 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.004s	sys 0.008s
W20260709 13:12:34.332257 22879 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.005s	sys 0.009s
W20260709 13:12:34.332903 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 658175us
W20260709 13:12:33.832417 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.565858 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:34.338940 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.339087 22761 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:34.339977 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:33.775382 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.472s	user 0.038s	sys 0.048s
W20260709 13:12:34.341718 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.039s	user 0.061s	sys 0.061s
W20260709 13:12:34.342198 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 2.03928e+06us
W20260709 13:12:33.827354 25188 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.393s	user 0.012s	sys 0.002s
I20260709 13:12:34.344732 25188 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:33.641669 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.031s	sys 0.009s
W20260709 13:12:34.346761 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.424s	user 0.040s	sys 0.054s
I20260709 13:12:33.974460 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 527) took 3351 ms. Trace:
I20260709 13:12:34.347767 23466 rpcz_store.cc:276] 0709 13:12:30.623275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.712239 (+ 88964us) service_pool.cc:224] Handling call
0709 13:12:30.712731 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.582432 (+869701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.974072 (+2391640us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b540063f954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:33.974373 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998340352}
I20260709 13:12:33.613456 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:34.350008 25188 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:34.351361 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 737781us
I20260709 13:12:34.212569 25496 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:33.542593 25313 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.356483 25313 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:34.346765 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.026s	sys 0.013s
W20260709 13:12:34.357913 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.026s	sys 0.014s
W20260709 13:12:34.358410 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:34.358470 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 953443us
W20260709 13:12:34.359124 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.360953 25496 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:34.361549 25496 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.361583 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.362074 25496 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:33.528055 25530 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.363523 25516 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.364602 25516 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.365231 25515 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:34.366382 25515 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.368592 25516 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:33.824842 25533 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.703s	user 0.016s	sys 0.000s
I20260709 13:12:34.371726 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1373) took 2938 ms. Trace:
I20260709 13:12:34.371809 25533 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.372748 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.373911 23466 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:34.374495 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:34.375499 25533 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:34.323302 22757 rpcz_store.cc:269] 0709 13:12:32.801654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.564822 (+763168us) service_pool.cc:224] Handling call
0709 13:12:34.322285 (+757463us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.414031 25434 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:34.125247 25575 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:33.554529 25622 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.423313 25434 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:34.424474 25622 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:34.391009 22911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 296) took 2528 ms (client timeout 2810 ms). Trace:
I20260709 13:12:34.395551 22760 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:34.385915 25576 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.408200 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:33.676908 25610 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.382030 22759 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:34.392547 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.411017 25188 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.554749 25687 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:33.532951 25636 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.089s	user 0.005s	sys 0.005s
I20260709 13:12:34.399749 25515 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:33.968238 25758 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.372592 22832 rpcz_store.cc:276] 0709 13:12:31.432962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.462438 (+ 29476us) service_pool.cc:224] Handling call
0709 13:12:31.462770 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.662878 (+200108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.370655 (+2707777us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540028a5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:34.371246 (+   591us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455974144}
I20260709 13:12:34.383244 22908 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.426492 22760 pending_rounds.cc:85] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 2
I20260709 13:12:34.376767 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:34.428745 22913 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:34.433113 22758 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:34.377256 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:34.445832 22806 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.009s	sys 0.010s
W20260709 13:12:34.446362 22806 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.010s	sys 0.010s
W20260709 13:12:34.446919 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 962871us
W20260709 13:12:34.383144 22912 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2213) took 3018 ms (client timeout 2283 ms). Trace:
W20260709 13:12:34.428267 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2102) took 1850 ms (client timeout 2192 ms). Trace:
W20260709 13:12:34.448990 22759 rpcz_store.cc:269] 0709 13:12:32.577470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.329959 (+1752489us) service_pool.cc:224] Handling call
0709 13:12:34.428128 (+ 98169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.386931 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:34.440040 22760 pending_rounds.cc:107] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 4.3
W20260709 13:12:34.447552 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1364) took 1671 ms (client timeout 2221 ms). Trace:
I20260709 13:12:34.427718 25622 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:33.556820 25731 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:34.428455 25610 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.429268 25188 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.456966 25188 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:34.452430 22913 rpcz_store.cc:269] 0709 13:12:32.771285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.781013 (+  9728us) service_pool.cc:224] Handling call
0709 13:12:34.391986 (+1610973us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:34.442344 (+ 50358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19242880}
W20260709 13:12:33.755666 25720 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.723s	user 0.007s	sys 0.009s
I20260709 13:12:34.457894 25530 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.556650 25767 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.916s
W20260709 13:12:33.696205 25716 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.918s	user 0.010s	sys 0.010s
W20260709 13:12:34.097515 25710 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 2.306s	user 0.000s	sys 0.018s
I20260709 13:12:34.458755 25697 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:33.977636 25748 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.002s
W20260709 13:12:34.108608 25747 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.473s	user 0.005s	sys 0.008s
I20260709 13:12:34.459419 25533 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.266602 25757 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:34.461632 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:34.032047 25491 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.018s	sys 0.006s
I20260709 13:12:34.505088 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 534) took 2379 ms. Trace:
I20260709 13:12:34.480917 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.505725 23458 rpcz_store.cc:276] 0709 13:12:32.125459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.128145 (+  2686us) service_pool.cc:224] Handling call
0709 13:12:32.142753 (+ 14608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.144249 (+  1496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.144286 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.144803 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.222642 (+ 77839us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:32.222706 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.222813 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.107835 (+1885022us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:34.504429 (+396594us) raft_consensus.cc:1685] finished
0709 13:12:34.504485 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.504847 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1032,"raft.queue_time_us":2548,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":63116416,"thread_start_us":76913,"threads_started":1,"wal-append.queue_time_us":295797}
W20260709 13:12:34.507324 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1362) took 1922 ms (client timeout 1536 ms). Trace:
I20260709 13:12:34.464579 25718 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.507721 22909 rpcz_store.cc:269] 0709 13:12:32.584593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.584918 (+   325us) service_pool.cc:224] Handling call
0709 13:12:34.480469 (+1895551us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:34.507244 (+ 26775us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15747456}
I20260709 13:12:34.508296 25718 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.509050 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:34.509409 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.510103 22835 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:34.510200 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.511030 22840 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:34.511245 25718 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:34.463183 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2091) took 1969 ms (client timeout 1870 ms). Trace:
I20260709 13:12:34.451372 22760 raft_consensus.cc:2889] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:34.511890 22758 rpcz_store.cc:269] 0709 13:12:32.480649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.480792 (+   143us) service_pool.cc:224] Handling call
0709 13:12:34.407663 (+1926871us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:34.449723 (+ 42060us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486144}
I20260709 13:12:34.512028 25718 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:34.512591 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2088) took 2079 ms. Trace:
I20260709 13:12:33.756899 25581 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:34.426070 22911 rpcz_store.cc:269] 0709 13:12:31.862759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.903746 (+ 40987us) service_pool.cc:224] Handling call
0709 13:12:34.390446 (+2486700us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:34.390731 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433534848}
I20260709 13:12:34.406739 25259 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:34.513429 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:34.476183 22915 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:34.513726 25581 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:34.514438 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2122) took 1531 ms (client timeout 2027 ms). Trace:
I20260709 13:12:34.515091 25776 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:34.515918 25776 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.516646 25581 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.369421 25516 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.973112 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1365) took 1158 ms. Trace:
I20260709 13:12:34.518133 22834 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:34.123258 25624 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:34.216975 25589 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:34.518419 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1368) took 3865 ms. Trace:
I20260709 13:12:34.518975 22839 rpcz_store.cc:276] 0709 13:12:30.652603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.652746 (+   143us) service_pool.cc:224] Handling call
0709 13:12:30.653031 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:30.653379 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:30.653408 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:30.653841 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.341125 (+687284us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b1800134da0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:12:31.675816 (+334691us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:31.675856 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.675923 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.096650 (+2420727us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:34.517937 (+421287us) raft_consensus.cc:1685] finished
0709 13:12:34.517981 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.518203 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15654,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":28217,"spinlock_wait_cycles":9239808,"thread_start_us":330689,"threads_started":1,"wal-append.queue_time_us":1090334}
I20260709 13:12:34.470969 25530 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.427376 25576 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.521879 25516 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.522971 25530 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.523547 25624 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:34.430425 25626 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:34.486858 25709 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.425436 25434 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.431717 25758 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.518028 22914 rpcz_store.cc:276] 0709 13:12:32.814780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.855305 (+ 40525us) service_pool.cc:224] Handling call
0709 13:12:33.972673 (+1117368us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b54003381d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:33.972953 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101245184}
I20260709 13:12:34.488386 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:34.037573 25573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.005s	sys 0.011s
I20260709 13:12:34.493106 25533 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.494345 22837 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:34.495256 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.497833 25667 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:33.691442 25586 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.279s	user 0.009s	sys 0.003s
W20260709 13:12:34.499101 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 122566us
I20260709 13:12:34.463316 25731 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.500046 25635 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:34.498976 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.125885 25705 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:34.502219 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57563.6us
I20260709 13:12:34.502442 25757 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.429790 25636 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:33.686573 25634 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.511389 22909 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:34.516340 22838 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:34.576066 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:34.448108 22912 rpcz_store.cc:269] 0709 13:12:31.364353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.364504 (+   151us) service_pool.cc:224] Handling call
0709 13:12:34.382410 (+3017906us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54000ea954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:34.382761 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939258240}
I20260709 13:12:34.480876 25564 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:34.523752 25576 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.499140 22757 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:34.483820 25697 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:34.514849 22758 rpcz_store.cc:269] 0709 13:12:32.982888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.513024 (+1530136us) service_pool.cc:224] Handling call
0709 13:12:34.514355 (+  1331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.513067 22760 rpcz_store.cc:276] 0709 13:12:32.479381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.537955 (+ 58574us) service_pool.cc:224] Handling call
0709 13:12:34.394104 (+1856149us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:34.394641 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.512275 (+117634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.512489 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41287552}
I20260709 13:12:34.429718 25687 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:33.688548 25707 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.845s	user 0.011s	sys 0.003s
W20260709 13:12:34.117347 25752 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.397s	user 0.013s	sys 0.000s
W20260709 13:12:34.555359 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61022.1us
I20260709 13:12:34.562479 25624 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.564901 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:33.594813 25765 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:34.525980 25610 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.518556 22834 pending_rounds.cc:85] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 3
I20260709 13:12:34.578513 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:34.498901 25748 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.584357 25709 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.585794 22832 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:34.585988 25757 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.587416 25636 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:34.513864 25259 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.635699 25259 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.636384 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.637156 22840 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:34.577656 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4923769750000702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 13:12:34.570083 25635 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.638176 25635 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:34.638502 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:34.639168 22840 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:34.605477 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 136323us
I20260709 13:12:34.640475 25748 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:34.640542 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.942s (SENT) [suppressed 35 similar messages]
I20260709 13:12:34.617895 22834 pending_rounds.cc:107] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:34.641872 22834 raft_consensus.cc:2889] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:34.642037 23432 connection.cc:256] RPC call timeout handler was delayed by -1.3506638100000146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.281s for a response
W20260709 13:12:34.642539 23432 connection.cc:256] RPC call timeout handler was delayed by -1.2884475520000933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:12:34.643105 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.643481 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:34.602152 25573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.005s	sys 0.012s
I20260709 13:12:34.644673 25697 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.645221 25636 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.647981 25636 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.648523 25667 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.649912 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:34.652809 23461 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:34.629683 22839 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:34.647974 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.697433 22761 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:34.632043 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.616854 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.635829 25757 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.649255 25667 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.699141 22838 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:34.591341 25516 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.700312 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.701860 22837 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:34.703815 25667 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.637718 22914 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:34.644536 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.706275 22836 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:34.645731 25573 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.648140 25586 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.624866 25765 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.709908 25573 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.641806 25707 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:34.711692 25586 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.712036 22910 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:34.712163 25573 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.712321 25707 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.625394 25624 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:34.712874 22907 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:34.653251 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.717152 25761 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.718001 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.717876 25761 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:34.718488 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4302769749999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 13:12:34.720438 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3924769750000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 13:12:34.723254 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.725277 22838 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:34.711714 25765 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.731176 22833 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:34.619045 22835 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:34.738231 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:34.724932 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3474769750000632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
I20260709 13:12:34.644762 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.630278 22911 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:34.631283 25709 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.739631 22915 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:34.747495 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1363) took 2073 ms. Trace:
I20260709 13:12:34.649271 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.748914 23458 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:34.749253 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.748284 22915 rpcz_store.cc:276] 0709 13:12:32.673930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.719016 (+ 45086us) service_pool.cc:224] Handling call
0709 13:12:34.385266 (+1666250us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:34.747291 (+362025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38483200}
I20260709 13:12:34.759649 22908 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:34.764499 22765 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:34.753702 22806 connection.cc:256] RPC call timeout handler was delayed by -1.298041583999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 13:12:34.768836 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2094) took 2252 ms (client timeout 2027 ms). Trace:
I20260709 13:12:34.795419 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.769282 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:34.801769 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:34.791818 22766 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:34.769323 22761 rpcz_store.cc:269] 0709 13:12:32.516804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.510398 (+993594us) service_pool.cc:224] Handling call
0709 13:12:34.753551 (+1243153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.814810 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:34.815703 22760 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4.
W20260709 13:12:34.785118 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2100) took 2266 ms (client timeout 2068 ms). Trace:
I20260709 13:12:34.827023 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.829946 22757 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:34.775022 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1704883950000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
I20260709 13:12:34.790118 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.832296 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:34.828445 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2124) took 1737 ms (client timeout 2064 ms). Trace:
W20260709 13:12:34.835147 22760 rpcz_store.cc:269] 0709 13:12:33.078717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.648921 (+1570204us) service_pool.cc:224] Handling call
0709 13:12:34.816321 (+167400us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:34.826574 22765 rpcz_store.cc:269] 0709 13:12:32.518757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.548223 (+1029466us) service_pool.cc:224] Handling call
0709 13:12:34.765298 (+1217075us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.838208 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:34.841740 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2110) took 2250 ms (client timeout 2240 ms). Trace:
I20260709 13:12:34.793816 25575 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.844151 22914 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:34.832211 22758 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:34.832518 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68741.4us
W20260709 13:12:34.849072 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2123) took 1844 ms (client timeout 2034 ms). Trace:
W20260709 13:12:34.849655 22758 rpcz_store.cc:269] 0709 13:12:33.004709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.658930 (+1654221us) service_pool.cc:224] Handling call
0709 13:12:34.848988 (+190058us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:34.844223 22757 rpcz_store.cc:269] 0709 13:12:32.579847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.376712 (+1796865us) service_pool.cc:224] Handling call
0709 13:12:34.830533 (+453821us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.851620 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.852368 25731 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:34.853379 23459 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:34.859527 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:34.860407 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:34.861678 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:34.862534 25496 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.838212 25776 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.865828 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.866169 25776 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:34.866531 22915 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:34.866874 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:34.867663 25313 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.004s
I20260709 13:12:34.867945 25776 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:34.868418 25776 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.868784 25776 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:34.862114 22834 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:34.871284 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.853212 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.867786 23461 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:12:34.878300 22759 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:34.790302 25667 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:34.792433 25792 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:34.878506 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:34.886134 22806 connection.cc:256] RPC call timeout handler was delayed by -1.130258045000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
I20260709 13:12:34.892424 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:34.892843 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.894939 22840 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:34.897815 22837 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:34.897842 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:34.892454 22832 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:34.889827 25564 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:34.901682 25564 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.902169 25564 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:34.901741 25259 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:34.905736 25575 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:34.906440 22915 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:34.908421 25575 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.913597 25533 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.873811 25496 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:34.914959 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.906529 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.918805 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.922410 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2128) took 1360 ms. Trace:
I20260709 13:12:34.922983 22765 rpcz_store.cc:276] 0709 13:12:33.555170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.883168 (+1327998us) service_pool.cc:224] Handling call
0709 13:12:34.915934 (+ 32766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.927263 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.927565 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:34.928869 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.932933 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.944044 23457 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:34.887589 25792 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:34.914417 25758 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:34.954423 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.920997 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2127) took 1406 ms. Trace:
I20260709 13:12:34.967317 22757 rpcz_store.cc:276] 0709 13:12:33.493838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.851126 (+1357288us) service_pool.cc:224] Handling call
0709 13:12:34.900409 (+ 49283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:34.951663 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:34.965461 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:34.978440 23458 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
W20260709 13:12:34.978796 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 92886us
I20260709 13:12:34.981431 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.993733 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:34.998692 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:34.913609 25667 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.008025 23461 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:35.079483 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.018685 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.014496 22806 connection.cc:256] RPC call timeout handler was delayed by -1.017688243000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:12:35.076172 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 174871us
I20260709 13:12:35.092928 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:35.064918 22758 raft_consensus.cc:2410] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:35.093613 22835 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.031133 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.064810 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.086194 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1368) took 3342 ms. Trace:
I20260709 13:12:35.009608 25530 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:35.107220 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2126) took 1658 ms (client timeout 2119 ms). Trace:
I20260709 13:12:35.008972 22760 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:35.070854 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:35.120209 22758 rpcz_store.cc:269] 0709 13:12:33.449090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.881029 (+1431939us) service_pool.cc:224] Handling call
0709 13:12:35.107064 (+226035us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:35.121091 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2125) took 1718 ms (client timeout 2091 ms). Trace:
I20260709 13:12:35.085804 25718 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:35.063812 25575 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.009341 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.111017 22757 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:35.017834 22761 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:35.114723 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.115823 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:35.129760 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.130749 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.103526 22764 rpcz_store.cc:276] 0709 13:12:31.743824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.755562 (+ 11738us) service_pool.cc:224] Handling call
0709 13:12:31.826687 (+ 71125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.826969 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:31.964204 (+137235us) log.cc:844] Serialized 14 byte log entry
0709 13:12:33.405739 (+1441535us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003931c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:34.893348 (+1487609us) log.cc:844] Serialized 14 byte log entry
0709 13:12:34.999979 (+106631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.009568 (+  9589us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:35.031727 (+ 22159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.074510 (+ 42783us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.086095 (+ 11585us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93328,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":173707,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":10852,"spinlock_wait_cycles":23195520,"thread_start_us":1358317,"threads_started":4,"wal-append.queue_time_us":2307688}
I20260709 13:12:35.083293 25496 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 13:12:35.128772 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 162241us
I20260709 13:12:35.132001 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:34.979034 25731 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:35.119297 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.123205 22760 rpcz_store.cc:269] 0709 13:12:33.402452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.843310 (+1440858us) service_pool.cc:224] Handling call
0709 13:12:35.121039 (+277729us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.124418 25790 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:35.133280 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.133807 25259 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:35.137843 22839 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:35.138480 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.082998 25652 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.255s
I20260709 13:12:35.118621 25748 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.170356 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:35.139372 25635 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
I20260709 13:12:35.140347 22832 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:35.143211 25761 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:35.018266 25586 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:12:35.144613 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:35.146519 25434 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:35.150625 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 291394us
I20260709 13:12:35.150327 25792 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.150514 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.132123 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2129) took 1468 ms (client timeout 1631 ms). Trace:
W20260709 13:12:35.187067 22761 rpcz_store.cc:269] 0709 13:12:33.663620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.878657 (+1215037us) service_pool.cc:224] Handling call
0709 13:12:35.132017 (+253360us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:35.151237 25589 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.002s
I20260709 13:12:35.163724 22835 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:35.158609 25718 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.168123 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.168671 22914 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:35.161896 25259 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.169701 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:35.195000 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50574.4us
I20260709 13:12:35.194918 23457 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:35.008173 25756 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.151s
I20260709 13:12:35.156973 25731 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.161650 25636 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.213649 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.215622 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:35.171701 25790 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.170925 25748 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.204912 25772 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:35.161412 25496 consensus_queue.cc:1243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:12:35.233683 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:35.232920 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.233060 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.239256 22762 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:35.241053 22832 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:35.204867 25259 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.247042 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.247207 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.248811 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.148274 25575 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.242594 22765 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:35.214412 25731 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.253623 22913 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
W20260709 13:12:35.253921 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2116) took 2626 ms (client timeout 3208 ms). Trace:
W20260709 13:12:35.254340 22765 rpcz_store.cc:269] 0709 13:12:32.627346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.955741 (+2328395us) service_pool.cc:224] Handling call
0709 13:12:35.253850 (+298109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.255230 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.255290 25575 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:35.255806 22765 raft_consensus.cc:2376] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6. Re-sending same reply.
I20260709 13:12:35.160928 25530 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.257678 23457 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:35.261318 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:35.262850 22837 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:35.180552 25635 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.192088 22758 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:35.265925 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:35.266041 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54756.2us
I20260709 13:12:35.267832 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.268186 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.269395 22758 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:35.270479 22914 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:35.270730 22761 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:35.271934 22760 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:35.275343 22837 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:35.274750 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.278750 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.259719 25789 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.283268 22838 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:35.285552 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.263604 25792 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.216341 25761 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:12:35.292593 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.258968 25772 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:35.288980 22806 connection.cc:256] RPC call timeout handler was delayed by -1.6168921009999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32080000000000003s for a response
I20260709 13:12:35.289529 22915 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:35.294675 22806 connection.cc:256] RPC call timeout handler was delayed by -1.576402311000038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
I20260709 13:12:35.290349 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:35.295015 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5575023110000075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:12:35.295661 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5157359530001031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2544s for a response
I20260709 13:12:35.286211 22760 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 4. Re-sending same reply.
I20260709 13:12:35.291632 22912 raft_consensus.cc:2368] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
I20260709 13:12:35.292043 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.293243 22758 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:35.297362 22914 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:35.292124 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.301016 22764 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:35.301028 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.312088 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.257337 25564 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.312978 22758 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:35.295527 25575 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:35.318882 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.319572 22765 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:35.277323 25636 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:35.309494 22806 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.524s (SENT)
I20260709 13:12:35.322393 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.325946 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.327296 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:35.327582 25801 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:35.328007 22764 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.329452 25748 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.314471 25793 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:12:35.332144 25718 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.335741 23457 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:35.336192 25667 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.340224 22761 raft_consensus.cc:2376] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
I20260709 13:12:35.325050 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:35.347710 22832 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:35.347750 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.351890 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.352707 22761 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:35.354451 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.307824 25707 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:12:35.321852 25575 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.358548 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.310461 25776 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.335788 25635 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.359282 22758 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:35.360569 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46811us
W20260709 13:12:35.360325 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109023us
I20260709 13:12:35.368126 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.372262 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.374176 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53190.3us
I20260709 13:12:35.322042 25789 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.361786 25575 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.376205 22757 raft_consensus.cc:2368] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:35.380559 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2131) took 1049 ms. Trace:
I20260709 13:12:35.380985 22758 rpcz_store.cc:276] 0709 13:12:34.331344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.370679 (+1039335us) service_pool.cc:224] Handling call
0709 13:12:35.380193 (+  9514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.382512 25537 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:35.384527 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1371) took 1940 ms. Trace:
W20260709 13:12:35.385847 25796 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:12:35.386039 25808 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:35.387176 25530 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.388020 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 507) took 1016 ms. Trace:
I20260709 13:12:35.388389 23462 rpcz_store.cc:276] 0709 13:12:34.371643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.453687 (+ 82044us) service_pool.cc:224] Handling call
0709 13:12:34.454492 (+   805us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:34.501430 (+ 46938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.501493 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:34.582383 (+ 80890us) log.cc:844] Serialized 52 byte log entry
0709 13:12:34.756649 (+174266us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:34.821435 (+ 64786us) log.cc:844] Serialized 14 byte log entry
0709 13:12:34.821697 (+   262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.821812 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:35.387647 (+565835us) raft_consensus.cc:1685] finished
0709 13:12:35.387667 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.387818 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34263,"prepare.queue_time_us":65049,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":38742,"raft.queue_time_us":46912,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":245,"spinlock_wait_cycles":13923200,"thread_start_us":173651,"threads_started":1,"wal-append.queue_time_us":676162}
I20260709 13:12:35.394387 25756 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:35.395099 25756 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:35.395294 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2112) took 2814 ms (client timeout 2402 ms). Trace:
W20260709 13:12:35.395784 22759 rpcz_store.cc:269] 0709 13:12:32.580404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.479156 (+1898752us) service_pool.cc:224] Handling call
0709 13:12:35.395040 (+915884us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400002dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:35.395216 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698282368}
I20260709 13:12:35.400789 25687 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.397058 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.404979 22765 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:35.402587 22759 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:35.406965 25564 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.407758 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.408542 22759 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:35.385007 22907 rpcz_store.cc:276] 0709 13:12:33.444485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.488968 (+ 44483us) service_pool.cc:224] Handling call
0709 13:12:33.489645 (+   677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.713321 (+1223676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.384166 (+670845us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001d0e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:35.384440 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812328320}
I20260709 13:12:35.409927 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:35.411293 25635 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:35.411250 22764 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:35.412148 25635 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:35.413470 25635 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:35.418141 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 100793us
I20260709 13:12:35.379459 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.419505 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.420312 22759 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.424136 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.425863 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:35.425547 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.426640 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.432297 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50945us
W20260709 13:12:35.439953 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 139161us
I20260709 13:12:35.440419 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2134) took 1107 ms. Trace:
W20260709 13:12:35.440802 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.566s (SENT)
I20260709 13:12:35.397030 25756 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.446233 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.447088 22765 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:35.449249 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:35.449538 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:35.450258 22838 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:35.440825 22757 rpcz_store.cc:276] 0709 13:12:34.332966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.146102 (+813136us) service_pool.cc:224] Handling call
0709 13:12:35.440191 (+294089us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:35.440352 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433700096}
I20260709 13:12:35.434844 22836 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:35.480126 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.484385 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.486274 22761 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:35.489866 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 122075us
I20260709 13:12:35.490319 22757 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:35.490476 22758 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:35.493784 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.490302 25662 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:35.488565 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.498623 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.497336 23462 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:35.498296 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:35.501245 25814 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:35.501593 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.502087 25814 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:35.503342 25687 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:35.503919 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.505088 25707 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:35.505702 25586 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:35.501111 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60366.6us
I20260709 13:12:35.506378 22909 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
W20260709 13:12:35.506812 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.593s (SENT)
I20260709 13:12:35.507272 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.507844 25636 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.508529 25636 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:35.508818 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:35.509253 25815 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:35.402369 25772 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.509843 25707 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:35.510444 25707 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.511029 25707 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.503154 25814 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.468066 25790 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.514551 23459 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:35.514194 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.471542 25573 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:35.514973 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.517969 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46451.8us
I20260709 13:12:35.520977 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.498930 25662 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.525789 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.524072 22758 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:35.529059 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.529878 22758 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:35.530262 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2143) took 1168 ms. Trace:
I20260709 13:12:35.530575 22758 rpcz_store.cc:276] 0709 13:12:34.438837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.528824 (+1089987us) service_pool.cc:224] Handling call
0709 13:12:35.530211 (+  1387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.523113 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.534487 22757 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:35.471237 25667 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:12:35.535323 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.537855 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:35.518632 25573 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.546651 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:35.547120 23457 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:35.548699 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:35.549335 23463 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.559746 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:35.560848 23459 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:35.525445 25758 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:35.560909 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.561692 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:35.563967 22758 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:35.534945 25790 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:35.565826 22757 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:35.535234 25662 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.567816 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:35.568121 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 134968us
I20260709 13:12:35.569020 22758 raft_consensus.cc:2410] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:35.570597 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2142) took 1209 ms. Trace:
I20260709 13:12:35.546119 25586 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:35.571151 22758 rpcz_store.cc:276] 0709 13:12:34.361138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.567521 (+1206383us) service_pool.cc:224] Handling call
0709 13:12:35.570531 (+  3010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.570192 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:35.561353 25573 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:35.582312 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47089.2us
W20260709 13:12:35.576676 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68119.2us
W20260709 13:12:35.579306 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78341us
I20260709 13:12:35.564508 25756 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:35.590143 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:35.593003 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.567536 25790 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:35.598344 22759 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:35.596172 22761 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:35.604784 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.635s (SENT)
I20260709 13:12:35.585063 25776 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.608677 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.610785 22761 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:35.614224 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2101) took 3034 ms. Trace:
I20260709 13:12:35.614895 22761 rpcz_store.cc:276] 0709 13:12:32.577076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.600951 (+3023875us) service_pool.cc:224] Handling call
0709 13:12:35.608910 (+  7959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.611317 (+  2407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.611704 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.614259 22759 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:12:35.616832 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2097) took 3099 ms. Trace:
I20260709 13:12:35.620291 22758 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:12:35.617286 22759 rpcz_store.cc:276] 0709 13:12:32.517863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.603214 (+3085351us) service_pool.cc:224] Handling call
0709 13:12:35.614018 (+ 10804us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.616448 (+  2430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.616707 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.608222 25795 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:35.607755 25586 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:35.632653 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2096) took 3108 ms. Trace:
I20260709 13:12:35.633324 25815 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:35.635918 22758 rpcz_store.cc:276] 0709 13:12:32.517383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.582226 (+3064843us) service_pool.cc:224] Handling call
0709 13:12:35.606179 (+ 23953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.624982 (+ 18803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.625349 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101504}
W20260709 13:12:35.641579 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 51361.1us
W20260709 13:12:35.647028 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.647739 25824 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:12:35.648516 22759 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:35.649282 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2103) took 3071 ms. Trace:
I20260709 13:12:35.649652 22759 rpcz_store.cc:276] 0709 13:12:32.577807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.621600 (+3043793us) service_pool.cc:224] Handling call
0709 13:12:35.648282 (+ 26682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.648974 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.649217 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.651197 22834 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:35.653012 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:35.654405 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54462.5us
I20260709 13:12:35.655391 22758 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:35.632134 25795 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:35.658351 22761 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:35.656313 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2107) took 3077 ms. Trace:
I20260709 13:12:35.659185 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2105) took 3080 ms. Trace:
I20260709 13:12:35.659216 22758 rpcz_store.cc:276] 0709 13:12:32.579002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.638188 (+3059186us) service_pool.cc:224] Handling call
0709 13:12:35.655126 (+ 16938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.655982 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.656238 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.659654 22761 rpcz_store.cc:276] 0709 13:12:32.578448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.636946 (+3058498us) service_pool.cc:224] Handling call
0709 13:12:35.658096 (+ 21150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.658945 (+   849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.659131 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.661406 22761 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:35.617838 25826 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:12:35.662202 22758 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:35.665828 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2130) took 1333 ms. Trace:
I20260709 13:12:35.667358 22758 rpcz_store.cc:276] 0709 13:12:34.330572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.660712 (+1330140us) service_pool.cc:224] Handling call
0709 13:12:35.661352 (+   640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.663075 (+  1723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.663644 (+   569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108288}
I20260709 13:12:35.664472 25817 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:35.669586 25817 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:35.670142 25817 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.674098 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1380) took 1033 ms. Trace:
I20260709 13:12:35.674446 22759 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:35.633736 25790 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.677887 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2109) took 3098 ms. Trace:
I20260709 13:12:35.678293 22759 rpcz_store.cc:276] 0709 13:12:32.579575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.650506 (+3070931us) service_pool.cc:224] Handling call
0709 13:12:35.674196 (+ 23690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.676629 (+  2433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.677823 (+  1194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.676951 22761 rpcz_store.cc:276] 0709 13:12:34.632869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.660811 (+1027942us) service_pool.cc:224] Handling call
0709 13:12:35.661164 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.661824 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.666085 (+  4261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.681274 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.665108 25795 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.658322 25776 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:35.686862 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.691109 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.692198 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:35.693759 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 90846.1us
I20260709 13:12:35.700049 25662 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.712688 22758 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:35.716675 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.718151 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 332) took 1067 ms. Trace:
I20260709 13:12:35.718619 23465 rpcz_store.cc:276] 0709 13:12:34.650612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.737200 (+ 86588us) service_pool.cc:224] Handling call
0709 13:12:35.685801 (+948601us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:35.718056 (+ 32255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28184320}
I20260709 13:12:35.723333 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:35.679910 25589 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:35.676043 25824 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:35.727641 25687 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:35.724928 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.731814 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.732707 22909 raft_consensus.cc:2410] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:35.734787 25662 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:35.740567 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.719372 25756 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:35.724295 25758 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:35.724222 25790 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.725127 25635 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:35.744640 25758 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:35.747923 25758 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.747999 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.748965 22906 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:35.750197 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:35.750929 22914 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.751353 25828 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.751964 25828 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.753168 22758 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:12:35.756639 25818 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:12:35.757750 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.757014 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.719576 25776 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.760593 22907 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:35.759130 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:35.765594 22915 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.737521 25687 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:35.768468 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.769152 22909 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:35.775681 22914 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
W20260709 13:12:35.770210 22806 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.729s (SENT)
I20260709 13:12:35.778878 22906 raft_consensus.cc:1217] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-8.8]   Dedup: 3.6->[6.7-8.8]
W20260709 13:12:35.759130 25821 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:12:35.746577 25635 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:35.780299 22906 pending_rounds.cc:85] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:12:35.782550 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.757151 25748 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:35.783170 22906 pending_rounds.cc:107] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:12:35.777344 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:35.783499 22906 raft_consensus.cc:2889] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:35.780987 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.785585 25687 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.787900 25758 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.742882 25824 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:35.794483 25259 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.000s
I20260709 13:12:35.795220 25259 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.784448 25635 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.798538 22911 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:35.799504 23460 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:35.800477 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.800621 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 331) took 1150 ms. Trace:
W20260709 13:12:35.802132 25822 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:12:35.806211 25259 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.807003 25828 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.819123 22909 raft_consensus.cc:2376] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8. Re-sending same reply.
W20260709 13:12:35.819989 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55448.4us
I20260709 13:12:35.820318 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:35.806386 22879 leader_election.cc:336] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.806s (SENT)
I20260709 13:12:35.815878 22911 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:12:35.861050 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.801098 23460 rpcz_store.cc:276] 0709 13:12:34.650212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.654075 (+  3863us) service_pool.cc:224] Handling call
0709 13:12:35.798843 (+1144768us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:35.799122 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.800298 (+  1176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.800540 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126258944}
W20260709 13:12:35.855592 22806 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.800s (SENT)
I20260709 13:12:35.814558 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:35.864053 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48664.5us
I20260709 13:12:35.865643 22907 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.765026 25589 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.868966 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.870812 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.877084 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2095) took 3360 ms. Trace:
I20260709 13:12:35.877601 22757 rpcz_store.cc:276] 0709 13:12:32.517077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.570411 (+3053334us) service_pool.cc:224] Handling call
0709 13:12:35.606104 (+ 35693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.610196 (+  4092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.610246 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.617960 (+  7714us) log.cc:844] Serialized 26 byte log entry
0709 13:12:35.701753 (+ 83793us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:35.724146 (+ 22393us) log.cc:844] Serialized 14 byte log entry
0709 13:12:35.782493 (+ 58347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.782573 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:35.876790 (+ 94217us) raft_consensus.cc:1685] finished
0709 13:12:35.876815 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.876977 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":908,"prepare.queue_time_us":22132,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":104482,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":2168,"spinlock_wait_cycles":14080,"thread_start_us":136433,"threads_started":2,"wal-append.queue_time_us":141866}
W20260709 13:12:35.883926 25807 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.625s	user 0.000s	sys 0.004s
I20260709 13:12:35.883737 25635 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:35.885789 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.887535 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:35.889823 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:35.890823 25767 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:35.892086 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1389) took 1071 ms. Trace:
I20260709 13:12:35.892823 25767 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:35.893682 25767 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:35.894169 25767 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.895088 25758 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:35.896548 25789 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.897181 25789 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.897703 25789 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.900954 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 299) took 4034 ms. Trace:
I20260709 13:12:35.866664 25828 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.901625 25828 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:35.902190 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2089) took 3422 ms (client timeout 2143 ms). Trace:
W20260709 13:12:35.902585 22766 rpcz_store.cc:269] 0709 13:12:32.479943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.480088 (+   145us) service_pool.cc:224] Handling call
0709 13:12:33.483235 (+1003147us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:35.901965 (+2418730us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:35.902099 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581409408}
I20260709 13:12:35.903167 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:35.903919 25828 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.904403 22766 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:12:35.905889 25731 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.007s
I20260709 13:12:35.906325 25731 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.880792 25573 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.909646 25776 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.910092 25573 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:35.910310 25776 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:35.914556 25707 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:35.916553 22909 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:35.906688 22765 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:35.918448 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1395) took 1007 ms. Trace:
I20260709 13:12:35.918834 22765 rpcz_store.cc:276] 0709 13:12:34.910998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.906134 (+995136us) service_pool.cc:224] Handling call
0709 13:12:35.906506 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.918100 (+ 11594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.918383 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.921075 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:35.921473 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.922793 25697 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:35.924027 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.892498 22758 rpcz_store.cc:276] 0709 13:12:34.820549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.727617 (+907068us) service_pool.cc:224] Handling call
0709 13:12:35.727929 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.753482 (+ 25553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.891787 (+138305us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b540037f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:35.891993 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57236864}
I20260709 13:12:35.865044 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:35.890597 25575 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.901387 22908 rpcz_store.cc:276] 0709 13:12:31.866275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.938128 (+ 71853us) service_pool.cc:224] Handling call
0709 13:12:34.382714 (+2444586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.760370 (+377656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.900684 (+1140314us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b54000ea950. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:35.900861 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5620890880}
I20260709 13:12:35.931234 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.931957 22915 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:35.867069 25792 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.015s
I20260709 13:12:35.900390 25789 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:35.938251 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:35.938961 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1396) took 1027 ms. Trace:
I20260709 13:12:35.940068 22765 rpcz_store.cc:276] 0709 13:12:34.911303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.919801 (+1008498us) service_pool.cc:224] Handling call
0709 13:12:35.924526 (+  4725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.938700 (+ 14174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.938721 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.938760 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:35.938785 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.938829 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.938913 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:35.942437 22911 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:35.940183 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:35.869534 25815 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:35.904524 25828 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:35.911231 25731 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:35.947811 22909 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:35.960707 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:35.961593 23464 raft_consensus.cc:2368] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
I20260709 13:12:35.962040 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 337) took 1309 ms. Trace:
I20260709 13:12:35.962359 23464 rpcz_store.cc:276] 0709 13:12:34.652796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.730486 (+ 77690us) service_pool.cc:224] Handling call
0709 13:12:35.960522 (+1230036us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:35.961953 (+  1431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44102016}
I20260709 13:12:35.964920 25575 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:35.954443 25819 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.315s	user 0.000s	sys 0.000s
I20260709 13:12:35.887521 25635 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:35.921840 25772 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:12:35.965440 25795 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.966327 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2225) took 1548 ms. Trace:
I20260709 13:12:35.972481 25707 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:35.973138 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45639.1us
I20260709 13:12:35.984663 22910 rpcz_store.cc:276] 0709 13:12:34.418137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.418607 (+   470us) service_pool.cc:224] Handling call
0709 13:12:34.419096 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.712647 (+293551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.966009 (+1253362us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54001d1d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:35.966255 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672903424}
I20260709 13:12:35.868603 25817 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:35.977595 22766 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:12:35.986353 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 165239us
I20260709 13:12:36.006060 22759 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:36.007627 25817 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:36.010082 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:36.011252 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 536) took 1367 ms. Trace:
I20260709 13:12:36.011648 23466 rpcz_store.cc:276] 0709 13:12:34.643542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.701608 (+ 58066us) service_pool.cc:224] Handling call
0709 13:12:36.009853 (+1308245us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:36.011166 (+  1313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110560128}
I20260709 13:12:36.019310 25820 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:36.020045 25820 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:36.021070 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 214731us
I20260709 13:12:36.022518 25820 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.038686 22908 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:36.061735 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59469.9us
I20260709 13:12:36.065460 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.068629 22765 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:36.067759 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:36.076176 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:36.076424 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:36.077148 23464 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:36.080693 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:12:36.082854 22806 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.870s (SENT)
I20260709 13:12:36.083535 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:36.075171 25819 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:12:36.073984 25817 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.091225 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1388) took 1365 ms. Trace:
I20260709 13:12:36.075006 25635 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:36.088372 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 223100us
I20260709 13:12:36.091641 22761 rpcz_store.cc:276] 0709 13:12:34.725486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.680082 (+954596us) service_pool.cc:224] Handling call
0709 13:12:35.703165 (+ 23083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.712655 (+  9490us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:35.927326 (+214671us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.035728 (+108402us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b180035e4e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:36.090912 (+ 55184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:36.090977 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:36.091014 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.091064 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.091177 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":449,"prepare.queue_time_us":30828,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":106464,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":6844288,"thread_start_us":297350,"threads_started":2}
I20260709 13:12:36.103304 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:36.066628 25767 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.087127 25575 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:36.088462 25748 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:36.088815 25772 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:36.089571 25820 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.113435 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:36.094363 25697 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:36.119822 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.120754 22761 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:36.119828 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44849.2us
W20260709 13:12:36.122766 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.911s (SENT)
I20260709 13:12:36.123230 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.124521 25795 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:36.124738 25815 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.101286 25817 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.126914 25817 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:36.124569 22806 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.983s (SENT)
I20260709 13:12:36.128684 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.131085 25815 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.133345 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:36.133677 23432 leader_election.cc:400] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
W20260709 13:12:36.134050 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1357) took 5614 ms (client timeout 4707 ms). Trace:
I20260709 13:12:36.134635 22763 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 13:12:36.135031 22763 pending_rounds.cc:85] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 1
I20260709 13:12:36.134876 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.135164 25772 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:36.135454 22763 pending_rounds.cc:107] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:12:36.135726 25772 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.134466 22807 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:36.136286 25772 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:36.136349 25697 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.137683 25813 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:36.138288 25813 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:36.139623 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:36.138907 22806 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.996s (SENT)
W20260709 13:12:36.140045 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.140209 25778 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:36.139194 25813 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.147264 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:36.147884 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:36.148442 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:36.148741 23460 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
W20260709 13:12:36.144774 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.027s (SENT)
I20260709 13:12:36.150055 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:36.150228 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.152217 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:36.152598 25434 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:36.153949 25824 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:36.088461 25687 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:12:36.155336 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.034s (SENT)
I20260709 13:12:36.129920 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:36.135757 22763 raft_consensus.cc:2889] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:36.160177 22763 pending_rounds.cc:107] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:12:36.160544 22763 raft_consensus.cc:2889] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:36.161821 23461 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.159723 22761 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:36.134485 22762 rpcz_store.cc:269] 0709 13:12:30.519789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.528503 (+  8714us) service_pool.cc:224] Handling call
0709 13:12:36.120513 (+5592010us) spinlock_profiling.cc:243] Waited 4.6 s on lock 0x7b5400340dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:36.133926 (+ 13413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10572923776}
I20260709 13:12:36.183368 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.111343 25575 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.184285 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.186481 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.187381 22765 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:36.192229 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.194788 25824 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.116070 25667 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:12:36.198714 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:36.201205 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1390) took 1292 ms. Trace:
I20260709 13:12:36.199230 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.206063 22761 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.205492 22757 rpcz_store.cc:276] 0709 13:12:34.909265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.879603 (+970338us) service_pool.cc:224] Handling call
0709 13:12:35.880289 (+   686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.880763 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.880794 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.882218 (+  1424us) log.cc:844] Serialized 548 byte log entry
0709 13:12:36.088201 (+205983us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:36.088266 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.088368 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:36.200985 (+112617us) raft_consensus.cc:1685] finished
0709 13:12:36.201006 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.201143 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":205506,"threads_started":1,"wal-append.queue_time_us":240058}
W20260709 13:12:36.202203 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 331507us
I20260709 13:12:36.207108 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.199505 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:36.208439 22911 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.214161 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:36.215059 22838 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.121342 25748 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:36.137723 25792 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:36.223079 22907 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
W20260709 13:12:36.206413 25844 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
W20260709 13:12:36.238015 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83038.2us
I20260709 13:12:36.239782 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.240538 25837 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:36.242759 22912 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:36.201663 25575 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.246284 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.250000 25259 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:12:36.250407 22806 leader_election.cc:336] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.056s (SENT)
I20260709 13:12:36.227620 25792 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.220474 25815 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.250945 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.251231 25792 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.251740 25815 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:12:36.252350 22806 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.064s (SENT)
I20260709 13:12:36.252794 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.250064 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.254495 22757 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:36.227247 25842 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:12:36.259443 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.183825 25434 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:12:36.269824 25847 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:36.250329 25748 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.272712 25434 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.275036 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.217775 25589 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:36.284948 22838 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:36.290068 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1403) took 1358 ms. Trace:
I20260709 13:12:36.292836 22836 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:36.298209 22761 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:36.296710 22764 rpcz_store.cc:276] 0709 13:12:34.955381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.924370 (+968989us) service_pool.cc:224] Handling call
0709 13:12:35.945223 (+ 20853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.981626 (+ 36403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.981665 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:36.051360 (+ 69695us) log.cc:844] Serialized 548 byte log entry
0709 13:12:36.222992 (+171632us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:36.223039 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.223138 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:36.289747 (+ 66609us) raft_consensus.cc:1685] finished
0709 13:12:36.289777 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.289975 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":120019,"threads_started":1,"wal-append.queue_time_us":187294}
I20260709 13:12:36.276219 25635 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:36.227649 25795 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:12:36.314373 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:36.317828 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58498.3us
I20260709 13:12:36.291939 25697 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:36.320017 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:36.320760 22838 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.319378 22757 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:36.324332 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.325093 22764 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:36.323493 22765 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:36.330634 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.334872 22757 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:36.331622 25824 raft_consensus.cc:2764] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:12:36.337906 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 111988us
I20260709 13:12:36.336737 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:36.307200 25586 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:36.314260 25635 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:36.323402 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1404) took 1021 ms. Trace:
I20260709 13:12:36.354317 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:36.324885 25828 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:36.324678 25837 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:36.359431 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.360121 22838 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:36.372596 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.375698 22765 raft_consensus.cc:2368] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
I20260709 13:12:36.375383 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:36.377816 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60136us
I20260709 13:12:36.379207 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.379976 22838 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:36.355347 25635 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.353219 22758 rpcz_store.cc:276] 0709 13:12:35.302503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.926215 (+623712us) service_pool.cc:224] Handling call
0709 13:12:35.926524 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.939623 (+ 13099us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:36.094729 (+155106us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.322815 (+228086us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.323027 (+   212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:36.323084 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:36.323159 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.323241 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.323348 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16725,"prepare.queue_time_us":36855,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":20275,"raft.queue_time_us":16627,"raft.run_wall_time_us":12131,"spinlock_wait_cycles":55560448,"thread_start_us":306057,"threads_started":2}
I20260709 13:12:36.384163 22840 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:36.360925 25828 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.379642 25586 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:36.412026 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75364.8us
I20260709 13:12:36.414608 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.414839 22833 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:12:36.415300 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:36.416272 22839 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:36.419727 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:36.419698 22764 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:36.415694 22833 pending_rounds.cc:85] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:12:36.422672 22833 pending_rounds.cc:107] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:12:36.423018 22833 raft_consensus.cc:2889] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:36.423462 22833 pending_rounds.cc:107] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:12:36.423240 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.423923 22833 raft_consensus.cc:2889] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:36.424320 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:36.432263 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:36.428133 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175770us
I20260709 13:12:36.428359 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.412774 25828 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.437623 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:36.444314 25859 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:36.445855 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.417892 25586 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.412407 25767 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:36.439010 22839 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:12:36.417126 22663 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:12:36.461634 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:36.462270 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:36.466318 25856 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:12:36.435566 25687 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:36.425354 25843 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.470712 25828 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.486444 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.446556 22836 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
W20260709 13:12:36.471437 25832 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.753s	user 0.006s	sys 0.000s
W20260709 13:12:36.512123 22806 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.068s (SENT)
I20260709 13:12:36.534874 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.491601 25586 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.534925 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:36.556519 22833 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:36.556668 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 94486.8us
I20260709 13:12:36.557687 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:36.566393 22757 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:36.566676 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.510603 25687 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:36.574687 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:36.575486 22837 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:36.578517 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:36.582940 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53102.4us
I20260709 13:12:36.583052 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:36.584012 22764 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:36.553634 25589 raft_consensus.cc:2764] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:12:36.591923 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 161925us
I20260709 13:12:36.577955 25687 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.571800 25863 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:36.602051 23461 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:36.581434 25748 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:36.603657 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46090.9us
I20260709 13:12:36.604534 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.609432 23458 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:36.614009 25868 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:36.614709 25868 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:36.615692 25868 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.619696 25828 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:36.603696 25748 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.624620 25748 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:36.602620 25863 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:36.629823 25863 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.634895 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:36.643829 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.665483 22837 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:36.665940 25313 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.670567 25872 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:36.671365 25872 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:36.676167 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1394) took 1765 ms. Trace:
I20260709 13:12:36.646656 25731 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.676688 22766 rpcz_store.cc:276] 0709 13:12:34.910704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.905338 (+994634us) service_pool.cc:224] Handling call
0709 13:12:35.925322 (+ 19984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.003600 (+ 78278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.675778 (+672178us) spinlock_profiling.cc:243] Waited 51.7 ms on lock 0x7b5400311354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:36.676035 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118962944}
I20260709 13:12:36.677734 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.682713 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.683040 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:36.683840 25731 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:36.684571 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1401) took 1771 ms (client timeout 1884 ms). Trace:
W20260709 13:12:36.685021 22760 rpcz_store.cc:269] 0709 13:12:34.913011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.312252 (+399241us) service_pool.cc:224] Handling call
0709 13:12:36.682880 (+1370628us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:36.684422 (+  1542us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4428672}
I20260709 13:12:36.687783 22911 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:36.676831 25687 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.735466 25687 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:36.767745 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 85132us
I20260709 13:12:36.649917 25589 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:12:36.737583 25790 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:36.757555 25313 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.761303 25635 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.788170 25635 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:36.672782 25872 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.792850 22908 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:36.775230 25575 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:36.794538 25772 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.001s
I20260709 13:12:36.794999 25772 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.796092 25575 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:36.797911 22832 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:12:36.799500 25792 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 13:12:36.799990 25792 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.805279 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:36.806339 22911 raft_consensus.cc:606] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.806836 22911 raft_consensus.cc:614] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:36.808135 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:36.808813 23461 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:36.813715 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 370824us
I20260709 13:12:36.815274 25792 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:36.822834 25814 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:36.824955 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:36.827239 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46729us
I20260709 13:12:36.828022 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.828873 25790 raft_consensus.cc:2764] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:12:36.831877 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.833750 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.834888 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:36.836108 22911 raft_consensus.cc:606] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.837108 22911 raft_consensus.cc:614] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:36.850193 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 245746us
I20260709 13:12:36.852025 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.852229 23461 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:36.797219 25772 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:36.854585 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.864054 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:36.864665 22833 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:36.864957 23461 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.866374 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.872035 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:36.873683 22839 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:36.874825 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.835764 25731 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:36.848420 25820 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:36.857445 25815 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.879338 22841 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:36.877063 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.852824 25814 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:36.881171 22838 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:36.906701 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:36.857685 25847 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:36.905675 25817 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:12:36.913686 25754 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:12:36.917925 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.921072 22839 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:36.926043 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49158.2us
I20260709 13:12:36.926132 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.863220 25835 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.927971 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af39123de9434cc0b451f4b03d305310"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:36.931277 22911 raft_consensus.cc:606] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.931670 22911 raft_consensus.cc:614] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:36.924088 25837 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:36.939059 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.875870 25575 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:12:36.906280 25772 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:12:36.953076 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 447004us
I20260709 13:12:36.863829 25790 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:12:36.898366 25868 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:12:36.915304 25758 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:12:36.933138 25707 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:36.977492 22841 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:12:36.973131 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:36.978016 22841 pending_rounds.cc:85] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:12:36.978461 22911 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:36.979460 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:36.980527 22836 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:36.973981 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:36.979790 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53781.7us
I20260709 13:12:36.929863 25767 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:12:36.974925 25586 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:36.916759 25815 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.974963 22913 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:36.992419 22911 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:36.976991 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:36.916138 25820 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:36.980967 23458 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:36.981547 22906 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.975335 25667 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:37.013823 25667 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.016572 25697 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:37.016847 25667 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:37.017130 25697 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:37.018220 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1387) took 2293 ms. Trace:
I20260709 13:12:37.018565 25697 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.004269 25815 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:36.988096 22841 pending_rounds.cc:107] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:12:37.020820 22841 raft_consensus.cc:2889] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:36.933615 25814 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:36.962823 25790 consensus_queue.cc:1243] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:36.961736 25817 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:36.992063 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.091s (SENT)
I20260709 13:12:37.041869 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:36.993978 22907 raft_consensus.cc:606] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.003101 25434 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:36.931140 25847 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:37.004171 25852 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.586s	user 0.009s	sys 0.001s
I20260709 13:12:37.005394 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2092) took 4489 ms. Trace:
I20260709 13:12:37.007220 25820 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:36.986764 25731 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:12:37.070000 25815 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:37.070012 22763 rpcz_store.cc:276] 0709 13:12:32.516072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.516395 (+   323us) service_pool.cc:224] Handling call
0709 13:12:36.134058 (+3617663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.207610 (+ 73552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.005030 (+797420us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b5400340dd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:37.005298 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7672506112}
I20260709 13:12:37.071578 25815 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:37.072242 25815 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:37.072324 25872 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:37.072846 25872 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.064399 25434 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:37.073987 25872 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.074584 25434 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.076284 25434 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.012352 25586 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:37.018646 22759 rpcz_store.cc:276] 0709 13:12:34.724652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.679112 (+954460us) service_pool.cc:224] Handling call
0709 13:12:35.706334 (+ 27222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.006662 (+300328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.017942 (+1011280us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540024b154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:37.018137 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688718080}
W20260709 13:12:37.078394 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89976.6us
W20260709 13:12:37.079344 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.119s (SENT)
I20260709 13:12:37.079715 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.080729 25537 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:37.019780 25697 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.083065 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:37.085428 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:36.996974 25863 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.085791 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.086274 22759 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:37.087746 25863 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:37.073369 25847 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:37.064648 25874 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.058s	user 0.000s	sys 0.006s
I20260709 13:12:36.982256 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:37.094105 25790 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:37.094278 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 112138us
I20260709 13:12:37.095799 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:37.097084 25790 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.099529 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:37.061581 25814 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.056833 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:37.081233 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.134s (SENT)
I20260709 13:12:37.063012 22907 raft_consensus.cc:614] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:37.041252 25667 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:37.077674 25259 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index 3 to 6, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:37.125317 22906 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:37.060529 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.101337 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:36.978813 23461 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:37.056030 25877 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.124s	user 0.003s	sys 0.003s
I20260709 13:12:37.105281 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.058202 25748 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.102751 25859 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:37.132699 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:37.126621 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 205850us
I20260709 13:12:37.115362 25790 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:37.151306 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.145887 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.073406 25815 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.156603 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:37.157567 25868 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:37.157902 25815 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.159281 25868 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:37.159862 25868 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.160822 25868 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.166747 22906 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:37.135602 25863 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.129277 25778 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:37.153108 25687 raft_consensus.cc:2764] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:12:37.127928 25667 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.155906 25575 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:37.172227 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.191514 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:37.179070 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.182492 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:37.171020 25761 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:37.200317 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119103us
I20260709 13:12:37.203308 22764 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:37.214541 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.216403 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.224748 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.227840 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.230010 25758 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:37.230754 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.231726 22764 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:37.233469 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 359) took 1070 ms. Trace:
I20260709 13:12:37.223043 25880 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:37.208591 25863 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:37.236913 22806 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.143s (SENT)
I20260709 13:12:37.237473 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.190587 25778 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.233982 23460 rpcz_store.cc:276] 0709 13:12:36.163335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.174830 (+ 11495us) service_pool.cc:224] Handling call
0709 13:12:36.199877 (+ 25047us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:36.216768 (+ 16891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:36.216820 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:36.250647 (+ 33827us) log.cc:844] Serialized 52 byte log entry
0709 13:12:36.802056 (+551409us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b1800123760. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f68a59c558f 00007f689ea04571 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd
0709 13:12:37.104403 (+302347us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b1800123760. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:12:37.194671 (+ 90268us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.198764 (+  4093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.204739 (+  5975us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.233204 (+ 28465us) raft_consensus.cc:1685] finished
0709 13:12:37.233229 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.233396 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18111,"raft.queue_time_us":24909,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":14843,"spinlock_wait_cycles":14272640,"thread_start_us":78135,"threads_started":1,"wal-append.queue_time_us":980162}
I20260709 13:12:37.243199 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.246760 22833 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.244611 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.251091 22764 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:37.239207 25847 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:37.239907 25635 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:37.243507 25883 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:37.290817 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.291386 25748 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:37.292922 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.293332 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.293689 22759 raft_consensus.cc:2368] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6.
I20260709 13:12:37.294492 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:37.273020 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43780.2us
I20260709 13:12:37.296370 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:37.275202 25635 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.284648 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.286134 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:37.287606 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.289546 22757 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:37.290268 25667 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:37.273991 25758 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.298799 22906 raft_consensus.cc:606] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:37.246269 25863 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.299304 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.302452 22763 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:37.302587 22841 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:37.320192 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.300679 22839 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6. Re-sending same reply.
I20260709 13:12:37.321143 22762 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:37.320387 22837 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 4.
I20260709 13:12:37.330521 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.332496 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.333179 22759 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:37.335940 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:37.336493 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 110029us
I20260709 13:12:37.339126 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.294912 25778 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:37.345945 23466 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:12:37.298954 25313 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.319902 25575 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:37.356320 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.350673 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.350987 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.347092 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:37.353762 23464 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 13:12:37.350324 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.375826 22837 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.366783 22833 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:37.368904 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48466.4us
I20260709 13:12:37.348356 25767 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.368476 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:37.375523 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:37.347669 25868 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:12:37.381812 22762 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:37.326284 25892 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:12:37.383797 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:37.386355 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:37.393805 22759 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:37.394198 22759 pending_rounds.cc:85] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 3
I20260709 13:12:37.394407 22759 pending_rounds.cc:107] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:37.394614 22759 raft_consensus.cc:2889] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:37.390812 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.396358 22762 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:37.398168 25635 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:37.400786 25882 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:12:37.361259 25575 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.401929 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.402163 25575 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.404608 22760 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:37.405660 22833 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:37.405737 25863 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.407805 25872 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:37.391467 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.366495 25748 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:37.414467 25748 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.416951 25748 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:37.383953 25875 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:12:37.420866 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:37.384405 25789 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:37.421854 25789 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.385273 25817 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:12:37.422147 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 384) took 1195 ms. Trace:
I20260709 13:12:37.422350 25789 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.422565 22832 rpcz_store.cc:276] 0709 13:12:36.226781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.226955 (+   174us) service_pool.cc:224] Handling call
0709 13:12:36.253203 (+ 26248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:36.798355 (+545152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.421826 (+623471us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:37.422053 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252089216}
I20260709 13:12:37.424300 22832 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:37.366791 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:37.360883 25313 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.428551 22834 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:12:37.431192 25881 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:12:37.432444 25758 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.431346 22832 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 3 to 6, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:37.434613 25824 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:37.435042 25824 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:37.439425 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48057.4us
I20260709 13:12:37.440548 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.441773 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:37.442090 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.445679 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47693.8us
I20260709 13:12:37.446465 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:37.447873 22765 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:12:37.454576 22762 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:37.457149 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:37.458114 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.458794 22765 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.462466 22834 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:37.428500 25313 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:37.467410 22760 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.466732 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.474859 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.475942 22766 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:37.466482 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:37.476985 22764 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 5.
I20260709 13:12:37.484678 22908 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:37.487037 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.488276 25313 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:37.488549 22912 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:37.488722 25313 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:37.489166 25313 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.489827 25890 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.490053 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.490309 22909 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:12:37.490726 22757 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.493777 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.494184 22915 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:37.494364 22764 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:37.494371 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.495193 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:37.495770 22915 raft_consensus.cc:2376] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7. Re-sending same reply.
I20260709 13:12:37.497063 22761 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:37.437705 25837 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:37.497691 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:37.499042 22911 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:37.421885 25875 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:37.494975 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:37.513568 22764 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:37.462390 25434 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:37.461998 25790 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:37.530576 22757 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:37.515529 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:37.531608 22761 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:37.519697 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:37.532924 22765 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:37.534749 22765 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:37.474478 25748 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:37.529006 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 172012us
I20260709 13:12:37.540887 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:37.547086 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 100712us
I20260709 13:12:37.528388 22760 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:37.558303 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:37.551941 22766 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:37.561802 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:37.562531 22915 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:37.568298 22761 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:37.473544 25754 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:12:37.467743 25589 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:12:37.514462 25875 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.540403 25767 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:37.595074 22757 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:37.576407 22761 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:37.482920 25815 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:12:37.564742 22764 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 3 to 6, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:37.490418 25890 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.530809 25790 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:37.512853 25894 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:12:37.608649 25790 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:37.608923 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111548us
I20260709 13:12:37.583531 25748 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:37.611083 25855 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.261s	user 0.004s	sys 0.000s
I20260709 13:12:37.611466 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.612028 25748 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:37.613183 25868 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:37.554818 25434 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:37.587610 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 100511us
I20260709 13:12:37.583148 25817 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:37.613854 25891 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:37.614234 25748 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:37.613215 25893 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:12:37.614619 25872 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:37.616799 22907 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:37.630564 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.643596 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:37.669888 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57127.4us
I20260709 13:12:37.589897 25895 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:37.677485 25895 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:37.678040 25895 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:37.678421 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 90560.1us
I20260709 13:12:37.663384 25767 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:37.596360 25816 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:12:37.604507 25313 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.681609 25868 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:37.682224 25868 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.682590 25313 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:37.683408 25868 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.685346 25890 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:37.617146 25902 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:12:37.686342 25434 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.664680 25860 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:37.680994 25767 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.689922 25860 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:37.690217 25901 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:12:37.691088 25860 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:37.694280 25888 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.280s	user 0.003s	sys 0.000s
I20260709 13:12:37.697957 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.636795 25789 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.662566 25748 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.704015 25904 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:37.704398 25748 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:37.704828 25904 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:37.705927 25904 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.667524 25892 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:37.708353 25892 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:37.709384 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78953.3us
I20260709 13:12:37.709311 25892 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.710899 25892 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.667335 25815 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:12:37.693399 25860 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:37.674911 25817 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:37.723457 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:37.725986 22763 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:37.727963 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:37.722047 25894 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:37.754801 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.755951 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.756515 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.756948 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:37.758644 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44730us
I20260709 13:12:37.730332 25815 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:37.722760 25837 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:12:37.771256 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:37.781374 25911 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:37.787611 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 337583us
I20260709 13:12:37.767818 25894 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.770665 25313 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:37.720405 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.779893 25837 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:37.722283 25875 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.802279 25313 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:37.846635 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 86051.7us
I20260709 13:12:37.875425 22805 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:37.877178 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:37.868203 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 106906us
W20260709 13:12:37.879565 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 151680us
I20260709 13:12:37.883710 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:37.866792 25899 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:12:37.805195 25872 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:37.902058 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:37.856617 25875 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:37.850457 25537 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.835175 25837 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:37.867885 25927 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:37.804011 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83611us
W20260709 13:12:37.909842 25907 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:12:37.909940 25872 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.913879 25925 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:37.914736 25313 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:37.847357 25817 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.932423 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.900683 25789 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:37.952250 25906 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:12:37.953217 25575 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:37.953415 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:37.953923 25575 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:37.964169 22761 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:37.929638 25927 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:37.969934 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.928534 25537 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:37.973806 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 90218.2us
W20260709 13:12:37.962626 25899 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:12:37.955082 25575 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.966692 25259 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:37.982748 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:37.931315 25758 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:12:37.964047 25925 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:37.950312 25890 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:12:37.991369 22757 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:37.972332 25927 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.974077 25789 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.963637 25586 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:12:37.908689 25894 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.029808 25789 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.008029 25259 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:38.038029 25912 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:12:38.039074 25789 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:38.040320 25914 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:38.040970 25914 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:38.041813 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47792.1us
I20260709 13:12:37.999081 25925 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:38.054687 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.068722 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:38.069037 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.038376 25259 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.051332 25925 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:38.042086 25748 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.049625 25767 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:12:38.014978 25872 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:37.989006 25837 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:38.068534 25895 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.132020 22834 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:12:38.120906 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52268.9us
I20260709 13:12:38.245483 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:38.246644 25434 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:38.247298 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:38.249787 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 290544us
I20260709 13:12:38.081264 25697 raft_consensus.cc:2764] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:12:38.222733 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 151645us
W20260709 13:12:38.132954 25790 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
W20260709 13:12:38.150123 25910 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
W20260709 13:12:38.150246 25908 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
W20260709 13:12:38.311568 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.149338 25936 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:38.326675 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 446191us
W20260709 13:12:38.294013 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.342581 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.342183 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:38.343240 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.242344 25921 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
W20260709 13:12:38.339999 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.352473 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:38.336849 25434 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:38.371840 25930 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.287s	user 0.000s	sys 0.002s
W20260709 13:12:38.372510 25924 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
W20260709 13:12:38.378330 25900 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.155s	user 0.003s	sys 0.006s
I20260709 13:12:38.337913 25790 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.348124 25883 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:38.294355 25589 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:12:38.347844 25537 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:38.399010 25790 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:38.399757 25790 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:38.402380 25934 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.108s	user 0.000s	sys 0.002s
I20260709 13:12:38.392800 25789 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:12:38.403299 25789 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:38.403692 25789 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:38.404934 25895 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:38.406340 25434 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.406888 25434 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:38.377306 25905 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:38.360291 25635 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:38.410341 25635 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.410426 25905 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:38.412763 25635 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:38.412928 25905 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:38.371948 25939 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:38.414031 25939 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.395063 25950 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:38.376535 25923 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:38.404726 25789 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:38.435173 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 101133us
W20260709 13:12:38.417109 25860 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.002s
I20260709 13:12:38.442831 25923 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:38.293351 25914 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:38.453341 25923 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:38.500365 22834 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:38.568194 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:38.552040 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 224115us
I20260709 13:12:38.501655 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:38.577934 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:38.581682 22766 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:38.505617 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:38.568706 23464 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:38.532651 23465 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:12:38.583868 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.582507 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:38.584712 22837 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.420320 25790 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.573588 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:38.586174 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:38.473482 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1438) took 1016 ms. Trace:
I20260709 13:12:38.587635 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:38.587997 22759 rpcz_store.cc:276] 0709 13:12:37.456840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.468777 (+ 11937us) service_pool.cc:224] Handling call
0709 13:12:37.469277 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:37.508397 (+ 39120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.508446 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:37.520609 (+ 12163us) log.cc:844] Serialized 26 byte log entry
0709 13:12:37.702837 (+182228us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:37.702904 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.703019 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:38.473098 (+770079us) raft_consensus.cc:1685] finished
0709 13:12:38.473139 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:38.473378 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27820,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":18056,"thread_start_us":129887,"threads_started":1,"wal-append.queue_time_us":151343}
I20260709 13:12:38.593765 25956 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.598340 25564 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:38.598796 25748 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:38.598920 25564 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:38.599200 23460 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:38.599565 25564 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:38.600477 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:38.571216 23458 raft_consensus.cc:2955] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 3 to 6, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:38.602525 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:38.605742 25847 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:38.605913 25916 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.586606 25754 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:38.606585 25847 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:38.499352 25824 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.609174 25824 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.609762 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:38.610651 22838 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:38.611471 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:38.357551 25936 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:38.614188 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.498677 25940 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:12:38.621742 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 395) took 1629 ms. Trace:
W20260709 13:12:38.623467 22879 connection.cc:256] RPC call timeout handler was delayed by -1.0134445219999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 13:12:38.608211 25564 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:38.477638 25933 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 13:12:38.623576 22835 rpcz_store.cc:276] 0709 13:12:36.992368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.003820 (+ 11452us) service_pool.cc:224] Handling call
0709 13:12:37.004223 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:37.004716 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.004755 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:37.415451 (+410696us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b18003be7e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a
0709 13:12:37.416573 (+  1122us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:37.417486 (+   913us) log.cc:844] Serialized 26 byte log entry
0709 13:12:37.662579 (+245093us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.662618 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.662684 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:38.621404 (+958720us) raft_consensus.cc:1685] finished
0709 13:12:38.621427 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:38.621669 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4388,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":18287744,"thread_start_us":195321,"threads_started":1,"wal-append.queue_time_us":292508}
I20260709 13:12:38.569921 25939 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:38.551005 25905 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:38.629586 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:38.533025 25880 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:38.573113 25789 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:38.645306 22841 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:38.646938 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:38.645777 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:38.645762 22906 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:38.626546 25921 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:12:38.635422 22835 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:38.650611 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:38.652422 22913 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:38.654820 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:38.650619 25905 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:38.662403 22910 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:12:38.682403 22837 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:38.633419 25313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.010s	sys 0.024s
I20260709 13:12:38.687191 22838 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:12:38.701292 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.706097 22663 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:12:38.709432 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.714855 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.599s (SENT)
I20260709 13:12:38.715446 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:38.716130 25897 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:38.716550 25897 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:38.717689 25897 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:38.721078 25550 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:38.726459 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2251) took 1270 ms. Trace:
I20260709 13:12:38.722373 25897 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:38.730594 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.727020 22907 rpcz_store.cc:276] 0709 13:12:37.456271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.467514 (+ 11243us) service_pool.cc:224] Handling call
0709 13:12:37.506586 (+ 39072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.663233 (+156647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.726142 (+1062909us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000fb754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:38.726381 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252521728}
W20260709 13:12:38.747272 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.632s (SENT)
I20260709 13:12:38.748112 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:38.750236 25550 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:12:38.660656 25875 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.003s
I20260709 13:12:38.690280 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:38.622962 25790 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:38.721690 22663 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 3 to 4, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 4 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:39.171624 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.030884 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.009s	sys 0.005s
W20260709 13:12:39.269558 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.542514 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 795277us
I20260709 13:12:39.079449 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1446) took 1575 ms. Trace:
I20260709 13:12:38.972350 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:39.554219 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:39.555455 22760 rpcz_store.cc:276] 0709 13:12:37.503620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.557716 (+ 54096us) service_pool.cc:224] Handling call
0709 13:12:37.589670 (+ 31954us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:37.624543 (+ 34873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.624604 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:37.659664 (+ 35060us) log.cc:844] Serialized 52 byte log entry
0709 13:12:37.769839 (+110175us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:37.834687 (+ 64848us) log.cc:844] Serialized 14 byte log entry
0709 13:12:38.310990 (+476303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.311075 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:39.079104 (+768029us) raft_consensus.cc:1685] finished
0709 13:12:39.079137 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.079360 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43174,"prepare.queue_time_us":130652,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":141196,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":8944,"spinlock_wait_cycles":22864128,"thread_start_us":574470,"threads_started":3,"wal-append.queue_time_us":183596}
W20260709 13:12:39.553622 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.014s	sys 0.009s
W20260709 13:12:39.545045 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 922316us
W20260709 13:12:39.558754 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 1.20661e+06us
W20260709 13:12:39.560554 22835 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
I20260709 13:12:39.551859 22766 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:38.646380 25761 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:39.546491 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.571537 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:38.693327 25313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.010s	sys 0.025s
W20260709 13:12:38.720933 25948 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.268s	user 0.004s	sys 0.001s
I20260709 13:12:39.586318 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1459) took 1170 ms. Trace:
W20260709 13:12:39.201885 25925 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.063s	user 0.013s	sys 0.000s
W20260709 13:12:39.335987 25707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.004s	sys 0.007s
W20260709 13:12:39.321864 25575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.007s	sys 0.004s
W20260709 13:12:39.597808 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.000s	sys 0.013s
I20260709 13:12:38.683064 25880 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.299203 25564 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:39.597882 25707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.008s	sys 0.008s
I20260709 13:12:39.600625 25880 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:39.603404 25905 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:39.604265 25905 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:39.589607 22913 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
W20260709 13:12:39.497159 25872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.005s	sys 0.053s
I20260709 13:12:38.688361 25939 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.303592 25936 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:39.319759 25960 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.178s	user 0.000s	sys 0.002s
W20260709 13:12:39.077724 25927 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.034s	user 0.012s	sys 0.000s
W20260709 13:12:39.450798 25817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.089s	sys 0.126s
W20260709 13:12:39.285033 25824 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.004s
I20260709 13:12:38.691620 25754 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:38.690251 25947 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:39.360074 25259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.005s	sys 0.005s
I20260709 13:12:39.593189 22760 rpcz_store.cc:276] 0709 13:12:38.415660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.567853 (+1152193us) service_pool.cc:224] Handling call
0709 13:12:39.585486 (+ 17633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:39.585810 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:39.585851 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.585915 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:39.585961 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.586039 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.586226 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:39.598088 25575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.007s	sys 0.012s
I20260709 13:12:39.589203 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:39.667155 22908 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:39.598276 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.000s	sys 0.014s
W20260709 13:12:39.584364 25434 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.177s	user 0.010s	sys 0.006s
I20260709 13:12:39.201399 25687 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.622756 22766 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:39.678056 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1458) took 1262 ms. Trace:
I20260709 13:12:39.678601 22766 rpcz_store.cc:276] 0709 13:12:38.415348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.633853 (+218505us) service_pool.cc:224] Handling call
0709 13:12:39.677934 (+1044081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:39.229732 25923 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:39.604962 25905 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:39.654942 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.610658 25872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.005s	sys 0.054s
I20260709 13:12:39.606087 25945 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:39.592305 25761 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:39.699268 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.657788 25817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.089s	sys 0.127s
W20260709 13:12:39.608026 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.001s	sys 0.013s
W20260709 13:12:39.707906 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.521s (SENT)
W20260709 13:12:39.715835 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.001s	sys 0.013s
I20260709 13:12:39.715996 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:39.717012 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.752s (SENT) [suppressed 20 similar messages]
W20260709 13:12:39.670061 25259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.006s	sys 0.006s
I20260709 13:12:39.609954 25925 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:39.708573 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 149091us
I20260709 13:12:39.679147 25824 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.745055 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:39.660952 25936 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:12:39.642619 25939 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.719061 25575 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.658291 25927 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.749155 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1453) took 1776 ms. Trace:
I20260709 13:12:39.770507 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1456) took 1460 ms. Trace:
I20260709 13:12:39.705866 25687 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.721987 25564 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:39.774372 22761 rpcz_store.cc:276] 0709 13:12:37.972772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.979331 (+  6559us) service_pool.cc:224] Handling call
0709 13:12:37.994315 (+ 14984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.034047 (+ 39732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.034098 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.052252 (+ 18154us) log.cc:844] Serialized 26 byte log entry
0709 13:12:38.307395 (+255143us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:38.307441 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.307525 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:38.712744 (+405219us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:39.748824 (+1036080us) raft_consensus.cc:1685] finished
0709 13:12:39.748848 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.749088 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24054,"raft.queue_time_us":52484,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":168,"spinlock_wait_cycles":7686784,"thread_start_us":202765,"threads_started":1,"wal-append.queue_time_us":372184}
W20260709 13:12:39.750392 25863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.341s	user 0.026s	sys 0.022s
I20260709 13:12:39.745533 25872 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:39.802846 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.794708 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 194705us
I20260709 13:12:39.775022 22762 rpcz_store.cc:276] 0709 13:12:38.309907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.310057 (+   150us) service_pool.cc:224] Handling call
0709 13:12:38.314675 (+  4618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.337430 (+ 22755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.337464 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.356203 (+ 18739us) log.cc:844] Serialized 26 byte log entry
0709 13:12:38.627394 (+271191us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:38.627438 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.627509 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:39.751619 (+1124110us) raft_consensus.cc:1685] finished
0709 13:12:39.751643 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.770395 (+ 18752us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":302,"raft.queue_time_us":32182,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":19099,"thread_start_us":199118,"threads_started":1,"wal-append.queue_time_us":298837}
W20260709 13:12:39.789371 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 181740us
I20260709 13:12:39.799516 22908 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:39.707185 25434 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.666082 25754 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.793375 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:39.725433 25923 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.777918 25259 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.829231 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:39.839200 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 122118us
I20260709 13:12:39.794651 25635 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.831589 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:39.795836 25863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.387s	user 0.026s	sys 0.022s
I20260709 13:12:39.829838 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:39.844453 22832 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:39.706820 25947 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.718425 25945 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.767969 25824 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.856389 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1455) took 1269 ms. Trace:
I20260709 13:12:39.858430 22835 rpcz_store.cc:276] 0709 13:12:38.587289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.634893 (+ 47604us) service_pool.cc:224] Handling call
0709 13:12:38.635190 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.635334 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:39.561184 (+925850us) log.cc:844] Serialized 14 byte log entry
0709 13:12:39.855965 (+294781us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.856037 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:39.856091 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.856175 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.856314 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":224801,"threads_started":1}
I20260709 13:12:39.783331 25575 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.834961 25964 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:39.226612 25890 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:39.841368 25817 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.852164 25859 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:39.865743 25897 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.138s	user 0.017s	sys 0.000s
I20260709 13:12:39.872875 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:39.874780 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:39.877458 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:39.862515 25984 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.835513 25872 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.856287 25975 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:39.879583 22766 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 6: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:39.882299 25925 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:39.876166 25564 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:12:39.910352 25754 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:12:39.912737 25931 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:39.913549 25931 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.914496 25951 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:39.883936 25932 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:39.915180 25951 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.915800 25932 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.919692 25945 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:39.885980 25966 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:12:39.877930 25817 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:39.923509 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2153) took 2556 ms (client timeout 2333 ms). Trace:
I20260709 13:12:39.924239 22758 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:12:39.924014 22763 rpcz_store.cc:269] 0709 13:12:37.367194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.380949 (+ 13755us) service_pool.cc:224] Handling call
0709 13:12:39.923234 (+2542285us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540037f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:39.923429 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743104896}
I20260709 13:12:39.924445 25927 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:39.886919 25434 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:39.925247 25975 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:39.887866 25978 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:39.874780 25964 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.888724 25761 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:39.853372 25635 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:39.868495 25790 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:39.950556 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:39.950822 23458 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:39.880613 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50381.3us
I20260709 13:12:39.951800 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:39.843920 25259 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.875588 25859 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:39.987916 25859 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:39.906461 25984 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:39.950340 25905 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:39.993703 25987 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:39.995366 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1436) took 2547 ms. Trace:
I20260709 13:12:40.186753 22758 rpcz_store.cc:276] 0709 13:12:37.447488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.447605 (+   117us) service_pool.cc:224] Handling call
0709 13:12:39.923974 (+2476369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:39.924765 (+   791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.995007 (+ 70242us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b540037f5d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:39.995274 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5694476672}
I20260709 13:12:39.879326 25963 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:39.976449 25434 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:12:39.876858 25957 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:12:39.928850 25977 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:40.206458 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:40.166538 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 157610us
W20260709 13:12:40.208755 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.196676 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:40.202013 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.209786 23458 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:40.210186 22911 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:40.211270 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:40.212170 22836 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:40.222810 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:40.266075 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.434053 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 394) took 3573 ms. Trace:
W20260709 13:12:40.247421 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.336817 22908 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:40.266906 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 91150.1us
I20260709 13:12:40.378978 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:40.479909 22915 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1391) took 1815 ms (client timeout 2098 ms). Trace:
W20260709 13:12:40.471256 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:40.481915 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.224433 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:40.378391 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 426653us
W20260709 13:12:40.198017 25789 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.515s	user 0.016s	sys 0.004s
I20260709 13:12:40.392213 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:40.187186 25861 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.489959 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.490509 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:40.459437 25938 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 2.165s	user 0.011s	sys 0.013s
W20260709 13:12:40.458501 25990 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
I20260709 13:12:40.467700 22840 rpcz_store.cc:276] 0709 13:12:36.860321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.872759 (+ 12438us) service_pool.cc:224] Handling call
0709 13:12:36.902778 (+ 30019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:36.903215 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:36.903257 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:37.018654 (+115397us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b180044c080. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:38.504592 (+1485938us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:38.505159 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:12:38.694621 (+189462us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:38.694674 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.694772 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.433781 (+1739009us) raft_consensus.cc:1685] finished
0709 13:12:40.433808 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.433967 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5515,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"spinlock_wait_cycles":46213248,"thread_start_us":189019,"threads_started":1,"wal-append.queue_time_us":1735231}
I20260709 13:12:40.507336 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:40.494603 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.518886 22760 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:12:40.519383 22760 pending_rounds.cc:85] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 7
I20260709 13:12:40.519681 22760 pending_rounds.cc:107] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:12:40.519982 22760 raft_consensus.cc:2889] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:40.518266 25987 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:40.354418 25824 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:40.363304 25902 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.490608 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:40.480459 22915 rpcz_store.cc:269] 0709 13:12:38.664514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.189069 (+1524555us) service_pool.cc:224] Handling call
0709 13:12:40.479809 (+290740us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:40.491681 23464 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:12:40.374540 25965 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:12:40.303071 25961 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.343s	user 0.000s	sys 0.010s
I20260709 13:12:40.459928 25984 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:40.186105 25880 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:40.302752 25972 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.439s	user 0.000s	sys 0.011s
W20260709 13:12:40.375365 25967 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
W20260709 13:12:40.376448 25988 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 13:12:40.307477 25937 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 2.004s	user 0.002s	sys 0.017s
I20260709 13:12:40.438830 25986 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:40.524490 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 132437us
I20260709 13:12:40.521752 25902 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.434540 25575 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:40.378768 25963 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:40.552484 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:40.537392 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:40.492668 25868 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.363222 25923 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:40.463066 25964 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:40.543159 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.312456 25815 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:40.546947 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:40.544461 25859 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.556s	user 0.015s	sys 0.007s
I20260709 13:12:40.534533 25824 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:40.534541 25550 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:40.551033 25890 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.008s
I20260709 13:12:40.561719 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.565157 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.566347 26001 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:12:40.566871 25902 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:40.567814 25971 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:12:40.568909 25991 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:12:40.574473 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:40.245201 25977 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.579840 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:40.570755 25815 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:40.590190 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:40.560352 25872 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:40.591046 22911 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:12:40.591747 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:40.527546 25861 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.608995 22911 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:40.609704 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:40.610608 22911 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 13:12:40.610982 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:40.611907 22911 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:12:40.611984 25995 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:12:40.612970 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:40.614151 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 339) took 1069 ms. Trace:
I20260709 13:12:40.614313 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:40.615402 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:40.614521 22914 rpcz_store.cc:276] 0709 13:12:39.544825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.580267 (+ 35442us) service_pool.cc:224] Handling call
0709 13:12:39.598333 (+ 18066us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:39.873793 (+275460us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:39.905213 (+ 31420us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.202009 (+296796us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.241936 (+ 39927us) log.cc:844] Serialized 26 byte log entry
0709 13:12:40.242141 (+   205us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:40.242193 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.379757 (+137564us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:40.379843 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.613856 (+234013us) raft_consensus.cc:1685] finished
0709 13:12:40.613958 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.614091 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24504,"prepare.queue_time_us":31233,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":11926,"raft.run_cpu_time_us":446,"raft.run_wall_time_us":65691,"spinlock_wait_cycles":620670720,"thread_start_us":245479,"threads_started":1,"wal-append.queue_time_us":426384}
I20260709 13:12:40.617028 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:40.617677 26000 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:40.622869 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1457) took 2292 ms. Trace:
I20260709 13:12:40.623353 22764 rpcz_store.cc:276] 0709 13:12:38.330532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.696340 (+365808us) service_pool.cc:224] Handling call
0709 13:12:39.544016 (+847676us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:39.723255 (+179239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:39.723290 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:39.757880 (+ 34590us) log.cc:844] Serialized 52 byte log entry
0709 13:12:39.905297 (+147417us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:39.905367 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.905440 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.622628 (+717188us) raft_consensus.cc:1685] finished
0709 13:12:40.622652 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.622803 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4429,"raft.queue_time_us":173548,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":21705,"spinlock_wait_cycles":20352,"thread_start_us":225959,"threads_started":2,"wal-append.queue_time_us":452875}
I20260709 13:12:40.572697 25824 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.640889 22758 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:40.544518 25880 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.643823 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:40.604696 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:40.609862 25861 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:40.617823 22761 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:40.601352 22840 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:40.633324 22914 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:40.634925 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:40.649621 22759 raft_consensus.cc:2376] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7. Re-sending same reply.
I20260709 13:12:40.638674 22913 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:40.603844 22879 connection.cc:256] RPC call timeout handler was delayed by -2.0736176030000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 13:12:40.561538 25868 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.580215 25550 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.645375 23461 raft_consensus.cc:2955] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:40.577342 25859 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:40.559037 25986 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.650029 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:40.682472 25963 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:40.840229 25947 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:40.843057 25859 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:40.844246 22879 connection.cc:256] RPC call timeout handler was delayed by -2.0267766570000276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 13:12:40.844947 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9317785399999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:12:40.716145 25969 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:40.681876 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:40.847048 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:40.747529 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1455) took 2689 ms. Trace:
I20260709 13:12:40.849530 23462 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:40.849790 22757 rpcz_store.cc:276] 0709 13:12:38.058392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.058631 (+   239us) service_pool.cc:224] Handling call
0709 13:12:38.073590 (+ 14959us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:38.131107 (+ 57517us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:40.201886 (+2070779us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.523867 (+321981us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:40.526575 (+  2708us) log.cc:844] Serialized 52 byte log entry
0709 13:12:40.526761 (+   186us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:40.526990 (+   229us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.527180 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.527255 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.747225 (+219970us) raft_consensus.cc:1685] finished
0709 13:12:40.747249 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.747450 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2229060,"prepare.queue_time_us":2110155,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":1819,"raft.queue_time_us":110082,"raft.run_cpu_time_us":1128,"raft.run_wall_time_us":53822,"spinlock_wait_cycles":23535104,"thread_start_us":226074,"threads_started":1,"wal-append.queue_time_us":345500}
W20260709 13:12:40.702615 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70158.3us
I20260709 13:12:40.686431 22836 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:40.654772 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 341) took 1098 ms. Trace:
W20260709 13:12:40.858814 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.858584 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 332) took 3367 ms. Trace:
I20260709 13:12:40.861900 23457 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:40.861606 22909 rpcz_store.cc:276] 0709 13:12:37.491551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.493009 (+  1458us) service_pool.cc:224] Handling call
0709 13:12:37.493340 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:37.493738 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.493764 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.263645 (+2769881us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b2c00051d28. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:40.278424 (+ 14779us) log.cc:844] Serialized 26 byte log entry
0709 13:12:40.582667 (+304243us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:40.641356 (+ 58689us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.641652 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.641804 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.843935 (+202131us) raft_consensus.cc:1685] finished
0709 13:12:40.858238 (+ 14303us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.858490 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60591,"raft.queue_time_us":552,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":4379,"spinlock_wait_cycles":23777536,"thread_start_us":268589,"threads_started":1,"wal-append.queue_time_us":384857}
I20260709 13:12:40.860864 22906 rpcz_store.cc:276] 0709 13:12:39.556644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.560507 (+  3863us) service_pool.cc:224] Handling call
0709 13:12:39.561788 (+  1281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:39.769323 (+207535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:39.769979 (+   656us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:39.780241 (+ 10262us) log.cc:844] Serialized 26 byte log entry
0709 13:12:39.949322 (+169081us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:40.104821 (+155499us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.165907 (+ 61086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.165987 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.654535 (+488548us) raft_consensus.cc:1685] finished
0709 13:12:40.654562 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.654694 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11413,"prepare.queue_time_us":141836,"prepare.run_cpu_time_us":11468,"prepare.run_wall_time_us":96491,"raft.queue_time_us":92583,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":46123,"thread_start_us":297916,"threads_started":2,"wal-append.queue_time_us":413676}
I20260709 13:12:40.859722 22840 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:40.865731 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:40.647173 25938 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:12:40.855345 22836 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:40.885478 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:40.887218 22911 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:40.643056 25824 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:40.890353 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:40.958706 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68429.2us
I20260709 13:12:40.891767 23459 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:40.893615 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.747406 25914 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:40.762842 26001 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.725885 25916 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:40.907706 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.797155 25434 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:40.978683 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:40.913998 22836 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:40.888599 25964 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.993908 25964 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:40.996379 25964 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:40.985342 25911 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:41.004455 26007 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:12:41.007494 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.009061 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:40.985174 26001 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:40.951241 25897 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:12:41.021482 25550 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:40.893587 25936 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:40.984663 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1445) took 3481 ms. Trace:
I20260709 13:12:40.986938 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:40.882719 25923 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:41.031281 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.880975 25859 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:41.040704 22836 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:41.051982 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:41.041486 22765 rpcz_store.cc:276] 0709 13:12:37.503305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.536040 (+ 32735us) service_pool.cc:224] Handling call
0709 13:12:37.536526 (+   486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:37.788462 (+251936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:37.788496 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:37.789238 (+   742us) log.cc:844] Serialized 52 byte log entry
0709 13:12:37.949540 (+160302us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:38.075612 (+126072us) log.cc:844] Serialized 14 byte log entry
0709 13:12:38.249140 (+173528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.292705 (+ 43565us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.458706 (+2166001us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:40.979409 (+520703us) raft_consensus.cc:1685] finished
0709 13:12:40.984321 (+  4912us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.984568 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238470,"prepare.queue_time_us":2038765,"prepare.run_cpu_time_us":9360,"prepare.run_wall_time_us":176837,"raft.queue_time_us":257670,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":30428,"spinlock_wait_cycles":56733440,"thread_start_us":501511,"threads_started":3,"wal-append.queue_time_us":305128}
I20260709 13:12:40.994016 25986 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:40.716701 25977 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:40.975721 26002 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:41.060384 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:41.068580 22906 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:40.918228 25987 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:12:41.147610 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:41.148036 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.040890 26001 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.041251 25936 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.893064 25969 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.150213 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:41.170997 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:41.171378 25815 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.003s
I20260709 13:12:41.172113 22915 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:41.187668 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:41.189211 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.063603 25923 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:41.227103 22762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.033s	sys 0.085s
W20260709 13:12:41.231894 22762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.033s	sys 0.086s
I20260709 13:12:40.987586 25914 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.068245 26002 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:41.038667 25434 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.248780 22839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:12:41.272142 22839 raft_consensus.cc:606] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:41.272815 22839 raft_consensus.cc:614] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:41.250797 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.154633 25789 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:41.293017 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 447116us
W20260709 13:12:41.297297 22879 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.751s (SENT)
I20260709 13:12:41.197237 25936 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:41.198038 26001 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.245230 26000 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.004s
I20260709 13:12:41.300803 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1396) took 1075 ms. Trace:
I20260709 13:12:41.067020 25859 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.317528 22837 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:41.327369 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:41.290088 26002 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:41.361948 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.330439 26015 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.144s	user 0.000s	sys 0.007s
I20260709 13:12:41.318763 25964 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.440972 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 110077us
I20260709 13:12:41.398684 26000 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:41.248378 25815 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.459543 25434 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:41.461930 25859 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.464084 25868 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:41.331064 26001 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.300318 25789 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:41.437379 26014 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:12:41.532878 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.500850 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.533912 22840 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:41.534718 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:41.532716 26008 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:12:41.475654 25964 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.569211 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:12:41.568765 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.567898 25824 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.001s
W20260709 13:12:41.580708 23461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.011s	sys 0.038s
I20260709 13:12:41.561115 25859 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:12:41.582780 23461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.012s	sys 0.038s
I20260709 13:12:41.563608 26000 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.570937 25914 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:41.578678 25894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.010s	sys 0.030s
I20260709 13:12:41.608525 25837 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:41.584717 25824 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.592367 25859 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:41.630357 22836 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.000s
I20260709 13:12:41.619055 25923 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.646448 22836 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:41.647521 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1457) took 1059 ms. Trace:
W20260709 13:12:41.642385 26017 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:12:41.612323 26000 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.651734 25923 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:41.652195 25434 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:41.670754 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.682072 25434 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.638641 25947 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:12:41.628444 25894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.010s	sys 0.031s
I20260709 13:12:41.709785 22913 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:41.694741 26028 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.710572 26028 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.685346 25434 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:41.714207 26025 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:12:41.686648 26031 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.715629 26031 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.717922 26031 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.708196 26018 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:41.696138 26013 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.164s	user 0.002s	sys 0.000s
W20260709 13:12:41.696064 26016 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:12:41.694842 25947 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:41.713295 26028 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.725623 26018 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.695652 26027 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:41.724296 25789 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.708666 26001 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.648193 22836 rpcz_store.cc:276] 0709 13:12:40.587743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.589483 (+  1740us) service_pool.cc:224] Handling call
0709 13:12:40.643508 (+ 54025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:41.647117 (+1003609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.647430 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:41.717924 25931 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.737684 25964 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:41.749706 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 160370us
I20260709 13:12:41.738945 25434 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.741151 22757 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:41.686547 25923 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:41.761258 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.742872 26033 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:41.761308 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.762611 22839 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:41.740306 26031 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:12:41.752439 26001 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.774832 26001 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.767985 22836 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:41.771318 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.755795 25964 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.776986 22839 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:41.777912 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.745733 26028 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:12:41.779350 22839 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:41.787375 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:41.781409 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:41.746026 26027 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.759598 25434 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.820451 22758 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:41.814196 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:41.819103 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:41.846803 22759 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:41.772814 26033 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.851850 22765 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:41.904249 22764 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:41.902491 22839 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:41.910547 22907 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:42.215742 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 342) took 2658 ms. Trace:
I20260709 13:12:42.216302 22907 rpcz_store.cc:276] 0709 13:12:39.556956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.908288 (+2351332us) service_pool.cc:224] Handling call
0709 13:12:41.910236 (+  1948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.215392 (+305156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.215665 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:42.217551 22759 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:42.025141 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 422) took 1292 ms (client timeout 1590 ms). Trace:
W20260709 13:12:42.052668 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 282747us
I20260709 13:12:42.234072 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:41.921945 22766 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:41.852283 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 333) took 1298 ms. Trace:
I20260709 13:12:42.235314 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:42.018065 22758 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:42.211719 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 392661us
I20260709 13:12:42.217975 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 345) took 1168 ms. Trace:
I20260709 13:12:42.236155 22764 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:42.236552 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:42.007033 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1445) took 3505 ms. Trace:
I20260709 13:12:42.236431 22757 rpcz_store.cc:276] 0709 13:12:41.049310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.049481 (+   171us) service_pool.cc:224] Handling call
0709 13:12:41.058708 (+  9227us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:12:41.147910 (+ 89202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:41.147947 (+    37us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:12:41.153731 (+  5784us) log.cc:844] Serialized 626 byte log entry
0709 13:12:41.606179 (+452448us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:41.722159 (+115980us) log.cc:844] Serialized 14 byte log entry
0709 13:12:41.978950 (+256791us) log.cc:844] Serialized 14 byte log entry
0709 13:12:42.129450 (+150500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.129600 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:42.129664 (+    64us) raft_consensus.cc:1685] finished
0709 13:12:42.129682 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:42.217905 (+ 88223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17856,"prepare.queue_time_us":206666,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":295715,"raft.run_cpu_time_us":673,"raft.run_wall_time_us":49246,"thread_start_us":406809,"threads_started":2,"wal-append.queue_time_us":436504}
I20260709 13:12:41.900823 22836 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:12:41.857226 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.235874 22762 rpcz_store.cc:276] 0709 13:12:40.554266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.557074 (+  2808us) service_pool.cc:224] Handling call
0709 13:12:40.557612 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:40.583096 (+ 25484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:40.583129 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.595837 (+ 12708us) log.cc:844] Serialized 26 byte log entry
0709 13:12:41.232347 (+636510us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:41.232391 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.232458 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:41.788277 (+555819us) raft_consensus.cc:1685] finished
0709 13:12:41.788736 (+   459us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.807035 (+ 18299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19773,"raft.queue_time_us":19877,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":45399,"thread_start_us":551310,"threads_started":1,"wal-append.queue_time_us":760321}
W20260709 13:12:41.933662 22880 connection.cc:256] RPC call timeout handler was delayed by -1.3131404729999758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
I20260709 13:12:42.026032 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:41.910162 22910 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:12:42.237362 22840 rpcz_store.cc:269] 0709 13:12:40.644653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.865353 (+220700us) service_pool.cc:224] Handling call
0709 13:12:41.936435 (+1071082us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54000a5d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:41.936715 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534198144}
I20260709 13:12:41.821653 25824 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:41.830333 26018 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.272746 22765 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:41.806929 26028 consensus_queue.cc:1243] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:42.314504 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:42.312661 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:42.314567 22759 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:42.240754 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 340) took 2684 ms. Trace:
W20260709 13:12:42.319083 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82633.8us
W20260709 13:12:42.054821 26041 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.229s	user 0.001s	sys 0.003s
I20260709 13:12:42.321372 22759 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:42.237774 22841 rpcz_store.cc:276] 0709 13:12:38.502101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.504305 (+  2204us) service_pool.cc:224] Handling call
0709 13:12:38.504619 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.646815 (+142196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.006628 (+3359813us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54001bc454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:42.006935 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299040128}
I20260709 13:12:42.337859 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:42.338829 22764 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:41.830888 26044 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:42.016693 26032 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:12:42.182122 26002 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.767s	user 0.014s	sys 0.001s
W20260709 13:12:42.306811 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.826766 26000 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.829857 25859 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.816244 26001 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.344937 22758 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:41.890594 25868 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:42.349172 26044 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:41.873323 26029 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:42.358392 26029 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.359179 26044 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.347433 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:42.360674 22841 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:41.805953 25964 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.022673 26031 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:41.909657 25880 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:41.817188 26039 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.524s
W20260709 13:12:42.376260 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.925316 26027 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.323968 22910 rpcz_store.cc:276] 0709 13:12:39.556102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.908989 (+2352887us) service_pool.cc:224] Handling call
0709 13:12:41.909733 (+   744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.240370 (+330637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.240678 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894336}
I20260709 13:12:42.212134 25687 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:41.827700 25923 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.241957 25758 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.021466 26046 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:42.390566 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67806.3us
W20260709 13:12:42.351332 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.230589 25916 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:42.178462 25993 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:42.313078 25789 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.242182 25434 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.353230 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.336498 25895 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:41.853266 25987 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.361889 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:42.382746 25859 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:41.872362 25936 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.372406 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:42.351961 25635 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.361390 26000 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.371779 26031 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:42.395438 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 161533us
I20260709 13:12:42.370987 26028 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:42.396741 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:42.435005 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:42.370419 26044 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:42.381238 26001 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:42.340440 26018 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:42.442101 25635 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:42.415685 22841 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:42.352972 26002 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.388221 26029 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.397879 25916 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:42.396843 22840 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:42.443651 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.463771 22841 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:42.405362 25993 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.461247 22839 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:42.426071 25859 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.447162 25895 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:12:42.454998 25936 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.455308 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.457937 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.451028 25880 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:12:42.488708 22912 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:42.458237 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.490175 22909 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:42.491094 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.492352 26033 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.404876 26046 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.493783 26054 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:12:42.494783 26046 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.444016 26044 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.420665 25987 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.448329 25923 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.497903 25987 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.497838 25923 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.457938 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.405256 25789 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.474274 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.506834 22840 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:42.437866 26050 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:42.458720 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.515910 22914 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:42.438926 25939 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.488029 22910 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:42.450033 25635 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:42.397403 25758 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.499936 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:42.500982 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 116118us
I20260709 13:12:42.447693 25687 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:42.506632 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:42.507879 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.436146 26000 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.487057 25859 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:42.519858 22840 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:42.485817 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 177475us
I20260709 13:12:42.486929 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:42.523583 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.631s (SENT)
I20260709 13:12:42.524113 22841 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:42.524142 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:42.524912 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.525017 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.525466 22840 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:42.525665 22841 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:42.528251 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:42.535610 22909 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:42.444260 26028 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.442597 26018 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.497345 26044 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.504199 26033 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.504612 25927 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:12:42.538049 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 92265.8us
I20260709 13:12:42.448055 26001 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.539940 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:42.552531 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.553233 22841 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:42.541363 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:42.543378 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.553010 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.514694 26050 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.505486 26039 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:12:42.579519 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:42.566836 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.683s (SENT)
I20260709 13:12:42.579509 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.580873 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:42.581760 26046 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:42.582980 22806 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.685s (SENT)
I20260709 13:12:42.571411 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.584838 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.585103 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.586045 26046 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.586614 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.586563 26046 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.588245 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.589232 26046 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:42.590445 22806 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.774s (SENT)
I20260709 13:12:42.593937 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.594805 22907 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:42.596119 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.598698 25993 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.600481 26028 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.602829 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.604012 26002 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:12:42.605084 22907 pending_rounds.cc:85] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 2
I20260709 13:12:42.605872 26001 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.556974 26033 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.608315 26033 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.608107 22907 pending_rounds.cc:107] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:42.609066 22907 raft_consensus.cc:2889] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:42.608963 25635 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.545473 25987 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:42.617944 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:42.545307 25936 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:42.621986 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77214.9us
I20260709 13:12:42.554306 25927 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:42.561862 26044 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:42.516049 26031 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:42.631716 22840 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:42.558089 26053 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:42.618309 26001 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.579610 25923 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:41.327405 22908 rpcz_store.cc:276] 0709 13:12:40.225793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.632831 (+407038us) service_pool.cc:224] Handling call
0709 13:12:40.633316 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:40.642231 (+  8915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:40.642282 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.678452 (+ 36170us) log.cc:844] Serialized 26 byte log entry
0709 13:12:40.847001 (+168549us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:41.289459 (+442458us) log.cc:844] Serialized 14 byte log entry
0709 13:12:41.300258 (+ 10799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.300358 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:41.300419 (+    61us) raft_consensus.cc:1685] finished
0709 13:12:41.300439 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.300602 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8338,"raft.queue_time_us":18511,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":31863,"thread_start_us":503489,"threads_started":2,"wal-append.queue_time_us":337117}
I20260709 13:12:42.651432 25259 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:42.666242 22663 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 5, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:12:42.667675 25758 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.668740 25707 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:42.669566 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:42.671054 22661 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 241) took 25154 ms (client timeout 15000 ms). Trace:
I20260709 13:12:42.672561 25778 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.676642 25927 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.677652 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1435) took 5662 ms. Trace:
I20260709 13:12:42.688369 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.728717 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:42.729164 22835 raft_consensus.cc:2410] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:12:42.619735 25987 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:42.728595 22833 rpcz_store.cc:276] 0709 13:12:37.014882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.406840 (+391958us) service_pool.cc:224] Handling call
0709 13:12:37.407189 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:39.827820 (+2420631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:39.827854 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.827898 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:39.827928 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.677321 (+2849393us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54000a5854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:42.677375 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:42.677559 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5561626496}
I20260709 13:12:42.631652 26044 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:42.750288 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1461) took 2149 ms (client timeout 2364 ms). Trace:
W20260709 13:12:42.750744 22835 rpcz_store.cc:269] 0709 13:12:40.600373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.008731 (+408358us) service_pool.cc:224] Handling call
0709 13:12:42.688246 (+1679515us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:42.750204 (+ 61958us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55010048}
W20260709 13:12:42.752104 22806 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.025s (SENT)
I20260709 13:12:42.729974 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:42.732757 22663 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 3 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 6 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:42.647006 26053 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:42.756502 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.710604 25902 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:42.778231 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48371.8us
W20260709 13:12:42.772112 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.779054 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:42.780352 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:42.782575 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:42.774168 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 124940us
I20260709 13:12:42.731026 26031 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:42.766682 22663 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:42.786099 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 239305us
I20260709 13:12:42.811795 22663 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:42.747980 25987 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:42.791278 26056 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:42.793025 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:42.758085 26044 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:42.793686 25707 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:42.801373 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.822186 22835 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:42.726513 25964 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:42.749053 25923 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:42.812403 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:42.788623 25975 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:42.790992 25758 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:42.793181 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1461) took 4210 ms. Trace:
W20260709 13:12:42.797469 25959 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 3.013s	user 0.001s	sys 0.005s
I20260709 13:12:42.784070 25911 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:42.722051 26001 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.747080 25778 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:42.799852 25932 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:42.837322 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44372.3us
I20260709 13:12:43.005578 22760 rpcz_store.cc:276] 0709 13:12:38.582415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.663543 (+1081128us) service_pool.cc:224] Handling call
0709 13:12:39.674440 (+ 10897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.520530 (+846090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.792918 (+2272388us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b5400368b54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:42.793115 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1941202304}
I20260709 13:12:42.995347 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:42.966090 22663 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 4, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:12:42.987226 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.777164 25927 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:42.784590 26004 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.810220 25994 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:42.838613 22833 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:12:42.845916 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.240s (SENT)
I20260709 13:12:43.011502 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1452) took 4507 ms. Trace:
I20260709 13:12:43.012724 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1474) took 1086 ms. Trace:
W20260709 13:12:42.836982 26061 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:12:43.091737 26062 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 13:12:43.090574 22765 rpcz_store.cc:276] 0709 13:12:41.926501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.313994 (+387493us) service_pool.cc:224] Handling call
0709 13:12:42.314544 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.338344 (+ 23800us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:42.424375 (+ 86031us) log.cc:844] Serialized 14 byte log entry
0709 13:12:42.543406 (+119031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:42.543472 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:42.543522 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.001018 (+457496us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b180043bee0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419
0709 13:12:43.012419 (+ 11401us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b540035dc54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:43.012500 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.012654 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":164936,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":117,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":12910,"spinlock_wait_cycles":59862528,"thread_start_us":204110,"threads_started":2,"wal-append.queue_time_us":208627}
I20260709 13:12:42.766865 26053 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:43.093014 25817 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:43.095633 22836 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1373) took 1383 ms (client timeout 1659 ms). Trace:
W20260709 13:12:43.096101 22836 rpcz_store.cc:269] 0709 13:12:41.711859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.776737 (+ 64878us) service_pool.cc:224] Handling call
0709 13:12:43.095431 (+1318694us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b540003a2d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:43.095561 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132264192}
W20260709 13:12:43.098201 26065 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:12:42.784802 26033 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:43.099004 26033 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:43.099437 26033 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:43.101106 26018 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:43.101804 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 340) took 2490 ms. Trace:
W20260709 13:12:43.102288 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96833.7us
I20260709 13:12:43.102222 22766 rpcz_store.cc:276] 0709 13:12:40.611743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.615172 (+  3429us) service_pool.cc:224] Handling call
0709 13:12:43.101585 (+2486413us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540035e154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:43.101729 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3000575744}
I20260709 13:12:43.105664 26000 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.106560 25969 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:43.106973 26066 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:12:43.107175 25969 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:43.108263 25969 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:42.785454 25993 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.098357 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1360) took 1423 ms. Trace:
I20260709 13:12:43.089740 22838 rpcz_store.cc:276] 0709 13:12:38.504172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.613587 (+109415us) service_pool.cc:224] Handling call
0709 13:12:38.686660 (+ 73073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.693404 (+  6744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.011261 (+4317857us) spinlock_profiling.cc:243] Waited 72.7 ms on lock 0x7b540027ea50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:43.011432 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167208192}
I20260709 13:12:43.110258 22839 rpcz_store.cc:276] 0709 13:12:41.674586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.787029 (+112443us) service_pool.cc:224] Handling call
0709 13:12:43.098141 (+1311112us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:43.098269 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139516672}
I20260709 13:12:43.096807 22663 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 3 to 5, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:12:43.101112 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1446) took 4598 ms. Trace:
I20260709 13:12:43.112913 22832 rpcz_store.cc:276] 0709 13:12:38.502410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.504784 (+  2374us) service_pool.cc:224] Handling call
0709 13:12:38.505104 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:39.844915 (+1339811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.100825 (+3255910us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000c6cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:43.101040 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079512832}
I20260709 13:12:43.071121 26074 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:43.072394 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.074247 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:43.097388 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:43.116178 23463 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:43.116470 22836 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:43.076573 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.076503 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.069285 25917 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.785239 25564 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.111820 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:43.142822 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:43.144026 22839 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:43.114729 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:43.145143 22835 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:43.075584 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:42.802219 25939 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:43.118297 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:42.915201 25964 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:43.120579 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:43.120745 22663 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 5, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 5 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:42.997205 26050 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:43.152289 22832 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 4.
I20260709 13:12:42.801123 25902 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.122298 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.124092 25993 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:42.720206 22661 rpcz_store.cc:269] 0709 13:12:17.536481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.564270 (+ 27789us) service_pool.cc:224] Handling call
0709 13:12:18.424073 (+859803us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5b090 00007f68a1af9f3a 00007f68a78b01cc 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc 00007f68a5e6e392 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:42.661153 (+24237080us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:32.197744 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:32.399363 (+201619us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:32.399407 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:12:32.399424 (+    17us) tablet.cc:662] Decoding operations
0709 13:12:32.626682 (+227258us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:32.666376 (+ 39694us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:32.744082 (+ 77706us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:12:33.560823 (+816741us) tablet.cc:701] Row locks acquired
0709 13:12:33.560864 (+    41us) write_op.cc:260] PREPARE: finished
0709 13:12:33.565895 (+  5031us) write_op.cc:270] Start()
0709 13:12:33.566050 (+   155us) write_op.cc:276] Timestamp: P: 1783602753565830 usec, L: 0
0709 13:12:33.566081 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:12:33.830862 (+264781us) log.cc:844] Serialized 9116 byte log entry
0709 13:12:34.285155 (+454293us) op_driver.cc:464] REPLICATION: finished
0709 13:12:34.287724 (+  2569us) write_op.cc:301] APPLY: starting
0709 13:12:34.287830 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:34.288287 (+   457us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:34.288411 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:34.509423 (+221012us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:34.578471 (+ 69048us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:12:34.578529 (+    58us) write_op.cc:312] APPLY: finished
0709 13:12:34.579395 (+   866us) log.cc:844] Serialized 152 byte log entry
0709 13:12:34.579686 (+   291us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:34.580384 (+   698us) write_op.cc:454] Released schema lock
0709 13:12:34.580478 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:34.581123 (+   645us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:34.581163 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37780,"reported_tablets":41,"spinlock_wait_cycles":707274496,"child_traces":[["op",{"apply.queue_time_us":2266,"apply.run_cpu_time_us":7963,"apply.run_wall_time_us":293290,"num_ops":22,"prepare.queue_time_us":290115,"prepare.run_cpu_time_us":20241,"prepare.run_wall_time_us":2084591,"raft.queue_time_us":61337,"raft.run_cpu_time_us":1372,"raft.run_wall_time_us":3365,"replication_time_us":718865,"thread_start_us":590972,"threads_started":4,"wal-append.queue_time_us":388926,"wal-append.run_cpu_time_us":8796,"wal-append.run_wall_time_us":1370341}]]}
I20260709 13:12:43.040287 25923 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:42.847669 26045 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.309s
W20260709 13:12:43.129912 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 284047us
I20260709 13:12:43.177160 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:43.178637 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1375) took 1466 ms. Trace:
I20260709 13:12:43.179090 22836 rpcz_store.cc:276] 0709 13:12:41.712476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.141196 (+1428720us) service_pool.cc:224] Handling call
0709 13:12:43.178569 (+ 37373us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:43.179702 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:43.180554 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.402s (SENT)
I20260709 13:12:42.817088 26042 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:43.114545 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:43.148370 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:43.176784 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 117760us
I20260709 13:12:43.185242 22833 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:43.150072 23458 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:42.871778 26063 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:43.155822 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.156239 22839 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:43.187702 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1462) took 2587 ms. Trace:
I20260709 13:12:43.188158 22839 rpcz_store.cc:276] 0709 13:12:40.600620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.155228 (+2554608us) service_pool.cc:224] Handling call
0709 13:12:43.155684 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.187347 (+ 31663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.187629 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.158603 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 419) took 2551 ms. Trace:
I20260709 13:12:43.109946 25994 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:43.181346 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 421) took 2537 ms. Trace:
I20260709 13:12:43.189692 22832 rpcz_store.cc:276] 0709 13:12:40.607504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.154886 (+2547382us) service_pool.cc:224] Handling call
0709 13:12:43.158089 (+  3203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.158291 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.158318 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.158359 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:43.158387 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.158440 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.158548 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.048483 25911 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.194773 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 424) took 2543 ms. Trace:
I20260709 13:12:43.192932 22836 rpcz_store.cc:276] 0709 13:12:40.644324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.180058 (+2535734us) service_pool.cc:224] Handling call
0709 13:12:43.180500 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.180754 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.180791 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.180855 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:43.180899 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.180970 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.181270 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.149492 25939 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:43.244942 23465 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:43.246106 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 425) took 2514 ms. Trace:
I20260709 13:12:43.201247 23466 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:43.109651 26004 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.252367 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 528) took 2872 ms. Trace:
I20260709 13:12:43.247471 22832 rpcz_store.cc:276] 0709 13:12:40.682519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.193363 (+2510844us) service_pool.cc:224] Handling call
0709 13:12:43.196464 (+  3101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.196635 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.196658 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.196698 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:43.196727 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.196776 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.196877 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.252774 23465 rpcz_store.cc:276] 0709 13:12:40.379618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.379757 (+   139us) service_pool.cc:224] Handling call
0709 13:12:43.200332 (+2820575us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:43.200558 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.251808 (+ 51250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.252302 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4052736}
I20260709 13:12:43.216615 23463 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:43.240204 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:43.183601 25923 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.206449 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:43.199263 22837 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1463) took 2598 ms (client timeout 2800 ms). Trace:
W20260709 13:12:43.257989 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51684us
W20260709 13:12:43.257894 22837 rpcz_store.cc:269] 0709 13:12:40.600853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.008766 (+407913us) service_pool.cc:224] Handling call
0709 13:12:43.199024 (+2190258us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b540017d9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:43.199175 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706245376}
I20260709 13:12:43.260394 22908 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:43.147578 25917 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.253059 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:43.255754 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.256614 22663 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 1 to 5, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 5 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:43.201151 22839 rpcz_store.cc:276] 0709 13:12:40.651234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.189203 (+2537969us) service_pool.cc:224] Handling call
0709 13:12:43.194213 (+  5010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.194394 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.194422 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.194483 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:43.194521 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.194596 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.194713 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.150861 25564 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:43.261909 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 156480us
I20260709 13:12:43.293386 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.299557 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.299852 22908 raft_consensus.cc:2410] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:43.246246 25939 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.193578 26063 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:43.194607 26060 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:43.151298 26067 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 13:12:43.245713 25817 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:43.157919 26056 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:12:43.258929 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:43.301592 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:43.374464 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 115625us
I20260709 13:12:43.374647 22915 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:43.375360 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:43.158108 25964 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:43.302340 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.195060 25941 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:43.304705 22909 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:43.253693 25916 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.000s
W20260709 13:12:43.306437 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53457us
I20260709 13:12:43.248247 25927 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:43.251765 26079 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:12:43.257169 25859 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.000s
I20260709 13:12:43.305898 22839 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:43.469132 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1368) took 1782 ms. Trace:
I20260709 13:12:43.469645 22839 rpcz_store.cc:276] 0709 13:12:41.687033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.305323 (+1618290us) service_pool.cc:224] Handling call
0709 13:12:43.305699 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.468754 (+163055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.469047 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:43.204453 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 472589us
I20260709 13:12:43.184585 26042 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:43.195544 25994 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:43.203545 25778 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.325598 22663 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:43.263512 25975 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:43.366176 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126152us
I20260709 13:12:43.371192 23457 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:43.147320 26074 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:43.377960 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.303334 25969 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:43.373234 25985 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.315734 26050 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:43.430024 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 124590us
I20260709 13:12:43.311792 26060 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:43.455483 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 141183us
I20260709 13:12:43.312414 26063 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.577229 22839 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:43.432485 25916 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.397082 25817 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.460846 25902 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.461344 25927 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.456185 25932 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:43.377983 26044 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 13:12:43.426585 26045 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:43.384655 25941 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:43.384297 26087 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.139s
I20260709 13:12:43.475917 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:43.570794 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:43.535185 23430 connection.cc:256] RPC call timeout handler was delayed by -1.4183634449999545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:12:43.613387 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 137566us
I20260709 13:12:43.591991 25975 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:43.582872 26075 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:12:43.582165 26060 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:43.578060 26050 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:43.588115 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.579903 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1369) took 1869 ms. Trace:
W20260709 13:12:43.540596 25987 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.000s
W20260709 13:12:43.542339 26076 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 13:12:43.458380 25816 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:43.599362 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.597740 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1458) took 2998 ms. Trace:
I20260709 13:12:43.600206 23462 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:43.466532 25951 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.477018 26042 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:43.601958 22663 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:43.604041 23459 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 369) took 3001 ms (client timeout 1940 ms). Trace:
I20260709 13:12:43.606252 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.568960 25969 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:43.609473 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 234181us
I20260709 13:12:43.532051 25911 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:43.582154 26085 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:12:43.619853 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1358) took 1945 ms. Trace:
I20260709 13:12:43.603361 25932 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:43.611660 26044 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:43.636852 26088 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
I20260709 13:12:43.617523 26021 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:12:43.607939 26045 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:43.585685 25758 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:43.640600 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:43.620139 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:43.641836 23430 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.590s (SENT)
W20260709 13:12:43.699607 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57812.2us
I20260709 13:12:43.537171 26033 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.643323 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:43.701174 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57896.6us
I20260709 13:12:43.647188 22906 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:43.701869 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:43.477569 25859 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:43.647944 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:43.703219 22758 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:43.647435 22839 rpcz_store.cc:276] 0709 13:12:41.710376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.478779 (+1768403us) service_pool.cc:224] Handling call
0709 13:12:43.576733 (+ 97954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.579595 (+  2862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.579839 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.537199 25923 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:43.635166 26004 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.618950 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 523) took 3848 ms. Trace:
I20260709 13:12:43.645139 25975 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.673096 22836 rpcz_store.cc:276] 0709 13:12:41.674084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.248388 (+1574304us) service_pool.cc:224] Handling call
0709 13:12:43.254123 (+  5735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:43.254481 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.254511 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:43.254978 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:12:43.518311 (+263333us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:43.518375 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.518493 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.619632 (+101139us) raft_consensus.cc:1685] finished
0709 13:12:43.619653 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.619778 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6971,"thread_start_us":147249,"threads_started":1,"wal-append.queue_time_us":324908}
I20260709 13:12:43.580960 25985 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:43.669365 23459 rpcz_store.cc:269] 0709 13:12:40.598355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.599171 (+   816us) service_pool.cc:224] Handling call
0709 13:12:43.599728 (+3000557us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:43.599877 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643297024}
I20260709 13:12:43.632907 25815 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:43.753211 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.619189 25941 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:43.662273 25895 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.662734 22834 rpcz_store.cc:276] 0709 13:12:40.599592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.488981 (+889389us) service_pool.cc:224] Handling call
0709 13:12:41.524534 (+ 35553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:41.561194 (+ 36660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:41.561227 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:41.561277 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:41.561311 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.593573 (+2032262us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b54000c76d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:43.593638 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.597647 (+  4009us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":692,"raft.queue_time_us":16863,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":152,"spinlock_wait_cycles":81716096}
I20260709 13:12:43.686950 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.631256 25564 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:43.622567 25964 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.633026 25917 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.649935 26050 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:43.663090 26053 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
I20260709 13:12:43.693490 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:43.649226 25993 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:43.696671 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 116035us
I20260709 13:12:43.698387 22766 raft_consensus.cc:2376] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7. Re-sending same reply.
I20260709 13:12:43.761701 25259 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:43.673326 25969 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:43.647464 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:43.617686 25916 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:43.705910 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:43.692893 25932 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:43.698737 22909 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:12:43.774646 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72866us
I20260709 13:12:43.712203 26018 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:43.712809 25789 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:43.720047 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.663604 25939 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:43.698383 26045 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.698046 25758 consensus_queue.cc:1243] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:43.662566 25987 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:43.730460 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.729094 26060 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:43.747170 25817 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:43.683629 25927 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:43.754931 22761 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:43.755169 22841 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:43.731521 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:43.732272 22836 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:43.756274 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.767406 22759 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:43.786455 22765 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260709 13:12:43.774614 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81216.1us
I20260709 13:12:43.725307 26001 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:43.788002 22766 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:43.787823 25259 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:43.789825 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84002.1us
I20260709 13:12:43.790012 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:43.791486 26074 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.793838 25941 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.794754 25859 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.795362 25687 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:43.796159 25923 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.797745 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:44.029125 23465 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:43.797837 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.743978 25985 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:43.799213 23463 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:12:43.711676 26099 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:43.788268 25970 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:43.800478 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.802779 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:43.806679 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1361) took 2131 ms. Trace:
I20260709 13:12:43.773761 25895 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.048167 22835 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:43.800951 26018 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:43.810797 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.053558 22834 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:43.889537 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.001276 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1374) took 2289 ms. Trace:
I20260709 13:12:44.054692 22836 rpcz_store.cc:276] 0709 13:12:41.712182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.727711 (+2015529us) service_pool.cc:224] Handling call
0709 13:12:43.732089 (+  4378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.000971 (+268882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.001200 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:43.874398 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.037595 23463 pending_rounds.cc:85] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 7
I20260709 13:12:44.056751 23463 pending_rounds.cc:107] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:12:44.057150 23463 raft_consensus.cc:2889] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:44.043241 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:43.809490 25951 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.049093 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1460) took 3449 ms. Trace:
W20260709 13:12:43.811560 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.788352 26053 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.048233 22837 rpcz_store.cc:276] 0709 13:12:41.674818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.260343 (+1585525us) service_pool.cc:224] Handling call
0709 13:12:43.260687 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.260968 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:43.562290 (+301322us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.806282 (+243992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.806340 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:43.806385 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.806510 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.806620 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20232,"prepare.queue_time_us":324484,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":4216,"raft.run_cpu_time_us":792,"raft.run_wall_time_us":68175,"thread_start_us":493451,"threads_started":2,"wal-append.queue_time_us":458166}
I20260709 13:12:44.063078 22835 rpcz_store.cc:276] 0709 13:12:40.600130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.155083 (+2554953us) service_pool.cc:224] Handling call
0709 13:12:44.047644 (+892561us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:12:44.047847 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.048758 (+   911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.049021 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3415424}
W20260709 13:12:44.065258 22732 connection.cc:256] RPC call timeout handler was delayed by -2.6198108789999424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:12:44.067867 22732 connection.cc:256] RPC call timeout handler was delayed by -2.5385061799998994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.46180000000000004s for a response
W20260709 13:12:44.068426 22732 connection.cc:256] RPC call timeout handler was delayed by -2.0335770419999335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 13:12:44.068821 22732 connection.cc:256] RPC call timeout handler was delayed by -1.9244346369999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:12:44.069186 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8893346369999335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:12:44.069556 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8614346370000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 13:12:44.070451 22732 connection.cc:256] RPC call timeout handler was delayed by -1.843434636999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:12:44.071352 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.075031 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:44.070935 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8209346369999366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260709 13:12:44.076095 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8173346369999308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260709 13:12:44.076526 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8020346369999061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 13:12:44.076896 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8020346369999061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
I20260709 13:12:43.900876 26004 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.151s
I20260709 13:12:44.065327 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:44.077286 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7291346369999019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 13:12:44.079986 22732 connection.cc:256] RPC call timeout handler was delayed by -1.720134637000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 13:12:44.080566 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6274346369999648s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 13:12:44.080878 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6166346369999474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 13:12:44.081152 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5842346370000087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:12:44.081422 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4924346369999739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
W20260709 13:12:44.081785 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4834346369999594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 13:12:44.019933 25313 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:12:44.047287 26099 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:44.095027 26099 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.030478 25817 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:12:44.097858 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.075706 25951 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.098577 22834 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:44.098865 25758 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:44.099704 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:44.029713 26044 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.028617 25687 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:43.904328 25916 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.155s
W20260709 13:12:44.082248 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4357346369999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
W20260709 13:12:44.060549 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.082737 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.086531 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.106755 22835 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:44.059803 26018 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:44.077617 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.051682 25970 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:44.103955 23465 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:44.107604 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:44.105545 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4222346369999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
W20260709 13:12:44.117592 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3952346369999304s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 13:12:44.118031 22732 connection.cc:256] RPC call timeout handler was delayed by -1.38263463699991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 13:12:44.118335 22732 connection.cc:256] RPC call timeout handler was delayed by -1.367334636999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
W20260709 13:12:44.118620 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3484346369999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
W20260709 13:12:44.118906 22732 connection.cc:256] RPC call timeout handler was delayed by -1.300734637000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
W20260709 13:12:44.119171 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2386346369999046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23020000000000002s for a response
W20260709 13:12:44.119424 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1542770419999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27290000000000003s for a response
W20260709 13:12:44.119673 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0172346370000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2548s for a response
I20260709 13:12:44.074144 25923 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:44.120885 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:44.075569 25939 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.117131 22906 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:44.126896 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.128925 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.129585 22834 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:44.121091 25970 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:44.121335 25313 consensus_queue.cc:1243] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:44.098645 25707 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:44.128473 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.143666 22763 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 5.
I20260709 13:12:44.109702 25687 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.105459 26033 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.134714 25969 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.151844 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:44.154157 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:44.165447 22841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1381) took 1684 ms (client timeout 2142 ms). Trace:
I20260709 13:12:44.104786 25946 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:12:44.171716 22841 rpcz_store.cc:269] 0709 13:12:42.480861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.579181 (+ 98320us) service_pool.cc:224] Handling call
0709 13:12:44.165080 (+1585899us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400038ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.165347 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701441280}
I20260709 13:12:44.165884 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.166054 22832 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 9. Ops: []
I20260709 13:12:44.130328 25817 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:12:44.098589 25259 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.135723 25923 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.233229 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.167416 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:44.184350 22834 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.217402 22907 raft_consensus.cc:2368] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9.
I20260709 13:12:44.206463 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.272068 22906 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:44.183465 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:44.273329 23464 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.269655 22835 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:44.273864 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:44.269543 22760 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:44.145149 25313 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:44.298729 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.299587 22760 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
W20260709 13:12:44.252794 26031 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.434s	user 0.015s	sys 0.001s
W20260709 13:12:44.232102 26107 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:12:44.305562 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.304292 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.307652 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:44.215890 26063 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.004s
I20260709 13:12:44.267489 25895 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.231012 25969 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.182819 25993 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:44.298003 25987 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.254637 25951 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:44.333108 22909 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.095s	sys 0.131s
I20260709 13:12:44.313910 22834 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.315110 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.170794 25707 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:44.315166 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:44.342408 22909 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.095s	sys 0.131s
I20260709 13:12:44.342877 22906 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.341909 22835 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:44.304795 25259 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:44.174557 25939 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.164s
I20260709 13:12:44.350373 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:44.314402 25985 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.351843 22835 raft_consensus.cc:2376] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6. Re-sending same reply.
I20260709 13:12:44.353447 22763 raft_consensus.cc:2368] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7.
I20260709 13:12:44.359409 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.347990 26053 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.355849 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.361461 22834 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.363056 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:44.365029 26110 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 13:12:44.366070 26116 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:12:44.334903 26063 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.376839 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:43.725878 23460 rpcz_store.cc:276] 0709 13:12:39.770773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.773423 (+  2650us) service_pool.cc:224] Handling call
0709 13:12:39.854062 (+ 80639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:39.879705 (+ 25643us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:40.268541 (+388836us) log.cc:844] Serialized 14 byte log entry
0709 13:12:40.716370 (+447829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:40.716462 (+    92us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:40.716537 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.618692 (+2902155us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b54002c8154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:43.618744 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.618885 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33114,"prepare.queue_time_us":496990,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":67427,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"spinlock_wait_cycles":34143104,"thread_start_us":724820,"threads_started":2,"wal-append.queue_time_us":894245,"wal-append.run_cpu_time_us":2845,"wal-append.run_wall_time_us":1370969}
I20260709 13:12:44.374527 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.380244 22763 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:44.379124 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.383193 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.384027 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:44.373827 26039 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:44.355369 25985 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.386090 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.389077 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:44.393397 22906 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:44.394826 26040 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.569s	user 0.009s	sys 0.004s
I20260709 13:12:44.389439 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.412994 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.409039 22835 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.492879 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.496105 25964 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:44.496734 25964 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:44.497722 26063 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.487142 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:44.500018 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.555s (SENT)
I20260709 13:12:44.500231 23458 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:44.500825 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.501147 25975 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.000s
I20260709 13:12:44.501895 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:44.503151 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.504231 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.504914 22836 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.486110 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:44.501319 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.440575 23464 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:44.507584 22835 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:12:44.448964 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 1.63712e+06us
I20260709 13:12:44.488404 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.509320 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:44.512637 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:44.423650 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.514314 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.506806 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:44.410065 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:44.516119 22759 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.516254 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:44.392905 25994 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.795s	user 0.002s	sys 0.007s
I20260709 13:12:44.518141 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.519143 23464 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:44.409786 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:44.520151 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.520999 23464 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:44.521198 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 348) took 1706 ms. Trace:
W20260709 13:12:44.522485 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1383) took 2021 ms (client timeout 2133 ms). Trace:
W20260709 13:12:44.523175 22840 rpcz_store.cc:269] 0709 13:12:42.500656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.584834 (+ 84178us) service_pool.cc:224] Handling call
0709 13:12:44.522215 (+1937381us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54000398d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.522400 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104969472}
I20260709 13:12:44.525038 23430 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:44.528563 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.528770 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 552) took 3644 ms. Trace:
I20260709 13:12:44.408003 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.529178 23462 rpcz_store.cc:276] 0709 13:12:40.884790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.884936 (+   146us) service_pool.cc:224] Handling call
0709 13:12:43.600003 (+2715067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.664437 (+ 64434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.528554 (+864117us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b54005136d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.528705 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6094401408}
I20260709 13:12:44.530663 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:44.476418 26045 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.004s
I20260709 13:12:44.529731 22834 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:44.399479 26024 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.724s	user 0.001s	sys 0.003s
W20260709 13:12:44.525377 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.532874 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.521808 22761 rpcz_store.cc:276] 0709 13:12:42.814491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.814640 (+   149us) service_pool.cc:224] Handling call
0709 13:12:42.817420 (+  2780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.002586 (+1185166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.520732 (+518146us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b5400312254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:44.521127 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155654528}
W20260709 13:12:44.452739 26042 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.759s	user 0.012s	sys 0.000s
W20260709 13:12:44.394809 25992 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 4.110s	user 0.010s	sys 0.017s
I20260709 13:12:44.538616 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:44.479506 26121 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:12:44.413961 26050 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.003s
I20260709 13:12:44.543699 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:44.463562 26115 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:12:44.558292 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.441478 26018 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.559684 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.395162 25923 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.548214 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:44.564284 22761 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:44.542706 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.398180 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.551084 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:44.551254 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:44.552977 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8033166449999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2364s for a response
I20260709 13:12:44.540652 26042 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:44.465571 26120 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:12:44.533727 25987 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.573648 25987 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.576400 25987 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.510322 26049 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.577688 26072 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:44.577520 26049 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:44.579730 26060 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.000s
I20260709 13:12:44.579909 26049 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.556105 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:44.556267 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.515084 22836 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.560657 22906 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:44.469156 26099 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.517064 25975 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.544184 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:44.531695 25983 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.564605 22835 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:12:44.565310 23458 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:44.590194 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 46076.4us
I20260709 13:12:44.590309 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.590731 22759 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:44.591127 22757 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:44.394248 26039 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:44.541522 25994 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.595561 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1401) took 4323 ms. Trace:
I20260709 13:12:44.567616 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.553042 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:44.569380 22879 connection.cc:256] RPC call timeout handler was delayed by -1.410916645000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27999999999999997s for a response
I20260709 13:12:44.596662 23457 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:44.596071 22911 rpcz_store.cc:276] 0709 13:12:40.272529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.612653 (+340124us) service_pool.cc:224] Handling call
0709 13:12:40.613060 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.888303 (+275243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.594260 (+3705957us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54002107d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:44.595492 (+  1232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629862656}
I20260709 13:12:44.431891 25970 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.569650 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:44.572254 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:44.548928 26045 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.556798 25956 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:44.398378 25985 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:44.594697 25932 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.786s	user 0.012s	sys 0.005s
I20260709 13:12:44.556917 25947 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.586892 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.542896 25964 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.613173 23464 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:44.591564 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.614455 22759 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:44.614467 22762 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:44.597616 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.572463 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.551757 25687 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.406551 25707 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.556134 25434 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:12:44.604383 22906 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:44.606873 25956 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:44.624169 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.561105 26050 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.608362 22836 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:44.609449 23458 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:44.610150 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:44.596009 25941 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 13:12:44.610949 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.547442 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:44.597023 26049 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:44.632015 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.604602 26072 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:44.621958 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:44.705152 23464 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.606158 23462 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:12:44.632261 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 84891.9us
I20260709 13:12:44.616187 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.614327 25947 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.625710 25956 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:44.623922 26039 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.692852 26049 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:44.694324 25970 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.695247 26018 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.624090 25964 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.695947 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.733675 22910 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:44.613812 25932 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.630038 25985 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.742420 26099 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:44.742358 25985 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:44.743563 25932 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.743925 26042 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.744603 25985 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.698174 25687 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:44.699664 25975 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.716813 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:44.747691 26039 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.748008 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:44.749722 25897 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:44.750594 25897 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:44.751014 25897 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.751431 25897 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:44.725885 22839 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:44.726244 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.755510 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.704406 26009 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:44.756695 26009 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.757184 26009 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:44.757766 22914 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.702400 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:44.739279 26124 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 13:12:44.740690 26045 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.741559 25947 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:44.738979 25994 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.768754 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:44.769256 22914 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:44.776245 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.783447 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.791600 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.792734 25983 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:44.792917 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:44.792919 26117 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.412s	user 0.000s	sys 0.011s
W20260709 13:12:44.794540 23466 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 536) took 1716 ms (client timeout 1655 ms). Trace:
I20260709 13:12:44.794937 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:44.795006 23466 rpcz_store.cc:269] 0709 13:12:43.077881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.122052 (+ 44171us) service_pool.cc:224] Handling call
0709 13:12:44.794336 (+1672284us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5400740c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.794448 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110411520}
I20260709 13:12:44.796172 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.797861 26049 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:44.801887 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.802112 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.803126 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.803821 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.806063 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.807082 22911 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:44.807367 26050 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.797326 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.810123 22910 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:44.810797 23464 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:44.812212 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.813663 22908 raft_consensus.cc:2376] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6. Re-sending same reply.
I20260709 13:12:44.814677 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.837203 26009 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:44.838160 26106 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.849s	user 0.005s	sys 0.003s
W20260709 13:12:44.842886 26109 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.768s	user 0.005s	sys 0.002s
I20260709 13:12:44.843354 26009 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:44.889308 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.621755 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.891419 22758 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:12:44.893675 22806 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.638s (SENT)
I20260709 13:12:44.895592 22835 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:44.899703 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1494) took 1257 ms. Trace:
I20260709 13:12:44.899827 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 356) took 1261 ms. Trace:
I20260709 13:12:44.905292 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:44.581727 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.910872 22837 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:44.821655 25956 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:44.937458 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.921556 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:44.948065 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.909087 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.963730 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2262) took 3814 ms. Trace:
I20260709 13:12:44.980468 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.982163 22906 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:44.982004 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:44.983004 22760 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:44.983428 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:44.984323 25956 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:44.984742 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:44.985014 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:44.991714 22761 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:44.993535 22837 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:44.988988 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2297) took 1181 ms. Trace:
I20260709 13:12:44.987569 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1359) took 3313 ms. Trace:
I20260709 13:12:44.971408 22838 raft_consensus.cc:2376] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8. Re-sending same reply.
I20260709 13:12:44.987548 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.010234 22906 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:44.989682 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96357.2us
W20260709 13:12:44.988242 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 384674us
W20260709 13:12:45.003018 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44279.1us
I20260709 13:12:44.966830 22757 rpcz_store.cc:276] 0709 13:12:43.641858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.645366 (+  3508us) service_pool.cc:224] Handling call
0709 13:12:43.650045 (+  4679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.591488 (+941443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.898579 (+307091us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54003408d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:44.899324 (+   745us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2163916160}
I20260709 13:12:44.998867 22914 rpcz_store.cc:276] 0709 13:12:43.807091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.594859 (+787768us) service_pool.cc:224] Handling call
0709 13:12:44.988458 (+393599us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001d0e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.988711 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294677248}
I20260709 13:12:44.980207 22913 rpcz_store.cc:276] 0709 13:12:41.149539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.152046 (+  2507us) service_pool.cc:224] Handling call
0709 13:12:41.194132 (+ 42086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:41.710322 (+516190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.963053 (+3252731us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b540010efd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:44.963507 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185092480}
W20260709 13:12:45.013664 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1387) took 2342 ms (client timeout 2083 ms). Trace:
W20260709 13:12:45.017489 22838 rpcz_store.cc:269] 0709 13:12:42.665174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.111418 (+446244us) service_pool.cc:224] Handling call
0709 13:12:44.904825 (+1793407us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:45.007249 (+102424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5692416}
I20260709 13:12:44.966902 22907 rpcz_store.cc:276] 0709 13:12:43.638065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.762886 (+124821us) service_pool.cc:224] Handling call
0709 13:12:44.899138 (+1136252us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54001e3cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.899480 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484739328}
I20260709 13:12:45.000593 22832 rpcz_store.cc:276] 0709 13:12:41.674347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.259773 (+1585426us) service_pool.cc:224] Handling call
0709 13:12:44.165694 (+905921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.271608 (+105914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.987057 (+715449us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400038ed0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:44.987301 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075218176}
W20260709 13:12:45.031368 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 623667us
I20260709 13:12:45.034749 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.029168 22910 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:45.035229 25955 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:45.036579 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.036648 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:45.036919 25955 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:45.037406 25955 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:45.037786 22910 raft_consensus.cc:2368] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7.
I20260709 13:12:45.038244 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.039027 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.038954 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.043722 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.044260 25817 consensus_queue.cc:1243] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:45.044692 22757 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:45.045924 25939 consensus_queue.cc:1243] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:45.046800 25939 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:45.047240 25939 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:45.047972 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.049638 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.050669 22913 raft_consensus.cc:2368] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:12:45.051412 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1372) took 3339 ms. Trace:
I20260709 13:12:45.051898 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.052796 25993 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:45.026996 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1502) took 1378 ms. Trace:
I20260709 13:12:45.041623 22759 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.053476 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:45.063690 22913 raft_consensus.cc:2368] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:12:45.048766 25939 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.051926 22839 rpcz_store.cc:276] 0709 13:12:41.711548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.709447 (+1997899us) service_pool.cc:224] Handling call
0709 13:12:43.709802 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.753822 (+1044020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.050865 (+297043us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540019ebd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:45.051194 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2336359808}
I20260709 13:12:45.054752 25993 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.072255 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.072451 25939 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:45.074515 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1495) took 1432 ms. Trace:
I20260709 13:12:45.075855 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.077890 25550 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:45.078284 22764 rpcz_store.cc:276] 0709 13:12:43.642173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.709487 (+ 67314us) service_pool.cc:224] Handling call
0709 13:12:43.712441 (+  2954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.722521 (+ 10080us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:43.764904 (+ 42383us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.996039 (+231135us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b180043be20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:45.073638 (+1077599us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.073780 (+   142us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:45.073920 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.074036 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.074262 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":680642,"prepare.queue_time_us":42019,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":888,"raft.queue_time_us":680784,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":182,"spinlock_wait_cycles":17635584,"thread_start_us":399313,"threads_started":2,"wal-append.queue_time_us":640762}
I20260709 13:12:45.038169 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.081799 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.083324 22907 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:45.084407 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:45.085386 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.086480 22915 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:12:45.087021 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 73598.9us
I20260709 13:12:45.087170 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.087643 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.088819 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.090950 22759 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:45.092268 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.093070 22759 raft_consensus.cc:2368] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:45.081230 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.095292 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1496) took 1452 ms. Trace:
I20260709 13:12:45.096760 22764 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:12:45.099196 22915 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:45.100023 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 355) took 1462 ms. Trace:
I20260709 13:12:45.100416 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 349) took 1515 ms. Trace:
I20260709 13:12:45.100945 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.101811 22764 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:45.038954 25955 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.105223 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:45.106274 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.107549 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.107957 25687 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:45.108886 22759 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 8 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:12:45.108719 26087 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:45.117641 22912 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.118508 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 352) took 1533 ms. Trace:
I20260709 13:12:45.109117 25687 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:45.109570 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 356) took 1011 ms. Trace:
I20260709 13:12:45.109619 25761 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:12:45.119549 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.642s (SENT)
I20260709 13:12:45.120158 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.120219 25687 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:45.100445 22915 rpcz_store.cc:276] 0709 13:12:43.637685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.098138 (+1460453us) service_pool.cc:224] Handling call
0709 13:12:45.098633 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.099670 (+  1037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.099875 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.118848 22912 rpcz_store.cc:276] 0709 13:12:43.585293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.094426 (+1509133us) service_pool.cc:224] Handling call
0709 13:12:45.094906 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.118172 (+ 23266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.118420 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.110562 22910 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:45.123682 22910 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:45.124531 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2290) took 1323 ms. Trace:
I20260709 13:12:45.124974 22910 rpcz_store.cc:276] 0709 13:12:43.801394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.122662 (+1321268us) service_pool.cc:224] Handling call
0709 13:12:45.123186 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.124269 (+  1083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.124471 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.057021 22765 rpcz_store.cc:276] 0709 13:12:43.648209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.011178 (+362969us) service_pool.cc:224] Handling call
0709 13:12:44.011698 (+   520us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:44.018456 (+  6758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:44.018509 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:44.019444 (+   935us) log.cc:844] Serialized 52 byte log entry
0709 13:12:44.190079 (+170635us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:44.225945 (+ 35866us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.025628 (+799683us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b180043bfa0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:45.026187 (+   559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.026325 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:45.026406 (+    81us) raft_consensus.cc:1685] finished
0709 13:12:45.026524 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.026749 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1240,"prepare.queue_time_us":35506,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":816603,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":7578,"spinlock_wait_cycles":104496512,"thread_start_us":134390,"threads_started":1,"wal-append.queue_time_us":320984}
I20260709 13:12:45.126834 22910 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:45.121017 25687 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:45.135794 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58209us
I20260709 13:12:45.105949 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.127321 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.121222 25999 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:45.120872 22759 rpcz_store.cc:276] 0709 13:12:44.098626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.107105 (+1008479us) service_pool.cc:224] Handling call
0709 13:12:45.109468 (+  2363us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:45.129055 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66573.2us
I20260709 13:12:45.095985 22758 rpcz_store.cc:276] 0709 13:12:43.642478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.704013 (+ 61535us) service_pool.cc:224] Handling call
0709 13:12:43.704360 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:44.990284 (+1285924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.094815 (+104531us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:45.095198 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2727820032}
I20260709 13:12:45.131374 22907 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:45.109174 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.137945 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2291) took 1330 ms. Trace:
I20260709 13:12:45.143007 22910 rpcz_store.cc:276] 0709 13:12:43.804701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.125831 (+1321130us) service_pool.cc:224] Handling call
0709 13:12:45.126454 (+   623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.134864 (+  8410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.135145 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.139320 22908 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:45.144830 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2296) took 1338 ms. Trace:
I20260709 13:12:45.145234 22908 rpcz_store.cc:276] 0709 13:12:43.806823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.814406 (+1007583us) service_pool.cc:224] Handling call
0709 13:12:44.908600 (+ 94194us) spinlock_profiling.cc:243] Waited 91.1 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:45.144742 (+236142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209452672}
I20260709 13:12:45.152978 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.158243 25687 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:45.161001 25687 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.162133 26002 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:45.162644 26002 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:45.163981 26002 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.167141 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2162) took 1025 ms. Trace:
I20260709 13:12:45.167704 22760 rpcz_store.cc:276] 0709 13:12:44.144608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.130532 (+985924us) service_pool.cc:224] Handling call
0709 13:12:45.167056 (+ 36524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.171149 22915 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:45.174710 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.175418 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2298) took 1364 ms. Trace:
I20260709 13:12:45.179543 22915 rpcz_store.cc:276] 0709 13:12:43.807335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.135886 (+1328551us) service_pool.cc:224] Handling call
0709 13:12:45.166621 (+ 30735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.171759 (+  5138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.172038 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35200}
W20260709 13:12:45.181913 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73007.9us
I20260709 13:12:45.181214 22765 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:45.183627 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:45.184051 22761 raft_consensus.cc:2376] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8. Re-sending same reply.
I20260709 13:12:45.183532 26087 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:45.186128 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2159) took 1045 ms. Trace:
I20260709 13:12:45.185849 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.100858 22909 rpcz_store.cc:276] 0709 13:12:43.584478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.802546 (+218068us) service_pool.cc:224] Handling call
0709 13:12:43.802894 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:43.803372 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.803417 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:43.804219 (+   802us) log.cc:844] Serialized 26 byte log entry
0709 13:12:44.348064 (+543845us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:44.800424 (+452360us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800416440. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:12:45.010302 (+209878us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.099509 (+ 89207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.099664 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:45.099739 (+    75us) raft_consensus.cc:1685] finished
0709 13:12:45.099873 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.100205 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10014,"prepare.queue_time_us":37175,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":131,"raft.queue_time_us":6723,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":8800,"spinlock_wait_cycles":26592256,"thread_start_us":703314,"threads_started":2,"wal-append.queue_time_us":553247}
I20260709 13:12:45.187014 26002 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:45.188380 22761 rpcz_store.cc:276] 0709 13:12:44.140739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.105231 (+964492us) service_pool.cc:224] Handling call
0709 13:12:45.186059 (+ 80828us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.192034 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:45.189771 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.642s (SENT)
I20260709 13:12:45.195930 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.199342 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:45.201259 22757 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.202472 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.202615 22910 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:45.203339 22757 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:45.203962 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2164) took 1057 ms. Trace:
I20260709 13:12:45.204330 22757 rpcz_store.cc:276] 0709 13:12:44.147026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.202072 (+1055046us) service_pool.cc:224] Handling call
0709 13:12:45.203897 (+  1825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.204856 25550 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:45.206511 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:45.207340 22758 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:45.208642 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.208925 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1491) took 1567 ms. Trace:
I20260709 13:12:45.209470 22758 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:45.210858 23461 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:12:45.213073 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 359) took 1192 ms. Trace:
I20260709 13:12:45.215121 22906 rpcz_store.cc:276] 0709 13:12:44.020452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.067859 (+1047407us) service_pool.cc:224] Handling call
0709 13:12:45.213002 (+145143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284352}
I20260709 13:12:45.213033 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.226619 22910 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:45.229897 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:45.231714 25890 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:45.209791 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:45.236343 25564 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:45.237000 25564 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:45.237470 25564 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.240167 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.240252 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.213963 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.209386 22762 rpcz_store.cc:276] 0709 13:12:43.640964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.642762 (+  1798us) service_pool.cc:224] Handling call
0709 13:12:43.643125 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.615267 (+972142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.208308 (+593041us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b5400221fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:45.208659 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2233461120}
I20260709 13:12:45.238773 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 423) took 4589 ms. Trace:
I20260709 13:12:45.211198 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:45.239012 22841 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:12:45.247973 22760 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.239701 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.262513 22761 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.267820 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.269645 22764 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:45.268739 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.268244 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.268734 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.270488 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2295) took 1460 ms. Trace:
I20260709 13:12:45.275216 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.275931 22764 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.276261 22757 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:45.191450 25707 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.229s
I20260709 13:12:45.277467 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.284312 22832 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:45.281160 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.247944 22833 rpcz_store.cc:276] 0709 13:12:40.648917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.186454 (+2537537us) service_pool.cc:224] Handling call
0709 13:12:43.250836 (+ 64382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.251051 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:43.365835 (+114784us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.206797 (+1840962us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b1800409de0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a
0709 13:12:45.237921 (+ 31124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.238058 (+   137us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:45.238156 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.238307 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.238524 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30643,"prepare.queue_time_us":199059,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":45555,"raft.queue_time_us":31917,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":190,"spinlock_wait_cycles":2546432,"thread_start_us":303851,"threads_started":2,"wal-append.queue_time_us":243176,"wal-append.run_cpu_time_us":1132,"wal-append.run_wall_time_us":1001175}
I20260709 13:12:45.282738 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 360) took 1251 ms. Trace:
I20260709 13:12:45.273288 22912 rpcz_store.cc:276] 0709 13:12:43.806512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.127378 (+1320866us) service_pool.cc:224] Handling call
0709 13:12:45.127749 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.128168 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:45.128588 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.263814 (+135226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.263870 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:45.266431 (+  2561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.266520 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.266633 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3605,"prepare.queue_time_us":250,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":15099,"spinlock_wait_cycles":92544,"thread_start_us":126723,"threads_started":1}
I20260709 13:12:45.292152 22759 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.286700 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.296468 22838 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:12:45.299310 22730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.655s (SENT) [suppressed 62 similar messages]
I20260709 13:12:45.300770 22911 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:45.301635 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:45.302297 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.303594 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:45.305578 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 135389us
I20260709 13:12:45.308650 26050 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:45.309329 26050 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:45.309831 26018 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:45.310672 26050 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:45.318835 22760 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:45.314514 22837 raft_consensus.cc:2376] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6. Re-sending same reply.
I20260709 13:12:45.305341 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:45.338301 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.341873 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.287171 22907 rpcz_store.cc:276] 0709 13:12:44.030964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.152213 (+1121249us) service_pool.cc:224] Handling call
0709 13:12:45.157228 (+  5015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.157771 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:45.158242 (+   471us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.282408 (+124166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.282468 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:45.282509 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.282594 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.282683 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7787,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":7062,"raft.queue_time_us":5815,"thread_start_us":114528,"threads_started":1}
I20260709 13:12:45.320503 22915 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:45.320497 22841 raft_consensus.cc:2368] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
I20260709 13:12:45.349905 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:45.349979 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.318140 25964 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.011s
I20260709 13:12:45.354030 22761 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:45.355085 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.355432 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.356127 22761 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 5: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.356889 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.338690 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.353015 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.362990 22760 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.366034 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.369235 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.369789 22839 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.370180 22761 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.371222 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.371703 22763 raft_consensus.cc:2410] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:12:45.371955 22761 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.372355 25313 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:45.372721 25313 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:45.358217 22762 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:45.375732 22832 raft_consensus.cc:2376] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8. Re-sending same reply.
W20260709 13:12:45.387372 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80730.4us
I20260709 13:12:45.388145 22758 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:45.389305 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.414912 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.417371 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.418102 22762 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:45.420593 22764 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:12:45.420722 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 121499us
I20260709 13:12:45.421793 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.386299 22835 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.422500 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 355) took 1324 ms. Trace:
I20260709 13:12:45.423614 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:45.423779 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.410912 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.387315 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.423519 22760 rpcz_store.cc:276] 0709 13:12:44.098273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.385845 (+1287572us) service_pool.cc:224] Handling call
0709 13:12:45.422438 (+ 36593us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.414424 22757 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:45.425565 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:45.426115 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.708s (SENT)
I20260709 13:12:45.426604 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.426169 22762 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
I20260709 13:12:45.429195 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.429862 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.419056 25890 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.429246 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 357) took 1330 ms. Trace:
I20260709 13:12:45.427225 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1500) took 1740 ms. Trace:
W20260709 13:12:45.431084 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125922us
I20260709 13:12:45.415478 25861 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:45.431555 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.431984 22912 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:45.433197 22757 rpcz_store.cc:276] 0709 13:12:44.098923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.287917 (+1188994us) service_pool.cc:224] Handling call
0709 13:12:45.429176 (+141259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.430757 22840 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8. Re-sending same reply.
I20260709 13:12:45.432713 26022 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 13:12:45.437616 26043 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:45.434695 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.428988 25999 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:45.440691 25313 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:45.440980 25890 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:45.425341 25550 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:12:45.442368 25313 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:45.443550 25890 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:45.441464 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.430673 26027 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:45.448693 22760 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:45.442432 25550 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:45.435065 22759 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7. Re-sending same reply.
I20260709 13:12:45.454061 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2299) took 1491 ms. Trace:
I20260709 13:12:45.456266 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.456279 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.458642 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.454591 22913 rpcz_store.cc:276] 0709 13:12:43.958812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.138997 (+1180185us) service_pool.cc:224] Handling call
0709 13:12:45.175637 (+ 36640us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:45.259176 (+ 83539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.259219 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:45.306244 (+ 47025us) log.cc:844] Serialized 52 byte log entry
0709 13:12:45.384300 (+ 78056us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:45.422538 (+ 38238us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.422783 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.422869 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:45.449636 (+ 26767us) raft_consensus.cc:1685] finished
0709 13:12:45.449665 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.449884 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38901,"raft.queue_time_us":20444,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":20049,"spinlock_wait_cycles":22514560,"thread_start_us":77511,"threads_started":1,"wal-append.queue_time_us":122717}
W20260709 13:12:45.460570 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.460992 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.482065 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.465910 25917 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:45.469831 22762 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:45.477797 22757 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.480031 25986 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:45.481469 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.481361 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.482072 22832 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:45.499084 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.485759 22758 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:45.433247 22766 rpcz_store.cc:276] 0709 13:12:43.647654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.022882 (+375228us) service_pool.cc:224] Handling call
0709 13:12:44.023751 (+   869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.024047 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:44.073531 (+ 49484us) log.cc:844] Serialized 14 byte log entry
0709 13:12:44.792334 (+718803us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:45.276316 (+483982us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.313996 (+ 37680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.320158 (+  6162us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:45.359688 (+ 39530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.377534 (+ 17846us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.388401 (+ 10867us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":868825,"prepare.queue_time_us":226521,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"raft.queue_time_us":870611,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":6005,"spinlock_wait_cycles":16474368,"thread_start_us":270264,"threads_started":3,"wal-append.queue_time_us":283072}
I20260709 13:12:45.502692 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.503938 22760 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:12:45.503850 22837 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:45.505336 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.506142 22764 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:45.453861 25550 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:45.510960 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.515110 22840 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:45.519249 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.520372 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.523128 25550 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.524510 25550 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:45.521409 22762 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:45.530043 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:45.537477 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:45.552500 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133106us
W20260709 13:12:45.553256 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69782.6us
I20260709 13:12:45.548673 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:45.561127 22762 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:45.504458 26018 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:12:45.547356 25917 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:45.567813 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1504) took 1912 ms. Trace:
I20260709 13:12:45.568414 22762 rpcz_store.cc:276] 0709 13:12:43.650661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.530562 (+1879901us) service_pool.cc:224] Handling call
0709 13:12:45.559365 (+ 28803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.562906 (+  3541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.563191 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.570223 22840 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:45.570387 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.571630 22766 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:45.551807 26043 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:45.574595 22760 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:12:45.572072 22841 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.559778 25564 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.580331 23457 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:45.581847 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1503) took 1928 ms. Trace:
I20260709 13:12:45.582449 22759 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:12:45.582304 22760 rpcz_store.cc:276] 0709 13:12:43.648506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.519825 (+1871319us) service_pool.cc:224] Handling call
0709 13:12:45.572601 (+ 52776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.576305 (+  3704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.576539 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.583323 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1505) took 1917 ms. Trace:
I20260709 13:12:45.583696 22759 rpcz_store.cc:276] 0709 13:12:43.665797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.551704 (+1885907us) service_pool.cc:224] Handling call
0709 13:12:45.582154 (+ 30450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.583055 (+   901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.583263 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:45.589241 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.589466 22764 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:45.590329 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 352) took 1537 ms. Trace:
I20260709 13:12:45.590735 22764 rpcz_store.cc:276] 0709 13:12:44.052932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.554744 (+1501812us) service_pool.cc:224] Handling call
0709 13:12:45.581444 (+ 26700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.589986 (+  8542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.590250 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:45.593034 22806 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.763s (SENT)
I20260709 13:12:45.566555 25917 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:45.600097 23461 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.574182 26050 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:45.607127 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.608074 23432 leader_election.cc:400] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9.
I20260709 13:12:45.608443 23432 leader_election.cc:403] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:45.594511 25897 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.735s	user 0.014s	sys 0.010s
I20260709 13:12:45.573817 26022 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:12:45.597301 26018 consensus_queue.cc:1243] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:45.618690 22761 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:45.616559 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 366) took 1071 ms. Trace:
I20260709 13:12:45.620944 22759 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:45.622903 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.622987 22840 raft_consensus.cc:1217] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-8.5]   Dedup: 4.3->[6.4-8.5]
I20260709 13:12:45.624189 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2165) took 1476 ms. Trace:
I20260709 13:12:45.624619 22759 rpcz_store.cc:276] 0709 13:12:44.147354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.597194 (+1449840us) service_pool.cc:224] Handling call
0709 13:12:45.620698 (+ 23504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.621471 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.624129 (+  2658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.621573 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.619472 22909 rpcz_store.cc:276] 0709 13:12:44.545129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.246620 (+701491us) service_pool.cc:224] Handling call
0709 13:12:45.273313 (+ 26693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.338267 (+ 64954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.338305 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:45.384663 (+ 46358us) log.cc:844] Serialized 26 byte log entry
0709 13:12:45.490357 (+105694us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:45.615532 (+125175us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.616203 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.616294 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:45.616340 (+    46us) raft_consensus.cc:1685] finished
0709 13:12:45.616356 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.616495 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1371,"raft.queue_time_us":19356,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":12891,"thread_start_us":134331,"threads_started":2,"wal-append.queue_time_us":94766}
W20260709 13:12:45.627493 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67606.5us
I20260709 13:12:45.623761 22840 pending_rounds.cc:85] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 3
I20260709 13:12:45.623661 22764 raft_consensus.cc:1217] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[7.4-7.4]   Dedup: 7.4->[]
I20260709 13:12:45.629588 22764 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:12:45.630218 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2168) took 1418 ms. Trace:
I20260709 13:12:45.630578 22764 rpcz_store.cc:276] 0709 13:12:44.211891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.594245 (+1382354us) service_pool.cc:224] Handling call
0709 13:12:45.623542 (+ 29297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.629945 (+  6403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.630160 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.641894 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 359) took 1542 ms. Trace:
I20260709 13:12:45.642431 22760 rpcz_store.cc:276] 0709 13:12:44.099444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.590244 (+1490800us) service_pool.cc:224] Handling call
0709 13:12:45.621471 (+ 31227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.624560 (+  3089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.624589 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.629151 (+  4562us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:45.641546 (+ 12395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.641684 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.641821 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.643188 25564 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:45.641731 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:45.643982 25564 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9.
I20260709 13:12:45.644852 26018 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:45.646145 25917 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.647766 25917 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:45.650908 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:45.651430 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.652274 22765 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:45.662233 23463 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:45.663496 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 83898.3us
I20260709 13:12:45.628284 22840 pending_rounds.cc:107] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:45.666201 22840 raft_consensus.cc:2889] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:45.666898 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2177) took 1313 ms. Trace:
I20260709 13:12:45.667378 22765 rpcz_store.cc:276] 0709 13:12:44.342236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.126948 (+784712us) service_pool.cc:224] Handling call
0709 13:12:45.655421 (+528473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.631386 22762 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:45.675596 22761 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:45.678129 25687 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.001s
I20260709 13:12:45.666592 26149 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:45.679232 26149 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.679667 26149 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.679940 22839 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.681223 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:45.682621 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.620688 25897 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.684638 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:45.684892 23462 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:45.686372 22764 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:12:45.686785 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.687453 22759 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: [6.2-6.2]
I20260709 13:12:45.688166 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2173) took 1411 ms. Trace:
I20260709 13:12:45.688508 22759 rpcz_store.cc:276] 0709 13:12:44.276641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.626219 (+1349578us) service_pool.cc:224] Handling call
0709 13:12:45.687149 (+ 60930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.687872 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.688095 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.688907 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.686765 22764 pending_rounds.cc:85] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 5
I20260709 13:12:45.689932 22764 pending_rounds.cc:107] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:12:45.689813 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.690260 22764 raft_consensus.cc:2889] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:45.690615 25890 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:45.691138 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2174) took 1392 ms. Trace:
I20260709 13:12:45.691211 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:45.691828 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98865us
I20260709 13:12:45.692842 22836 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.693889 26134 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:45.694964 26134 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:45.695304 26134 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:45.695761 26134 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.696220 26039 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:45.696946 25993 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:45.697445 25993 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:45.692869 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.805s (SENT)
I20260709 13:12:45.699097 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:45.699812 22758 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:45.699925 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:45.701858 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 56564.5us
I20260709 13:12:45.702121 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:45.704409 22759 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:45.704981 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:45.691670 22764 rpcz_store.cc:276] 0709 13:12:44.298587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.631254 (+1332667us) service_pool.cc:224] Handling call
0709 13:12:45.685956 (+ 54702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.690909 (+  4953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.691080 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.706898 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.708868 22832 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
W20260709 13:12:45.675613 26147 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:12:45.703356 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2166) took 1528 ms. Trace:
I20260709 13:12:45.711378 22762 rpcz_store.cc:276] 0709 13:12:44.147761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.596081 (+1448320us) service_pool.cc:224] Handling call
0709 13:12:45.631053 (+ 34972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.675211 (+ 44158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.676191 (+   980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.718717 22764 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:45.735561 22757 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:45.741120 22765 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:45.742666 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:45.741844 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2181) took 1362 ms. Trace:
W20260709 13:12:45.747455 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2175) took 1416 ms (client timeout 1638 ms). Trace:
I20260709 13:12:45.747148 26154 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 13:12:45.747967 22763 rpcz_store.cc:269] 0709 13:12:44.311244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.385701 (+ 74457us) service_pool.cc:224] Handling call
0709 13:12:45.701588 (+1315887us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:45.727984 (+ 26396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859640704}
I20260709 13:12:45.757776 26049 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:12:45.760146 26049 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:45.747087 22765 rpcz_store.cc:276] 0709 13:12:44.381892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.668276 (+1286384us) service_pool.cc:224] Handling call
0709 13:12:45.718129 (+ 49853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.741599 (+ 23470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.741780 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.763000 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:45.787695 22762 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:45.790230 22759 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:45.792001 22759 pending_rounds.cc:85] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:12:45.792343 22759 pending_rounds.cc:107] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:12:45.792618 22759 raft_consensus.cc:2889] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:45.765296 25564 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.795776 25956 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:12:45.809422 22761 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:45.818744 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 125927us
I20260709 13:12:45.819146 22765 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:45.825646 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 107060us
I20260709 13:12:45.786855 26018 consensus_queue.cc:1243] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:12:45.821228 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46179.8us
I20260709 13:12:45.782114 25983 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:45.786096 26050 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:45.810638 26049 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:45.772540 25707 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:12:45.818627 25964 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.881844 22761 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:12:45.938608 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.772097 25993 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:45.942732 22806 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.827s (SENT)
I20260709 13:12:45.841760 25983 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:45.954180 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:45.956986 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 206139us
I20260709 13:12:45.883711 26049 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:45.952262 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:45.966557 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.873559 25564 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.786752 26043 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:45.991097 22762 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:45.943200 25964 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:45.996719 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 257066us
I20260709 13:12:46.017035 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:46.021709 22762 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:46.029824 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.045707 22762 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:46.027583 26159 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:46.020804 25983 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.049717 26171 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:46.050449 25983 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:46.051566 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.048457 26159 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:46.059021 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86630.2us
W20260709 13:12:46.050594 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 122111us
W20260709 13:12:46.070700 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.042549 25993 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:46.077536 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 195638us
W20260709 13:12:46.097669 25550 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.573s	user 0.014s	sys 0.000s
W20260709 13:12:46.129050 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.134634 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.149716 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 353) took 2085 ms. Trace:
I20260709 13:12:46.154620 22766 rpcz_store.cc:276] 0709 13:12:44.063839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.578199 (+1514360us) service_pool.cc:224] Handling call
0709 13:12:45.617467 (+ 39268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.661861 (+ 44394us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:45.679256 (+ 17395us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.124149 (+444893us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.149136 (+ 24987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:46.149232 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:46.149307 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.149414 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.149625 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24681,"prepare.queue_time_us":19883,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":16543,"raft.queue_time_us":215022,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":17434,"thread_start_us":434298,"threads_started":2,"wal-append.queue_time_us":157722}
I20260709 13:12:46.162846 26171 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:46.134188 26156 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:46.181404 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:46.182116 26165 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:46.182945 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47703.6us
I20260709 13:12:46.186847 22766 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:46.190342 26162 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.312s	user 0.002s	sys 0.000s
I20260709 13:12:46.190737 22766 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:46.192283 22766 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:46.193276 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.068945 26050 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.204771 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.199206 22766 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:46.216740 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 221658us
W20260709 13:12:46.218123 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.216028 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68003.7us
I20260709 13:12:46.181042 26156 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:46.124248 26049 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.233670 25564 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:46.163128 26159 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.237591 26167 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:46.238380 26173 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:46.238790 26173 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.239842 26173 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.241852 26170 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:46.246373 25964 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:46.247018 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 230161us
I20260709 13:12:46.247953 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:46.248970 26182 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:46.250465 23461 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:12:46.270413 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 148920us
I20260709 13:12:46.271934 22879 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:46.254514 26173 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.171864 26166 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.236s
I20260709 13:12:46.258765 26167 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:46.287415 25861 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:46.288081 25861 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.292217 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:46.292928 22763 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:46.294098 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.296749 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.299105 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.300231 22909 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:12:46.300704 26172 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:12:46.302165 26022 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:46.302513 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.302772 26022 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:46.303344 22835 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.304143 26022 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.305763 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2176) took 1975 ms. Trace:
I20260709 13:12:46.264665 26170 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:46.305884 26175 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:46.307143 26170 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.311021 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.262320 26161 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:12:46.312557 22914 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.313848 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.314503 22841 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.314865 22763 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:46.319154 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 365) took 1774 ms. Trace:
I20260709 13:12:46.319376 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.319814 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.320180 22913 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:46.320514 22838 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.260600 25707 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:12:46.322288 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.322409 26156 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.323675 26156 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.324368 26148 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:46.324815 26148 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:46.319648 22915 rpcz_store.cc:276] 0709 13:12:44.544963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.187671 (+642708us) service_pool.cc:224] Handling call
0709 13:12:45.313839 (+126168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.349407 (+ 35568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.318937 (+969530us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002111d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:46.319090 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257246080}
I20260709 13:12:46.327078 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.328790 22832 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:46.330054 22838 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:46.330372 22833 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:46.332301 26022 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.334900 26166 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:46.335372 26166 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.336289 26166 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:46.339572 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.341018 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 86790.9us
I20260709 13:12:46.342458 26167 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.343029 26169 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:46.343811 26169 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.346253 26054 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:46.345112 26169 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.349550 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1510) took 1044 ms. Trace:
I20260709 13:12:46.350025 22759 rpcz_store.cc:276] 0709 13:12:45.304778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.791236 (+486458us) service_pool.cc:224] Handling call
0709 13:12:45.791725 (+   489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:45.928840 (+137115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.928891 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:45.946853 (+ 17962us) log.cc:844] Serialized 52 byte log entry
0709 13:12:46.150209 (+203356us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:46.150253 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.150332 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.349286 (+198954us) raft_consensus.cc:1685] finished
0709 13:12:46.349314 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.349454 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13345,"raft.queue_time_us":246649,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":126,"thread_start_us":157351,"threads_started":1,"wal-append.queue_time_us":311897}
W20260709 13:12:46.351303 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.351706 22907 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:46.351876 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.353691 22912 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:46.353705 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:46.357426 22837 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:46.360064 22837 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:46.306299 22760 rpcz_store.cc:276] 0709 13:12:44.330459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.643406 (+1312947us) service_pool.cc:224] Handling call
0709 13:12:45.643812 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.648283 (+  4471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.648315 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:45.648836 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:12:45.993235 (+344399us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:46.009238 (+ 16003us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.160832 (+151594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.160919 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.305449 (+144530us) raft_consensus.cc:1685] finished
0709 13:12:46.305474 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.305685 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4771,"prepare.queue_time_us":16077,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":167560,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":22463,"spinlock_wait_cycles":111488,"thread_start_us":495033,"threads_started":2,"wal-append.queue_time_us":474410}
I20260709 13:12:46.358641 25999 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:46.362999 25999 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:46.365722 25999 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.366477 26163 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:46.366932 25897 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:46.367377 25999 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:46.367980 22837 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.369002 22837 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.366990 26163 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.362730 22914 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:46.358705 25707 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:46.375010 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.376247 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.376433 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100063us
I20260709 13:12:46.385016 22837 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:46.391208 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:46.391949 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.372969 26149 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.693s	user 0.014s	sys 0.000s
W20260709 13:12:46.401641 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.412827 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:46.363636 26166 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.415220 22841 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:12:46.416865 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.390729 25955 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.387686 26163 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:46.429657 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.430469 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.429776 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.431491 22907 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:46.431378 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:46.442162 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.443969 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.444190 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.449872 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.456507 22909 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:46.479290 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.481165 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.482091 22907 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:46.392791 25707 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.417439 26149 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.416625 26148 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.507846 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:46.447403 26018 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:46.485306 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72597.7us
I20260709 13:12:46.520893 22835 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:46.508059 25687 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:46.468467 26134 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.772s	user 0.008s	sys 0.012s
W20260709 13:12:46.531314 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.484874 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 367) took 1939 ms. Trace:
I20260709 13:12:46.486076 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:46.489797 26187 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:12:46.491089 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:46.496857 25917 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.849s	user 0.010s	sys 0.005s
I20260709 13:12:46.496481 22913 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:46.505797 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:46.509057 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.542524 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1514) took 1152 ms. Trace:
I20260709 13:12:46.427603 25861 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.535013 22910 rpcz_store.cc:276] 0709 13:12:44.545289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.204216 (+658927us) service_pool.cc:224] Handling call
0709 13:12:45.213636 (+  9420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:45.226321 (+ 12685us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:45.423527 (+197206us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.563861 (+140334us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:45.579334 (+ 15473us) log.cc:844] Serialized 52 byte log entry
0709 13:12:45.579671 (+   337us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:45.763608 (+183937us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.958260 (+194652us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.371245 (+412985us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b180034de60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:46.484415 (+113170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.484592 (+   177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.484645 (+    53us) raft_consensus.cc:1685] finished
0709 13:12:46.484658 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.484810 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23893,"prepare.queue_time_us":131435,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":116,"raft.queue_time_us":447739,"raft.run_cpu_time_us":773,"raft.run_wall_time_us":46957,"spinlock_wait_cycles":16712960,"thread_start_us":530689,"threads_started":4,"wal-append.queue_time_us":242398}
I20260709 13:12:46.543241 22765 rpcz_store.cc:276] 0709 13:12:45.390157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.929924 (+539767us) service_pool.cc:224] Handling call
0709 13:12:45.949913 (+ 19989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:46.173395 (+223482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.180731 (+  7336us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:46.220442 (+ 39711us) log.cc:844] Serialized 26 byte log entry
0709 13:12:46.422401 (+201959us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:46.422455 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.422549 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.538843 (+116294us) raft_consensus.cc:1685] finished
0709 13:12:46.538876 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.542415 (+  3539us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28576,"raft.queue_time_us":188023,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":28833,"thread_start_us":362327,"threads_started":2,"wal-append.queue_time_us":253308}
I20260709 13:12:46.558183 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:46.561475 22907 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:46.530110 25983 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:46.561154 22806 connection.cc:256] RPC call timeout handler was delayed by -1.51747286300008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2634s for a response
W20260709 13:12:46.573344 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4098375290000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:12:46.549321 26018 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:46.573786 22806 connection.cc:256] RPC call timeout handler was delayed by -1.387628272000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
W20260709 13:12:46.579665 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3813282720000188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
I20260709 13:12:46.549535 25955 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.571391 25983 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.540026 26134 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.586905 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:46.591708 25983 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.591780 26188 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:46.594631 26188 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:46.595036 26188 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.599529 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.582665 25687 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:46.618868 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.619957 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1481) took 1033 ms. Trace:
W20260709 13:12:46.622246 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 166839us
W20260709 13:12:46.630015 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.631678 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3260975790000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
W20260709 13:12:46.632110 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2455647990000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:12:46.632423 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2085374110000657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 13:12:46.632704 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1756157030000622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:12:46.632992 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1282483710000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 13:12:46.633378 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0528233109999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
I20260709 13:12:46.638170 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.639127 22908 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:46.638279 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:46.641752 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.641772 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52850.1us
W20260709 13:12:46.641950 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 571) took 1873 ms (client timeout 2123 ms). Trace:
I20260709 13:12:46.588218 25955 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:46.645984 23462 rpcz_store.cc:269] 0709 13:12:44.763289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.768334 (+  5045us) service_pool.cc:224] Handling call
0709 13:12:46.618411 (+1850077us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:46.636822 (+ 18411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075234048}
I20260709 13:12:46.600625 26054 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:12:46.634832 25917 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.650410 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.660631 22913 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:46.652612 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1507) took 1349 ms. Trace:
I20260709 13:12:46.620393 22840 rpcz_store.cc:276] 0709 13:12:45.586365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.586659 (+   294us) service_pool.cc:224] Handling call
0709 13:12:45.589348 (+  2689us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:45.825291 (+235943us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:45.937975 (+112684us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.295812 (+357837us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b1800427cc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:12:46.459996 (+164184us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:46.507277 (+ 47281us) log.cc:844] Serialized 52 byte log entry
0709 13:12:46.507463 (+   186us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:46.507500 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.507611 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.601408 (+ 93797us) raft_consensus.cc:1685] finished
0709 13:12:46.619691 (+ 18283us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.619882 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12031,"prepare.queue_time_us":117086,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":4486,"raft.queue_time_us":53064,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":18961,"spinlock_wait_cycles":3386624,"thread_start_us":244370,"threads_started":2,"wal-append.queue_time_us":619446}
I20260709 13:12:46.642364 26050 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:46.662053 22758 rpcz_store.cc:276] 0709 13:12:45.303680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.740212 (+436532us) service_pool.cc:224] Handling call
0709 13:12:45.765603 (+ 25391us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:46.044866 (+279263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.045165 (+   299us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:46.069110 (+ 23945us) log.cc:844] Serialized 78 byte log entry
0709 13:12:46.228277 (+159167us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:46.228336 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.228410 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.652351 (+423941us) raft_consensus.cc:1685] finished
0709 13:12:46.652372 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.652543 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6556,"raft.queue_time_us":257946,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":25649,"thread_start_us":336649,"threads_started":2,"wal-append.queue_time_us":207868}
I20260709 13:12:46.644039 25687 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.659818 26153 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.690038 26153 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.693075 25861 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:46.702451 26200 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:12:46.713968 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:46.715263 26022 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:46.716260 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:46.717597 26022 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:46.724130 22913 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:12:46.730545 22760 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:46.733315 25707 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:46.741000 26171 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:46.741784 26171 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.743130 26171 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.745067 26171 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:46.747176 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57651us
W20260709 13:12:46.750142 26151 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:12:46.761870 26050 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:46.763376 26050 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:46.767184 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 85575.8us
I20260709 13:12:46.770429 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 577) took 1344 ms. Trace:
I20260709 13:12:46.771221 23457 rpcz_store.cc:276] 0709 13:12:45.425806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.425986 (+   180us) service_pool.cc:224] Handling call
0709 13:12:45.430921 (+  4935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.588796 (+157875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.769955 (+1181159us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400093dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:46.770287 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339313920}
I20260709 13:12:46.774085 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1494) took 1132 ms. Trace:
I20260709 13:12:46.774716 22839 rpcz_store.cc:276] 0709 13:12:45.641166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.682352 (+ 41186us) service_pool.cc:224] Handling call
0709 13:12:46.773746 (+1091394us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b5400038ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:46.773995 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998075648}
W20260709 13:12:46.780200 25993 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.670s	user 0.004s	sys 0.007s
I20260709 13:12:46.782827 26134 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:46.784972 26169 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:46.785604 26169 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.786921 26169 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.787850 22836 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:46.790239 26149 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:46.796528 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 573) took 1666 ms (client timeout 1952 ms). Trace:
I20260709 13:12:46.796993 25955 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:46.807438 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 448) took 1087 ms. Trace:
I20260709 13:12:46.798560 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.800318 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:46.800545 26168 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:46.684056 26018 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.803061 25687 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:46.805403 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.797251 23463 rpcz_store.cc:269] 0709 13:12:45.175271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.183764 (+  8493us) service_pool.cc:224] Handling call
0709 13:12:46.796283 (+1612519us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54000b6654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:46.796434 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579270912}
I20260709 13:12:46.749049 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:46.808800 22762 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:12:46.808080 22838 rpcz_store.cc:276] 0709 13:12:45.719619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.806252 (+1086633us) service_pool.cc:224] Handling call
0709 13:12:46.806665 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:46.806908 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.806951 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:46.807019 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:46.807068 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.807186 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.807354 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:46.816606 22732 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.676s (SENT)
W20260709 13:12:46.817106 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100978us
I20260709 13:12:46.789192 26169 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.813309 26039 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:46.825913 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77087.7us
I20260709 13:12:46.826156 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:46.826802 22838 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:46.822790 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:46.823844 22915 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:46.829149 26039 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.829545 26043 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:12:46.831911 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.810948 25897 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.834443 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.835059 22910 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.811426 26168 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:46.822850 25955 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:46.679723 25917 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:46.845957 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.832233 26170 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:46.852411 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:46.851961 26039 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:46.862795 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:46.864197 26149 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:46.865141 26151 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:46.865449 26149 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:46.865715 26151 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:46.865877 26149 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:46.867274 26190 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.233s	user 0.000s	sys 0.003s
I20260709 13:12:46.866868 26149 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.866816 26151 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:46.870644 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.843585 26134 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:46.872946 22906 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:46.880330 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:46.880576 26149 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.881031 22839 raft_consensus.cc:2376] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6. Re-sending same reply.
I20260709 13:12:46.842888 25897 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.885030 26049 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:46.878048 26160 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:46.847759 26173 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:46.900063 22910 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:46.902704 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:46.906489 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89914.8us
W20260709 13:12:46.908823 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.906064 25313 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.914062 25313 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.916325 25313 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:46.917070 25564 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.908353 26182 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.917670 25564 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.917984 26182 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.920567 25564 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:12:46.921847 26168 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:46.922681 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.923043 26168 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:46.923475 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.908231 26160 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:46.930218 22835 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:46.930328 26160 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:46.931700 26160 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:46.932423 22732 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.020s (SENT)
I20260709 13:12:46.932971 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:46.934836 22839 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:12:46.935146 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:46.937666 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:46.937964 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:46.939982 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1495) took 1298 ms. Trace:
I20260709 13:12:46.950843 22838 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:12:46.941856 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:46.941843 26195 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:46.952118 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:46.942970 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:46.953110 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.906080 26167 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:12:46.946938 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:46.960096 26134 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:46.960481 26194 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.300s	user 0.002s	sys 0.000s
I20260709 13:12:46.947137 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1506) took 1644 ms. Trace:
I20260709 13:12:46.960752 26134 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:46.947947 26166 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.947402 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:46.879392 26170 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:46.961822 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:46.962716 22836 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:46.963110 26170 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.964110 26134 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:46.965718 26188 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:46.958510 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:46.959625 22838 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:46.949714 22832 rpcz_store.cc:276] 0709 13:12:45.641432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.361069 (+719637us) service_pool.cc:224] Handling call
0709 13:12:46.393986 (+ 32917us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:46.485632 (+ 91646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.485670 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:46.537849 (+ 52179us) log.cc:844] Serialized 26 byte log entry
0709 13:12:46.635121 (+ 97272us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:46.806590 (+171469us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.824909 (+ 18319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.824998 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.930950 (+105952us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b1800260f20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:46.939656 (+  8706us) raft_consensus.cc:1685] finished
0709 13:12:46.939771 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.939917 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11175,"prepare.queue_time_us":236111,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":4794,"raft.queue_time_us":41979,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4276,"spinlock_wait_cycles":27529344,"thread_start_us":290627,"threads_started":3,"wal-append.queue_time_us":262286}
I20260709 13:12:46.908221 25955 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:46.961197 25983 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:46.972208 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.961052 22757 rpcz_store.cc:276] 0709 13:12:45.303182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.739687 (+436505us) service_pool.cc:224] Handling call
0709 13:12:45.740363 (+   676us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:46.236930 (+496567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.236965 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:46.237798 (+   833us) log.cc:844] Serialized 78 byte log entry
0709 13:12:46.541142 (+303344us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:46.541199 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.541296 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.866407 (+325111us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:46.946782 (+ 80375us) raft_consensus.cc:1685] finished
0709 13:12:46.946807 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.947054 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10229,"raft.queue_time_us":291783,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":34761,"spinlock_wait_cycles":9867648,"thread_start_us":719081,"threads_started":2,"wal-append.queue_time_us":340613}
W20260709 13:12:47.002935 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.977288 26182 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:46.976560 26163 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.001s
I20260709 13:12:46.960180 26167 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:46.982867 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 138496us
I20260709 13:12:46.967163 26173 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.006886 22909 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:47.015928 22840 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:47.019025 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60593.9us
I20260709 13:12:47.022047 26167 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:47.022936 26156 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.699s	user 0.000s	sys 0.019s
I20260709 13:12:47.032109 26183 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:47.034373 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.037279 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 99806.7us
W20260709 13:12:47.038930 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.035926 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:47.041958 26167 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.042761 25999 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:47.043522 26173 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.043401 25999 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.023140 25955 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:47.046067 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 113699us
W20260709 13:12:47.047238 22732 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.083s (SENT)
I20260709 13:12:47.047441 25955 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.047727 26153 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:47.048139 22732 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.098s (SENT)
I20260709 13:12:47.049264 26199 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.050268 26182 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:47.050679 26182 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:47.024430 25964 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:47.052596 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:47.043666 26167 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:47.055281 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1496) took 1362 ms. Trace:
I20260709 13:12:47.056030 22833 rpcz_store.cc:276] 0709 13:12:45.693188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.780274 (+1087086us) service_pool.cc:224] Handling call
0709 13:12:46.780922 (+   648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:46.809569 (+ 28647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:46.809607 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:46.819681 (+ 10074us) log.cc:844] Serialized 26 byte log entry
0709 13:12:46.876968 (+ 57287us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:47.054674 (+177706us) log.cc:844] Serialized 14 byte log entry
0709 13:12:47.054943 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.055020 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:47.055065 (+    45us) raft_consensus.cc:1685] finished
0709 13:12:47.055079 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:47.055208 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44313,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":29759,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":18010,"thread_start_us":198559,"threads_started":2,"wal-append.queue_time_us":78442}
I20260709 13:12:47.055321 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.069192 22841 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:47.036789 25993 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:47.079779 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:47.080693 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.080597 22832 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:47.086654 25999 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:47.087441 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.024845 26163 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.054714 26183 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:47.090989 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55388.2us
I20260709 13:12:47.075106 26043 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.103415 22912 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:47.104048 22910 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:12:47.105194 22732 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.159s (SENT)
I20260709 13:12:47.106554 22837 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:47.114581 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.122840 26176 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:47.123375 26176 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:47.124297 26176 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.102228 25993 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:47.127737 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.132874 25983 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:47.140740 26202 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.492s	user 0.000s	sys 0.002s
I20260709 13:12:47.148485 25897 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.152383 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.153231 22760 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:47.160418 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.162110 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.163048 22764 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:12:47.167272 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 70914.2us
I20260709 13:12:47.127632 26049 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:12:47.117208 26188 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.172076 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 449) took 1450 ms. Trace:
W20260709 13:12:47.173870 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72281.4us
I20260709 13:12:47.174718 25993 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:47.176059 25993 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:47.179549 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.180704 25917 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:47.130244 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:47.186740 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56584.5us
I20260709 13:12:47.187565 25917 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.189733 25955 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:47.190971 25955 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:47.190191 22662 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.191324 25955 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:47.192431 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:47.192688 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:47.193377 23461 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:47.126065 26183 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:47.197355 26193 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.537s	user 0.001s	sys 0.003s
I20260709 13:12:47.172644 22841 rpcz_store.cc:276] 0709 13:12:45.721684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.417959 (+696275us) service_pool.cc:224] Handling call
0709 13:12:46.476982 (+ 59023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.069853 (+592871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.171732 (+101879us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54003219d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:47.171992 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361352576}
I20260709 13:12:47.192186 22662 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:47.204877 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:47.205433 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:47.211069 22879 leader_election.cc:400] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
I20260709 13:12:47.211555 22879 leader_election.cc:403] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:12:47.212272 26182 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:47.214071 26213 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.081s	user 0.000s	sys 0.002s
I20260709 13:12:47.204020 26158 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.218096 26148 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:47.192277 25955 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:47.222321 22732 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.233s (SENT)
I20260709 13:12:47.222537 25955 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.222982 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:47.224287 22732 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.302s (SENT)
I20260709 13:12:47.226714 26182 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:47.204341 26176 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:47.229791 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126490us
I20260709 13:12:47.225164 26163 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.230669 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:47.232797 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.233440 26163 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.233995 26163 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.234404 26163 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.241307 26188 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:47.247006 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44274.9us
W20260709 13:12:47.250703 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.225258 22732 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.548s (SENT)
I20260709 13:12:47.252938 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:47.254604 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.255366 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54688.9us
W20260709 13:12:47.256386 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49520us
I20260709 13:12:47.249109 22805 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:47.254861 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:47.261229 22764 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:12:47.272583 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 249080us
I20260709 13:12:47.272433 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:47.273487 22912 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:47.266634 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:47.203868 25897 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:47.285557 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.261129 26049 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:47.288960 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:47.289687 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:47.265740 26043 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:47.302842 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77610.6us
I20260709 13:12:47.265282 26188 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:47.308358 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.265369 25707 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7.
I20260709 13:12:47.271781 26022 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:47.272140 25964 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:47.314560 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48463.3us
I20260709 13:12:47.281342 26153 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:47.324312 22732 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.713s (SENT)
W20260709 13:12:47.325959 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93218us
I20260709 13:12:47.272028 26168 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:12:47.316561 26148 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:47.355476 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.361917 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 151011us
I20260709 13:12:47.364339 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.366554 26009 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:47.367189 26009 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.367676 26009 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:47.383430 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:47.322198 26022 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:47.294341 25897 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:47.395566 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.395625 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:47.385771 22662 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 3 to 6, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:47.339504 26221 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:47.406002 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46661.8us
I20260709 13:12:47.318249 26043 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.358341 26148 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.410238 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.388654 26022 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:47.395164 26050 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:47.405124 26170 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.407969 26151 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:47.417022 26169 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.004s
W20260709 13:12:47.466329 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.433290 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92183.7us
I20260709 13:12:47.426266 25964 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:47.452965 22732 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.839s (SENT)
I20260709 13:12:47.474046 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:47.457824 25983 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:47.458707 23461 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:47.461525 26043 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:47.502936 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.413455 26148 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.408442 26221 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:47.470263 26223 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 13:12:47.470721 25687 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:47.472550 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:47.455976 26227 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:12:47.472846 26173 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
W20260709 13:12:47.452106 26171 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.001s
W20260709 13:12:47.493726 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.523548 22913 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:47.420055 26170 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:47.524259 26170 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:47.527130 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.418589 25313 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:47.529018 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.507372 26134 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:47.530493 22914 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:47.473580 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.397459 26153 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.506016 23462 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:47.508719 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.414601 25707 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:12:47.512957 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:47.518383 22907 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.508242 26221 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:47.532866 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 110716us
I20260709 13:12:47.492139 25550 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.536757 22912 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:47.539914 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.498549 22662 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 4 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:47.547612 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:47.546675 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74808.8us
W20260709 13:12:47.552625 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2081105629999911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.42169999999999996s for a response
W20260709 13:12:47.552712 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53607.3us
W20260709 13:12:47.552480 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 80024.7us
I20260709 13:12:47.543030 26043 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:47.555123 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.557073 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 105868us
I20260709 13:12:47.516974 26159 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:12:47.532186 26163 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.567870 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:47.565263 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.542176 26027 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:12:47.566959 22662 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 4, term changed from 4 to 5, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 5 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:47.555642 26171 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:47.543470 25550 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.560640 26149 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.591038 26009 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.593346 25707 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:47.596165 26158 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:47.563392 26134 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:47.596987 26199 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.610296 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.554286 26018 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:12:47.566486 26195 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:12:47.583762 26167 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:47.583986 26050 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:47.603201 26149 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.631173 26173 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:12:47.632602 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70831.2us
W20260709 13:12:47.618220 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.584897 25897 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.579762 26168 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:47.636916 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:47.638073 22662 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:47.630342 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.640938 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 93434.3us
I20260709 13:12:47.652161 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:47.656490 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71093.8us
I20260709 13:12:47.636229 26050 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:47.636498 25313 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:47.643028 26149 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.656126 25564 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.650573 26134 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.686875 26134 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:47.687639 26134 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.665275 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.667645 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:47.644501 26168 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:47.642872 26173 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:47.694916 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.679251 26229 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:12:47.650233 26171 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:12:47.698207 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:47.738325 26171 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:47.739022 26163 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.694204 25564 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:47.752719 26228 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.275s	user 0.003s	sys 0.004s
I20260709 13:12:47.753590 25564 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.710384 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:47.754103 23462 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 10. Ops: []
I20260709 13:12:47.712694 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:47.755280 22841 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.713791 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:47.714140 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46650.1us
I20260709 13:12:47.714704 26237 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:47.716486 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:47.705787 26170 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.717420 23464 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:47.721767 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:47.722198 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.723999 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.729270 22913 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:47.732529 25313 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:47.734127 26173 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.734565 26153 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:47.735221 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101987us
W20260709 13:12:47.735658 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78843.2us
I20260709 13:12:47.670001 26050 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.674120 26167 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:47.736083 26168 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.737334 25550 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.738401 23457 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:47.748196 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.750433 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.773641 22765 raft_consensus.cc:2368] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9.
I20260709 13:12:47.774498 26237 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:47.775851 26237 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.777786 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.770568 25313 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.778807 25999 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:47.781015 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.782707 26236 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:47.783385 26173 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.784955 22806 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:47.786607 26151 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:47.788229 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:47.783624 25313 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.763578 23463 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:47.789713 26018 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.789856 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.790166 26018 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:47.790577 26018 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.790701 25313 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.791278 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.792575 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.792829 25313 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.792515 26148 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.793463 26167 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:47.729318 26022 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.753335 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:47.795734 26022 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.802023 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:47.804157 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.698303 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:47.814097 22762 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:47.814929 22907 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:47.767087 26163 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.818076 22841 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:47.820184 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:47.823046 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.824245 22761 raft_consensus.cc:2410] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:12:47.828486 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.831161 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:47.845593 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.846160 22841 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:47.846751 22759 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:47.847980 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:47.848740 22765 raft_consensus.cc:2376] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6. Re-sending same reply.
I20260709 13:12:47.832036 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:47.853555 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.854425 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:47.855733 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.795241 26050 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.859130 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.859905 22761 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:47.859640 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:47.856547 22760 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:47.858933 25313 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:47.862653 26167 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:47.863452 26220 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:47.865797 25993 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:47.866734 25313 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:47.867728 25313 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.870265 25550 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.870832 25550 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.873013 25550 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.873878 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:47.874104 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72181.8us
I20260709 13:12:47.875389 23459 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:12:47.858250 26049 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.004s
I20260709 13:12:47.825323 26188 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:12:47.827443 26236 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:12:47.873371 26043 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:47.910784 26188 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.911417 26188 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:47.834672 26163 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.900941 26050 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.902463 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:47.917841 23457 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.920926 26168 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.922474 26018 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.889818 26049 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:47.905893 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:47.909540 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.899410 26148 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:47.929770 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.930758 26148 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:12:47.931607 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.931857 25564 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:47.932278 25564 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:47.932890 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:47.899565 26170 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.913946 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:47.923907 26182 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:48.004012 26182 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:48.004997 26182 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.006119 26182 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.924731 26163 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.929231 26153 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:47.933283 25564 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.010941 25564 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:47.912395 26169 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:48.013706 26169 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:47.904547 26221 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.015142 26169 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.017099 26169 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:47.931883 26148 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:12:48.017570 26221 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:48.018229 26148 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:47.903987 26134 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:47.952677 26188 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.964831 26018 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:47.965885 26049 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.073477 26049 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.075858 26049 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.967730 26039 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:47.967468 26149 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.076537 26039 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:48.077913 26039 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.078435 26149 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:47.968905 26009 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:48.079367 26039 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.974414 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:48.082281 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 108019us
I20260709 13:12:48.083608 23431 leader_election.cc:400] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9.
I20260709 13:12:48.084676 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:48.087052 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:48.087559 26009 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.087939 25897 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.088689 25897 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:48.089627 23431 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:47.978021 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 197129us
I20260709 13:12:48.090711 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:48.091104 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 178105us
I20260709 13:12:47.980346 26166 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:48.092161 26166 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.092589 26166 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:47.981585 22757 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:47.981712 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.097070 22759 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:47.982739 25993 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:12:47.982928 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.983529 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.099164 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:48.099807 22766 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:47.984302 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:47.986878 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:47.987089 26220 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:47.988327 26248 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.989861 23464 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:12:47.991573 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.991550 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:47.993287 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:47.994634 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:47.995312 22763 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:48.001396 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.001153 25964 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:48.018530 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:48.019862 22732 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.943s (SENT)
I20260709 13:12:48.004014 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:47.911849 22662 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 5 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:48.026607 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:48.062567 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.063526 26134 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:48.064556 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:48.065090 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.238256 22762 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:48.065529 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:48.239212 22833 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:48.069113 22914 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:48.070273 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:48.089810 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 81248.2us
W20260709 13:12:48.240676 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 307875us
I20260709 13:12:48.241334 22907 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:47.979853 25707 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:47.977713 26167 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.101408 22761 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:48.162271 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.227046 26248 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.251339 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.252090 22766 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:48.131594 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.098348 25993 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:48.229990 25564 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:48.102017 26251 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.234349 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.235922 22765 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:48.018639 26148 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:48.242682 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 241388us
I20260709 13:12:48.076407 23457 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:48.244261 23462 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:48.244858 22764 raft_consensus.cc:2410] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:12:48.244917 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:48.245875 25550 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.245987 23460 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:48.247339 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.281337 22761 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:48.282511 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:48.275161 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:48.249758 22841 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:48.284008 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:48.284483 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 264651us
I20260709 13:12:48.257623 26018 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:48.258080 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.259016 26248 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:48.289790 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:48.259461 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:48.290498 22839 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:48.260221 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.238873 26009 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:48.296870 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:48.248680 26134 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.289115 22840 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:48.240706 26237 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:48.291270 22760 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:48.284009 22759 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
W20260709 13:12:48.299873 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:48.299710 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 601520us
I20260709 13:12:48.253050 26163 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:48.300244 22841 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:12:48.300282 22732 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.110s (SENT)
I20260709 13:12:48.301807 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:48.303033 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 206774us
I20260709 13:12:48.263935 22758 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:48.315457 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:48.318534 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:48.319205 23459 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:48.322139 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:48.315757 26237 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.324998 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:48.284801 26169 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:48.295806 25564 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:48.323364 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:48.327194 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:48.291355 26182 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.287813 25707 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:48.328866 26237 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:48.358450 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58207.6us
I20260709 13:12:48.330070 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1532) took 1179 ms. Trace:
I20260709 13:12:48.359486 22757 rpcz_store.cc:276] 0709 13:12:47.150353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.158499 (+  8146us) service_pool.cc:224] Handling call
0709 13:12:47.158839 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:48.096379 (+937540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.329793 (+233414us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:48.330006 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891458432}
I20260709 13:12:48.330850 25917 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:48.333977 22663 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 264) took 21708 ms (client timeout 15000 ms). Trace:
I20260709 13:12:48.305270 26151 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:48.334408 26255 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:12:48.292484 26168 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:48.294652 26160 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.293531 26009 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:48.384295 22732 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.338s (SENT)
I20260709 13:12:48.387979 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.362511 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:48.339175 25955 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.339927 22662 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 2 to 6, now has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:12:48.289664 25550 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.294111 26018 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.285128 26220 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.325793 22907 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:48.363379 22909 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
W20260709 13:12:48.394160 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 177205us
W20260709 13:12:48.402691 26253 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.238s	user 0.000s	sys 0.002s
I20260709 13:12:48.316177 26049 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:12:48.363945 25917 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:48.400795 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.403790 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:48.394827 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74506.4us
W20260709 13:12:48.436141 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.436039 22912 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:12:48.437160 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:48.409735 22758 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:48.389135 26009 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.366415 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:48.432005 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.442323 22757 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:48.445847 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:48.455744 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:48.459214 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.455729 22907 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:48.475977 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:48.476990 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.428789 25897 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:48.478475 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49428.9us
I20260709 13:12:48.365906 25707 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.506319 22839 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:48.365851 26169 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.392123 26168 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:48.481698 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52742.7us
I20260709 13:12:48.430006 25313 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:48.390710 26236 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:12:48.484334 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:48.533313 22841 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:48.485005 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:48.485944 26257 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:12:48.481001 26160 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:48.486567 25897 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:48.486773 26220 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:48.383246 26151 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.488193 26049 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:48.447359 25917 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.488955 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:48.515610 26163 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:48.478557 26250 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:48.485280 26259 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.595882 26259 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.490195 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:48.364090 22663 rpcz_store.cc:269] 0709 13:12:26.625146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.713683 (+ 88537us) service_pool.cc:224] Handling call
0709 13:12:48.333674 (+21619991us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:44.210234 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:44.262499 (+ 52265us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:44.277520 (+ 15021us) write_op.cc:435] Acquired schema lock
0709 13:12:44.296204 (+ 18684us) tablet.cc:662] Decoding operations
0709 13:12:44.485486 (+189282us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:44.485608 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:44.485642 (+    34us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:12:44.487127 (+  1485us) tablet.cc:701] Row locks acquired
0709 13:12:44.487182 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:12:44.487453 (+   271us) write_op.cc:270] Start()
0709 13:12:44.487620 (+   167us) write_op.cc:276] Timestamp: P: 1783602764487371 usec, L: 0
0709 13:12:44.487649 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:12:44.488550 (+   901us) log.cc:844] Serialized 5133 byte log entry
0709 13:12:45.029247 (+540697us) op_driver.cc:464] REPLICATION: finished
0709 13:12:45.210844 (+181597us) write_op.cc:301] APPLY: starting
0709 13:12:45.211030 (+   186us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:45.211487 (+   457us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:45.211628 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:45.225227 (+ 13599us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:45.225837 (+   610us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:12:45.225910 (+    73us) write_op.cc:312] APPLY: finished
0709 13:12:45.226580 (+   670us) log.cc:844] Serialized 95 byte log entry
0709 13:12:45.234402 (+  7822us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:45.234659 (+   257us) write_op.cc:454] Released schema lock
0709 13:12:45.234770 (+   111us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:45.262509 (+ 27739us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:45.262556 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5832279,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":83321,"apply.run_cpu_time_us":6160,"apply.run_wall_time_us":51666,"num_ops":13,"prepare.queue_time_us":254007,"prepare.run_cpu_time_us":41092,"prepare.run_wall_time_us":361105,"raft.queue_time_us":214828,"raft.run_cpu_time_us":19357,"raft.run_wall_time_us":176945,"replication_time_us":521919,"thread_start_us":422863,"threads_started":4,"wal-append.queue_time_us":77714,"wal-append.run_cpu_time_us":40738,"wal-append.run_wall_time_us":1663349}]]}
I20260709 13:12:48.597927 22907 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:48.473925 26022 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:12:48.599114 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:48.599644 26259 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:48.599844 26022 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:48.600492 26022 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:48.494452 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:48.354416 25993 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.546901 26160 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.499786 22763 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:48.501251 22766 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:48.489486 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.365641 26182 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.480209 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:48.524964 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 140702us
I20260709 13:12:48.534375 22838 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:48.534825 22833 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:48.542584 22837 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:12:48.543237 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:48.545785 22840 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:48.529348 25983 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 13:12:48.554778 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.618535 26160 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.482825 26262 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:48.593410 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104549us
I20260709 13:12:48.594039 26163 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.594882 25707 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:48.498555 22909 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:48.538697 26251 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
W20260709 13:12:48.623333 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 80235.7us
I20260709 13:12:48.640206 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:48.607846 26049 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:48.548738 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 222112us
W20260709 13:12:48.622716 22732 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.805s (SENT)
I20260709 13:12:48.540225 25313 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.631210 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.641849 25897 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.645695 26259 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:48.645300 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:48.646319 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.646797 26043 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:48.646347 26260 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:48.653324 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.687937 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:48.655797 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 332474us
I20260709 13:12:48.656438 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.660749 25955 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.678067 26049 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.683583 25313 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:48.707896 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51575.4us
I20260709 13:12:48.708683 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:12:48.709579 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64357.3us
I20260709 13:12:48.710299 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:48.716696 22805 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:48.717252 22805 leader_election.cc:403] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:12:48.539774 25564 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.614523 26252 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.360s
W20260709 13:12:48.727898 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 370686us
I20260709 13:12:48.661082 25707 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:48.663308 26250 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.733366 22761 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:48.706213 26043 consensus_queue.cc:1243] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:48.703549 26220 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:48.753021 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 258305us
I20260709 13:12:48.705121 25550 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:48.723601 26156 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:48.726709 25313 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:48.734030 26262 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.729216 26265 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:48.779947 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:48.782161 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:48.757108 26220 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.780920 22762 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:48.768605 26156 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:48.759125 26264 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:48.784981 26166 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.005s
I20260709 13:12:48.771474 26018 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:48.814837 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:48.812549 23459 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:48.787129 26265 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:48.788043 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:48.754932 26250 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.802534 26220 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.786301 25955 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:12:48.840232 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:48.818322 26256 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:12:48.780941 25550 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:12:48.813387 26251 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.818194 26166 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.764494 26022 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.814780 26156 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.812041 26259 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:48.815580 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 99067.6us
I20260709 13:12:48.816952 26260 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:48.815831 26264 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:48.819886 26153 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:48.851855 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.786490 25313 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.846930 23460 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:12:48.853438 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 98614.6us
I20260709 13:12:48.816689 26168 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:48.803879 26237 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.850556 26151 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:48.878191 26271 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:12:48.841439 26265 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:48.864243 26273 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:12:48.888758 22911 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:48.853171 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126413us
I20260709 13:12:48.863799 26251 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.851580 26270 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:48.894491 22805 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:48.871562 26182 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:48.853179 26262 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:48.899163 26261 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:12:48.877173 26264 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:48.955946 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61541.2us
I20260709 13:12:48.958179 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:48.960009 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:48.960707 22838 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:48.961369 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:48.962880 26260 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:48.963292 26260 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.963614 26260 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:48.966116 23460 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:48.970055 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.910543 25550 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:48.981650 26281 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:12:48.982237 25550 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:48.982093 25707 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:48.983426 26281 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:48.983376 25550 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:48.984825 25550 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:48.978889 26279 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.985816 26279 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:48.902212 26166 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:48.981586 26262 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:48.980162 26291 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:12:48.984009 26281 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:48.905802 25955 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:48.988875 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.033164 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.033409 22840 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:49.034179 22839 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:48.903355 26237 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:48.958043 26270 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:48.865813 26263 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:12:48.899045 25313 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.048744 23459 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:49.015290 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.053082 22914 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:12:49.053591 22914 pending_rounds.cc:85] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:12:49.053872 22914 pending_rounds.cc:107] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:12:49.054175 22914 raft_consensus.cc:2889] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:49.055040 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.055835 22907 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
W20260709 13:12:49.057490 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 347238us
I20260709 13:12:49.019845 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:49.035748 22833 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:12:49.038847 22841 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:49.044418 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.059860 22909 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:12:49.045449 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 422769us
W20260709 13:12:49.061973 22732 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 3.353s (SENT)
I20260709 13:12:48.988234 26279 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:48.975028 26151 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.976199 26182 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:49.068647 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.032397 26050 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.075083 22839 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:48.893855 26160 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:49.090533 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.076237 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.077548 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.073274 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:49.079187 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.092351 22913 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:49.092788 22915 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7. Re-sending same reply.
I20260709 13:12:49.080222 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.081800 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.095088 22912 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:12:49.082530 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.084830 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.087679 23457 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:49.088058 22765 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.022s
W20260709 13:12:49.098618 22765 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.006s	sys 0.022s
I20260709 13:12:49.104238 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:49.108672 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.109440 22915 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6. Re-sending same reply.
I20260709 13:12:49.110600 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.087826 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:49.071979 25955 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:49.115052 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.113615 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.124974 22913 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
W20260709 13:12:49.120286 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 162123us
I20260709 13:12:49.119478 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.046451 25917 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.145067 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:49.147111 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 88641.3us
I20260709 13:12:49.148175 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:49.153985 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.154268 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:49.156682 26009 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 13:12:49.157191 26009 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.159447 26009 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:12:49.159546 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.160210 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:49.161087 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.162075 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.162235 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.162739 22914 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6. Re-sending same reply.
I20260709 13:12:49.164079 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.165830 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.167436 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.168228 22913 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:49.160267 22912 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:12:49.169910 26274 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:12:49.170068 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.170467 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:49.171259 26292 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:12:49.171464 22912 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:12:49.129451 25955 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:49.171694 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.172261 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.173355 22912 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:49.174151 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.175209 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:49.166049 26258 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:12:49.181269 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2331) took 1425 ms. Trace:
I20260709 13:12:49.181659 22906 rpcz_store.cc:276] 0709 13:12:47.755718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.768885 (+ 13167us) service_pool.cc:224] Handling call
0709 13:12:47.782800 (+ 13915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:48.085707 (+302907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:48.085739 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:48.086485 (+   746us) log.cc:844] Serialized 26 byte log entry
0709 13:12:48.346896 (+260411us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:48.346957 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.347071 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:49.181042 (+833971us) raft_consensus.cc:1685] finished
0709 13:12:49.181064 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:49.181208 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195299,"raft.queue_time_us":199065,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":366774,"threads_started":2,"wal-append.queue_time_us":415277}
I20260709 13:12:49.183597 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.101604 26182 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.179196 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.179247 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:49.188699 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.189181 22911 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7. Re-sending same reply.
I20260709 13:12:49.189282 22763 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.169287 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.190109 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.193166 22908 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:49.201401 22833 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:12:49.206576 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.207480 22911 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:49.206480 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.209982 22908 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:49.211678 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.212881 22913 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:49.215590 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:49.221516 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:49.097311 26050 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.196504 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.234164 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.234970 22838 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:49.236390 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.237761 26284 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:49.238279 26284 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.238706 26284 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:49.230712 22913 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:49.231127 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.098932 25897 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:12:49.243319 25897 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.247817 26289 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:49.256419 26170 raft_consensus.cc:2764] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:49.257866 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.260545 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:49.258560 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:49.232203 26273 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:49.263690 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.264032 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.264146 26273 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.264750 26273 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.151535 26281 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.267851 26148 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.269785 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.271018 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.271696 22912 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:49.272783 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.274916 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:49.265157 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:49.277086 23461 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:49.277709 22766 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:12:49.280049 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 131991us
I20260709 13:12:49.280652 22914 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:49.096788 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:49.285449 22910 raft_consensus.cc:606] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.281024 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.233294 26163 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:49.293726 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.298270 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:49.299408 22906 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:49.301954 22908 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:49.303668 22906 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:12:49.353681 22906 pending_rounds.cc:85] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 4
I20260709 13:12:49.356494 22906 pending_rounds.cc:107] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:12:49.358428 22906 raft_consensus.cc:2889] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:49.358515 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77635.9us
I20260709 13:12:49.402238 22908 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:49.403918 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:49.409624 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.409844 22836 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:49.404266 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.287015 26166 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:49.310395 26290 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:49.433096 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 134901us
I20260709 13:12:49.294155 26163 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:49.348275 26279 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.403452 26022 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.441241 22908 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:49.446837 22837 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:49.298975 26250 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.447088 22832 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:49.348403 25964 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:12:49.150051 26262 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.456864 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.424743 26166 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.477020 22840 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:49.476867 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.304091 26291 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.143s
W20260709 13:12:49.487180 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.440793 26163 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:49.497838 22763 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
W20260709 13:12:49.503144 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.507061 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63942.7us
I20260709 13:12:49.508018 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.505486 22763 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:49.261812 26270 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.446722 26236 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.510165 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:49.520732 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111202us
I20260709 13:12:49.521211 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.521870 22760 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:49.451550 26251 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.485355 26166 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.445020 26279 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:49.529722 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.540174 22838 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:49.426302 26290 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.521445 26277 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.523139 26160 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.468647 26260 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.554157 26301 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:49.555354 26286 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:49.555826 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:49.556336 26313 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:49.544049 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:49.556365 26310 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:12:49.526867 26169 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:49.589751 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.590432 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.569592 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.570664 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:49.579450 23464 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:12:49.580325 26299 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:12:49.580725 26312 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:49.581225 26314 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:12:49.583191 23461 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:49.538487 26265 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.633s	user 0.013s	sys 0.000s
I20260709 13:12:49.592657 22908 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:49.594478 22837 raft_consensus.cc:2368] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
I20260709 13:12:49.600222 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:49.590857 26259 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.005s
W20260709 13:12:49.662329 26264 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.003s
I20260709 13:12:49.658047 22912 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:49.668462 22836 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:49.622260 26050 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:49.650491 26300 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 13:12:49.652431 26252 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.649749 26319 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:49.641798 26167 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:49.646345 26316 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:49.650638 26260 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.665848 22833 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:49.673270 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:49.640015 26322 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.585271 25917 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:49.588510 26160 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.582811 26182 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.560698 26277 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.633446 26265 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.602210 26321 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
W20260709 13:12:49.718603 25550 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.005s
I20260709 13:12:49.690495 22766 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:49.571938 25964 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:49.622437 22840 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:49.621567 26312 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.681632 26327 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.662400 26284 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.715921 23466 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:49.716995 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 116862us
W20260709 13:12:49.720840 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 212929us
I20260709 13:12:49.707247 26259 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:49.722338 26323 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:49.690053 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 114704us
W20260709 13:12:49.738783 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.740183 22834 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:49.740809 22914 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:49.705230 26273 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.708479 26264 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.731022 26134 raft_consensus.cc:2764] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:49.713755 26252 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.731068 26319 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:49.728559 26160 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.762784 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:49.766119 26332 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:49.766185 26220 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.000s
I20260709 13:12:49.764453 26297 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:49.741057 26279 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.778357 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.778409 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:49.802615 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50260.4us
I20260709 13:12:49.723392 26322 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.744050 25964 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:49.807833 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.816615 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.773425 23460 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:49.794337 26330 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:49.781780 26169 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:49.794343 26283 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:49.825286 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:49.753717 26327 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.828842 26009 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:12:49.756181 26323 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.829306 26169 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.857173 26166 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.862875 26169 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:49.864419 26166 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.847978 26322 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.848572 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.849371 26273 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:49.857105 26323 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.794950 26332 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:49.851064 26329 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:12:49.874742 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.851759 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:49.852080 26312 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:49.876062 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.876308 26312 raft_consensus.cc:2764] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:12:49.879048 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.852389 26327 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.853088 26265 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.853173 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:49.853618 23461 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:12:49.878255 26338 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:49.854063 25917 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:49.880417 26332 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.880954 25917 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:49.882000 25917 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.885819 26322 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.887307 26160 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:49.887421 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.888370 26221 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:49.888931 26221 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:49.858448 26252 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.858712 26163 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.781780 26220 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.893838 26221 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.896313 26220 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:49.899451 26163 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:49.901033 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.860340 26156 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.968s	user 0.014s	sys 0.000s
I20260709 13:12:49.859890 26330 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.861908 26264 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:49.841711 26319 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:49.871526 23466 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:49.849798 26284 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:49.872306 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46408.3us
I20260709 13:12:49.937462 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.874567 26182 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.947849 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 391) took 1022 ms. Trace:
I20260709 13:12:49.948482 22912 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:49.948810 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:49.949414 22763 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.948482 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.950898 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.951467 22762 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:49.952317 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:49.954921 26339 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.147s	user 0.003s	sys 0.000s
I20260709 13:12:49.948271 22764 rpcz_store.cc:276] 0709 13:12:48.925124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.943926 (+ 18802us) service_pool.cc:224] Handling call
0709 13:12:48.944361 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.408678 (+464317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:49.408727 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:49.408785 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:49.408825 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.947614 (+538789us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b540033fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:49.947665 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:49.947790 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959458560}
I20260709 13:12:49.958279 22764 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:49.959133 26251 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:49.959569 26251 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:49.889659 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:49.912113 26330 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.906421 26156 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.814937 26297 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:49.971695 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.938941 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:49.939915 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.973064 22913 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:49.940623 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:49.872540 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56077.2us
I20260709 13:12:49.973785 22911 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:49.974072 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.941915 26281 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:49.943117 25964 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:49.980620 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.943933 25993 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.945812 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.946934 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.982468 22906 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:49.961460 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:49.983454 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:49.962178 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72606.1us
I20260709 13:12:49.965910 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:49.973070 23461 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:49.979388 22763 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:49.943562 26290 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.985234 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.985929 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.988695 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:49.989900 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:49.990558 22910 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:49.913468 26277 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:49.999347 23462 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:50.003473 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:50.000183 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:49.998817 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.006002 22911 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:12:50.006750 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:50.007987 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:49.960489 26251 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.009102 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:50.013260 22913 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:50.019232 22912 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:50.019372 22911 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:49.987136 26330 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:49.989164 26039 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.021565 22915 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:50.022233 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:49.989233 26250 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:49.989316 26332 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:49.983395 25313 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:12:50.032157 22879 leader_election.cc:400] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
I20260709 13:12:50.034811 22879 leader_election.cc:403] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:50.035082 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:49.983765 26220 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:12:50.006078 25964 raft_consensus.cc:2764] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:49.998476 26266 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:49.999400 26252 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.014340 26277 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.039536 22915 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:50.006304 26323 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:12:50.009621 26279 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:50.018787 26264 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:50.031186 26176 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.066105 22833 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:50.000386 26166 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:12:50.107372 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.112901 22832 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:50.069054 26326 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:50.070626 26267 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.071652 26335 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:12:50.072315 26334 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:50.051096 26341 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:12:50.049903 26252 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.125869 26252 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.127434 22911 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:12:50.005786 26327 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:12:50.130092 26327 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:50.036248 26322 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:50.006436 26290 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:12:50.131048 26322 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.130584 26327 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:50.131501 26322 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:50.132820 26327 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.134004 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:50.134595 26327 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.026800 26039 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:50.137213 26039 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.050184 26277 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.141111 26277 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.115861 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.056421 26265 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:50.121269 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.145426 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.122464 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:50.053534 26330 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:50.131608 26290 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:50.074961 26332 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:50.119827 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.143198 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:50.143812 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.122007 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.049448 26266 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.175233 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:50.176052 22915 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:50.177254 22915 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:50.113018 26176 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:50.152586 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 137457us
W20260709 13:12:50.150596 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 115629us
W20260709 13:12:50.183459 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.151625 22833 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:50.151908 22840 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:50.115272 26267 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.114889 26264 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:50.150454 26232 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.146577 22841 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:50.183317 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:50.184298 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:50.170845 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.184885 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 129350us
W20260709 13:12:50.196342 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.148991 26290 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:50.115487 26279 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.174502 26334 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:50.185496 26176 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.212327 26285 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:50.201845 26323 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.190881 26267 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.151031 26265 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.229727 26264 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.200328 26248 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:50.202636 26332 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.192875 26236 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:12:50.215862 26151 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:50.244810 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60589.7us
I20260709 13:12:50.248445 22837 raft_consensus.cc:2410] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:50.258030 22909 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:50.222972 26156 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.235709 26323 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.230297 26039 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.224571 26335 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:50.268532 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.199530 26232 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.273885 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:50.272941 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:50.276563 22915 raft_consensus.cc:606] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.244923 26151 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.234809 26330 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.244899 26290 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.294471 26279 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.243404 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:50.270494 26297 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:50.297189 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53876.1us
I20260709 13:12:50.244721 26332 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.260957 26163 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:12:50.295171 26290 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:50.281975 26151 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.242722 26248 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:50.284391 26347 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:12:50.324766 22879 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.311506 22841 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:12:50.313365 26345 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:12:50.266008 26334 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.327572 26326 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
W20260709 13:12:50.328480 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.299310 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:50.330395 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:50.283708 26356 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:50.313021 26039 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.300166 26236 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:50.300649 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:50.332551 26356 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:50.338354 26279 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:50.332155 25999 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.341235 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.342341 26266 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.315593 26332 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.353482 26326 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:50.361692 26326 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.362167 26326 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:50.351209 22911 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:12:50.353376 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52798.3us
W20260709 13:12:50.355178 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.314383 26232 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.362140 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 602) took 1069 ms. Trace:
I20260709 13:12:50.331662 25993 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:50.288290 26169 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.374396 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "776baf137f8745dd866379ca15ab8cee"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:50.374679 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.375252 22833 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:50.357591 26160 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.357326 25964 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:50.385689 22836 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:50.381810 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.387494 22840 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:50.384649 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 54358.2us
I20260709 13:12:50.389149 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.390816 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.383570 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.359575 26236 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:50.399962 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:50.400990 22832 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.397581 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.367764 25999 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.403131 22833 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 5.
I20260709 13:12:50.405315 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.374143 26354 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:50.408033 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:50.382419 26160 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.367162 23460 rpcz_store.cc:276] 0709 13:12:49.293116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.297433 (+  4317us) service_pool.cc:224] Handling call
0709 13:12:49.420610 (+123177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.831124 (+410514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.361416 (+530292us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b540063f954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:50.362001 (+   585us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774328192}
I20260709 13:12:50.410048 26355 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:50.397526 26265 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.397833 26167 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:12:50.400575 26252 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.446082 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.423207 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.417044 26176 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.426891 26248 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.427209 26250 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:50.374570 26151 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.427556 26335 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:50.429092 26169 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.419389 26356 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.432912 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:50.435731 26148 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:50.438454 26308 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.128s	user 0.004s	sys 0.000s
I20260709 13:12:50.439585 26355 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.463591 26176 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.458983 26248 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:50.464468 26355 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.464954 26248 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:50.383909 26266 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:12:50.466337 26266 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:50.465937 26248 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.466706 26266 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:50.467531 26266 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.422240 26160 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.421242 26354 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.384696 26322 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.416499 22915 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:50.447240 22833 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:50.447679 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:50.491012 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0246922020000966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
I20260709 13:12:50.460326 22913 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:50.480628 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:50.472092 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 479) took 1498 ms. Trace:
I20260709 13:12:50.475960 22907 raft_consensus.cc:606] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Received request to transfer leadership to TS 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.471043 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:50.481048 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:50.519024 22907 raft_consensus.cc:614] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:50.520105 23461 raft_consensus.cc:2368] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
I20260709 13:12:50.520539 22908 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:50.518901 22834 rpcz_store.cc:276] 0709 13:12:48.974120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.974295 (+   175us) service_pool.cc:224] Handling call
0709 13:12:48.974689 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.756567 (+781878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.471853 (+715286us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540019f0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:50.472030 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760013184}
I20260709 13:12:50.480458 23463 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:50.480967 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:50.526988 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47353.4us
I20260709 13:12:50.530010 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.530248 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.531008 22761 raft_consensus.cc:2368] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:12:50.531457 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.532155 22764 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:50.532117 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.533142 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.489355 26167 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:12:50.534348 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.534688 26188 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.535065 22764 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:50.483409 26323 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.482782 26252 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.535255 26188 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.486281 26199 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:50.491611 25313 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.536355 26199 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:50.536744 25313 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.537355 26199 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.538343 26270 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.538503 26199 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:50.535884 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:50.475308 26324 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.835s	user 0.004s	sys 0.000s
I20260709 13:12:50.530930 22762 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:50.543819 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.539669 22764 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:50.539996 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:50.540465 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.459318 22832 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:50.544968 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:50.545739 22761 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:50.546041 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.546947 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.468798 26266 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:50.548954 22762 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:50.554045 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.554613 22762 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:50.483292 23460 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:50.549933 22761 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:50.558432 22761 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:12:50.558750 22761 pending_rounds.cc:85] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:12:50.558964 22761 pending_rounds.cc:107] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:12:50.559211 22761 raft_consensus.cc:2889] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:50.559347 22764 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:50.550997 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:50.491999 26335 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.534286 26298 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:50.552345 26295 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.475430 26354 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.556860 22832 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:50.526109 25564 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.551963 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.539026 25313 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.487191 26220 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:12:50.474555 26151 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.562148 26188 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.489755 26269 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:12:50.569890 22764 raft_consensus.cc:2955] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:12:50.576261 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.538928 26270 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:50.586644 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.591636 22841 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:50.600559 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.578349 26283 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:50.485013 26236 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.618973 26283 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.619654 26283 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:50.616992 22879 leader_election.cc:400] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
I20260709 13:12:50.627811 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:50.631728 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.632957 22761 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:50.569598 26267 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.638136 26267 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.596441 26327 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.645701 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.639744 26286 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:50.648490 26277 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.650715 26277 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.655409 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.656734 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.656960 26251 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:50.567070 26335 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.578125 25564 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.580693 26326 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.632134 26323 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:50.689682 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 43973.4us
I20260709 13:12:50.697960 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.699079 22766 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:50.663467 26327 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.674288 26169 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:50.600925 26295 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.713766 22910 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:50.643266 26354 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:12:50.720244 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.694190 26250 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:50.683986 26252 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:12:50.736196 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:50.737047 22910 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:50.707212 26327 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.739830 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:50.744156 26365 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:12:50.744550 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:50.746120 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:50.747229 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:50.747737 22910 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:50.749992 26360 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:12:50.578886 26298 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.751679 26307 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:50.752079 26307 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:50.756975 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:50.768998 26363 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:12:50.770640 26362 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:12:50.771696 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 599) took 2046 ms. Trace:
I20260709 13:12:50.772104 23462 rpcz_store.cc:276] 0709 13:12:48.725013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.725140 (+   127us) service_pool.cc:224] Handling call
0709 13:12:48.738311 (+ 13171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:50.005696 (+1267385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.771410 (+765714us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54005253d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:50.771628 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2899699072}
I20260709 13:12:50.774696 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1543) took 2275 ms. Trace:
I20260709 13:12:50.761581 26220 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:50.780316 26338 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:50.780525 26270 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.786183 22762 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:50.787675 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.788848 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:50.775249 22759 rpcz_store.cc:276] 0709 13:12:48.499723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.520269 (+ 20546us) service_pool.cc:224] Handling call
0709 13:12:48.577725 (+ 57456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.578055 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:48.766842 (+188787us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.695312 (+1928470us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b180033cdc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:12:50.695608 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.773785 (+ 78177us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.774049 (+   264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:50.774127 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:50.774197 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.774313 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:50.774593 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117892,"prepare.queue_time_us":2070208,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":7659,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":8532,"spinlock_wait_cycles":27431168,"thread_start_us":312618,"threads_started":2,"wal-append.queue_time_us":327337}
I20260709 13:12:50.789692 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:50.803913 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.808763 23462 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:50.802322 26153 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:12:50.806291 26148 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:12:50.872094 23461 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:50.769054 25897 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.814056 26335 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.814304 26301 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:50.820945 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:50.823457 26265 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:12:50.824476 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1484) took 1666 ms (client timeout 1628 ms). Trace:
I20260709 13:12:50.825562 26259 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:50.826649 26343 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:50.827030 26326 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.827040 26151 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.792547 26322 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.828646 26295 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.830698 25564 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.788910 26338 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.817714 26264 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:12:50.836339 26160 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:50.838078 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:50.839013 26290 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:12:50.838927 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:50.909082 22835 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:50.909288 23463 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
W20260709 13:12:50.847361 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.628s (SENT) [suppressed 43 similar messages]
I20260709 13:12:50.851187 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1540) took 2352 ms. Trace:
I20260709 13:12:50.852952 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1542) took 2353 ms. Trace:
W20260709 13:12:50.912519 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.821756 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64950.4us
I20260709 13:12:50.855728 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.856997 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1539) took 2358 ms. Trace:
I20260709 13:12:50.862844 22766 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:50.862623 22910 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:50.876128 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:50.881728 22909 rpcz_store.cc:269] 0709 13:12:49.158530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.161984 (+  3454us) service_pool.cc:224] Handling call
0709 13:12:50.824290 (+1662306us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000fc154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:50.824405 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2452954368}
I20260709 13:12:50.883325 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:50.885324 26266 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:50.852167 26332 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:50.886335 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 65452.3us
I20260709 13:12:50.911305 22758 rpcz_store.cc:276] 0709 13:12:48.498690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.519431 (+ 20741us) service_pool.cc:224] Handling call
0709 13:12:48.519789 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:48.520033 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:48.737907 (+217874us) log.cc:844] Serialized 14 byte log entry
0709 13:12:49.077885 (+339978us) log.cc:844] Serialized 14 byte log entry
0709 13:12:49.158110 (+ 80225us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:50.713662 (+1555552us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.850396 (+136734us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:50.850473 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:50.850557 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.850765 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:50.851097 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1668865,"prepare.queue_time_us":1960728,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":108,"raft.queue_time_us":106579,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":5665,"spinlock_wait_cycles":6508928,"thread_start_us":646520,"threads_started":4,"wal-append.queue_time_us":550835,"wal-append.run_cpu_time_us":10221,"wal-append.run_wall_time_us":1028841}
I20260709 13:12:50.916257 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.888046 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:50.854730 26262 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:50.891614 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:50.895063 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.926736 22836 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:50.883884 26301 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:50.901355 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:50.879029 23460 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 5.
I20260709 13:12:50.895833 26163 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:12:50.910748 26160 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:50.911388 22765 rpcz_store.cc:276] 0709 13:12:48.499434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.500193 (+   759us) service_pool.cc:224] Handling call
0709 13:12:48.500551 (+   358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:48.501043 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:48.501072 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:48.501597 (+   525us) log.cc:844] Serialized 52 byte log entry
0709 13:12:49.099050 (+597453us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:50.737008 (+1637958us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.852533 (+115525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.852664 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:50.852716 (+    52us) raft_consensus.cc:1685] finished
0709 13:12:50.852728 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:50.852885 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1622865,"prepare.queue_time_us":1651318,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":101335,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":30562,"thread_start_us":658603,"threads_started":2,"wal-append.queue_time_us":743138,"wal-append.run_cpu_time_us":3443,"wal-append.run_wall_time_us":1374917}
I20260709 13:12:50.914527 22763 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:50.891542 26265 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.922720 22832 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.923467 22766 pending_rounds.cc:85] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 7
I20260709 13:12:50.884364 26307 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.917368 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:50.914732 22757 rpcz_store.cc:276] 0709 13:12:48.498362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.499141 (+   779us) service_pool.cc:224] Handling call
0709 13:12:48.499608 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:48.500013 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:48.500045 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:48.500708 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 13:12:48.944924 (+444216us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:49.163922 (+218998us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:50.713191 (+1549269us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.821399 (+108208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.821600 (+   201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:50.821647 (+    47us) raft_consensus.cc:1685] finished
0709 13:12:50.856732 (+ 35085us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:50.856936 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1753780,"prepare.queue_time_us":1805075,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":102025,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":26360,"spinlock_wait_cycles":23109504,"thread_start_us":550116,"threads_started":2,"wal-append.queue_time_us":739238,"wal-append.run_cpu_time_us":6050,"wal-append.run_wall_time_us":1028403}
I20260709 13:12:50.930222 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:50.886294 25897 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.922335 22758 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:50.946943 22765 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:50.948081 22758 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:50.894434 26259 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:50.913785 25564 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:50.940505 22766 pending_rounds.cc:107] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:12:50.961828 22766 raft_consensus.cc:2889] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:50.924724 22910 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:50.949569 22758 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:50.922521 26311 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:12:50.960459 22832 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:50.956065 22763 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:50.907545 26151 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:50.939013 26149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.529s	user 0.006s	sys 0.016s
I20260709 13:12:50.922696 26322 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:12:50.939486 26298 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:50.973093 23466 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:50.978010 22841 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:50.986454 26151 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:50.961558 26160 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:50.936640 26237 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:12:50.934291 26262 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:50.980515 26149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.571s	user 0.006s	sys 0.017s
I20260709 13:12:50.974433 26163 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:12:50.995833 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104345us
I20260709 13:12:50.978946 26307 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:51.003619 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 156302us
W20260709 13:12:51.004937 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 82807.4us
I20260709 13:12:50.993458 26160 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:50.968223 25564 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.012812 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1498) took 1810 ms. Trace:
I20260709 13:12:51.002529 26163 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:51.015005 22914 rpcz_store.cc:276] 0709 13:12:49.202046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.286170 (+ 84124us) service_pool.cc:224] Handling call
0709 13:12:49.312109 (+ 25939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.753671 (+441562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.012468 (+1258797us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54001d2254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:51.012695 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985033984}
I20260709 13:12:51.012941 26160 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:51.021585 26266 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:51.032912 26265 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.032356 26383 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:12:51.047880 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2357) took 1006 ms. Trace:
I20260709 13:12:51.048588 22911 rpcz_store.cc:276] 0709 13:12:50.041801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.133730 (+ 91929us) service_pool.cc:224] Handling call
0709 13:12:51.047654 (+913924us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400041854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:51.047790 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447851776}
I20260709 13:12:51.054932 26259 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:51.067238 26373 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:12:51.070668 26266 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.078727 25564 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.081820 26163 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.084267 26163 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:51.090631 26370 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.341s	user 0.004s	sys 0.000s
I20260709 13:12:51.096724 26263 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.097383 23463 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:51.097311 26263 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.099602 26263 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:51.100477 23460 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:51.101742 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.102967 26283 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.104075 22909 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:51.106473 26283 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:51.108234 26369 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
W20260709 13:12:51.109710 26199 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.571s	user 0.015s	sys 0.002s
W20260709 13:12:51.116899 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.117908 26386 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 13:12:51.118382 26039 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:51.118793 26039 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:51.119247 26301 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.119658 26039 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.120752 26039 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.121956 26385 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:51.122262 26383 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:51.122799 26383 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:51.123886 26383 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.124785 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:51.121760 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 90442.1us
I20260709 13:12:51.125401 22915 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.126305 22805 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:12:51.126746 22805 leader_election.cc:403] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:51.127486 26259 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.112851 26327 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:51.133322 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.133925 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.134299 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.134414 25707 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:51.134893 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.135035 25707 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.134191 26251 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:51.138484 26251 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.147899 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.149740 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.150641 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.142303 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:51.151249 22763 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:51.152443 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.153223 22911 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:51.153458 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.153751 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.158977 22806 leader_election.cc:400] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:12:51.159348 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.160200 22806 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:51.160742 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.161118 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.162698 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:51.166570 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.189010 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.141486 26251 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:51.182152 22840 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:51.137208 25707 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.186683 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.188620 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.192060 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:51.167656 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.192250 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.202116 22765 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:51.193854 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.203727 22910 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:51.206621 22910 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:51.137243 26327 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:51.201123 23462 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:12:51.199932 23464 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:51.188280 26265 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:51.227456 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:51.232074 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.234721 22914 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:51.236675 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.238513 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.239161 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.241668 22763 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:51.243197 23461 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:51.242877 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.246578 22764 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:51.245289 22761 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:51.249532 26385 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:51.250010 26385 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:51.250423 26385 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.258782 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.262233 22915 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:51.195974 26160 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.127519 25955 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.330066 22764 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:51.332894 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.279758 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.309988 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.312474 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.320971 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.233904 26265 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.338361 23463 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:51.345752 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 118458us
I20260709 13:12:51.310303 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.244565 26384 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:51.350694 22833 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:51.352191 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:51.352873 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.325495 26301 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.343765 26148 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:12:51.348851 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 186256us
W20260709 13:12:51.263235 26289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.574s	user 0.036s	sys 0.043s
I20260709 13:12:51.354094 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.345883 26265 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:51.378794 26384 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:51.380633 26384 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:51.381009 26384 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.263396 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.340937 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.359469 22911 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.275416 22840 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:51.359191 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.367889 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.369379 26323 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:51.386055 26323 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:51.387090 26323 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.369833 26391 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.389554 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.389853 26391 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.373136 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.391392 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.391952 22912 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.393057 26391 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.395576 26301 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:51.377873 26289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.690s	user 0.036s	sys 0.044s
I20260709 13:12:51.401876 26259 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:12:51.402685 22832 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
W20260709 13:12:51.402920 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43849.5us
I20260709 13:12:51.403820 22806 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8.
I20260709 13:12:51.404714 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:51.406020 26259 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:51.406474 26259 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:51.407141 26259 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:51.375713 26388 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:12:51.411818 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.412439 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.412664 22906 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:51.413340 26266 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:51.413964 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.414950 22906 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:51.414990 26266 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:51.416442 22906 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:51.419020 22907 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:51.377250 26356 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.423975 26338 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.427896 25897 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:51.430230 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.436268 22834 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:12:51.438469 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.444461 26153 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:51.451390 22838 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:51.451424 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.452607 26153 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:51.442837 26391 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:12:51.445475 22907 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:51.462831 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.442181 22911 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:12:51.440580 26160 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.458770 22839 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:51.453281 26301 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:51.447068 22912 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:12:51.472295 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.443209 26266 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:12:51.490777 22911 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:51.495335 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:51.498246 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.500181 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2355) took 1459 ms. Trace:
W20260709 13:12:51.500996 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.500579 22908 rpcz_store.cc:276] 0709 13:12:50.040922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.041085 (+   163us) service_pool.cc:224] Handling call
0709 13:12:50.041472 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:50.520903 (+479431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.499911 (+979008us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:51.500105 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1101043072}
I20260709 13:12:51.507298 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:51.512754 22908 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:12:51.500097 26393 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:51.496625 26395 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:51.516439 26160 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:51.519064 26160 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:51.520049 26160 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:51.529040 26367 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.777s	user 0.002s	sys 0.000s
I20260709 13:12:51.530622 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2363) took 1111 ms. Trace:
I20260709 13:12:51.512913 26285 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:12:51.531051 22913 rpcz_store.cc:276] 0709 13:12:50.419439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.525159 (+105720us) service_pool.cc:224] Handling call
0709 13:12:50.525589 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:50.526079 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:50.526109 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:50.542096 (+ 15987us) log.cc:844] Serialized 26 byte log entry
0709 13:12:50.647303 (+105207us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:50.647429 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.647682 (+   253us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:51.528612 (+880930us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:51.530251 (+  1639us) raft_consensus.cc:1685] finished
0709 13:12:51.530273 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:51.530526 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13410,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6986,"spinlock_wait_cycles":72297856,"thread_start_us":75342,"threads_started":1,"wal-append.queue_time_us":143221}
I20260709 13:12:51.523623 26385 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:51.546248 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 161830us
I20260709 13:12:51.547706 26326 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:51.551345 26188 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:12:51.544623 26268 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.005s	sys 0.005s
W20260709 13:12:51.559496 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.570758 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:51.563983 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 633829us
I20260709 13:12:51.574977 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.588230 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:51.589099 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.589771 22837 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.589846 22840 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:51.591238 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.591859 22832 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.574467 22834 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:51.563037 26268 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.005s	sys 0.005s
I20260709 13:12:51.599087 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.600154 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.600742 22834 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.569489 26265 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.601645 26302 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.603829 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.605211 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.620844 22759 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:51.564179 26151 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:51.568691 26259 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.606017 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.622259 22762 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:12:51.623358 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.624084 22762 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.606738 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.625216 22763 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.626945 22832 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:51.628384 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.606834 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.629068 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.629884 22759 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 7: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:51.608814 22758 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:51.591253 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.613766 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.615887 22837 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:51.630981 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:51.631429 22913 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:51.565274 22912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:51.587291 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.629657 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:51.632907 22912 raft_consensus.cc:606] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.629004 22909 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8. Re-sending same reply.
I20260709 13:12:51.608245 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1566) took 1661 ms. Trace:
I20260709 13:12:51.633304 22912 raft_consensus.cc:614] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:51.634706 22760 rpcz_store.cc:276] 0709 13:12:49.946291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.946434 (+   143us) service_pool.cc:224] Handling call
0709 13:12:49.946762 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.946991 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:50.771765 (+824774us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.939850 (+168085us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.941646 (+  1796us) log.cc:844] Serialized 14 byte log entry
0709 13:12:51.425217 (+483571us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18002b5640. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:51.554257 (+129040us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b18002b5640. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:51.563012 (+  8755us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b18002b5640. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:51.607573 (+ 44561us) log.cc:844] Serialized 14 byte log entry
0709 13:12:51.607778 (+   205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:51.607828 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:51.607872 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.607935 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:51.608035 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1085248,"prepare.queue_time_us":900270,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":6235,"raft.queue_time_us":635750,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":16969,"spinlock_wait_cycles":48702080,"thread_start_us":167243,"threads_started":1,"wal-append.queue_time_us":205662}
I20260709 13:12:51.638756 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:51.641076 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.642896 22761 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:51.643549 22834 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:51.642042 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:51.563722 26335 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.645901 22908 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:51.648217 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61023.2us
I20260709 13:12:51.651708 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:51.652819 22841 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:51.652974 22763 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:51.653323 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:51.662273 22832 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:51.665344 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:51.562878 26326 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.562855 26188 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:51.617944 26343 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.617420 26049 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.607461 26328 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:12:51.670765 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:51.619570 26039 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:51.631042 26151 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:51.678788 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.667405 26322 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:51.674696 25707 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.694221 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:51.685552 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:51.664023 26169 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:51.666574 26270 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:51.680510 26402 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:12:51.677892 26332 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:51.668663 25993 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:12:51.562234 26384 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.699354 22762 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:51.645632 26134 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:12:51.644760 26302 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.690917 26049 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.710356 22757 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:51.684942 26311 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:51.568168 26153 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.713234 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:51.717131 25955 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:51.712232 26311 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:51.686035 26250 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:12:51.759123 26250 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:51.759502 26250 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:51.760403 26250 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.717782 26049 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.719652 26151 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.724898 26039 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:51.727744 25999 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.706435 26188 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.698058 26338 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:51.774946 26384 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.739686 22839 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:51.740618 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:51.703179 26332 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.702394 26394 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.685606 26343 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:51.804545 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.806370 26343 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.807128 26343 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:51.807734 26343 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.702977 26270 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.810281 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.812490 26270 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.701248 25707 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.814661 26385 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.816459 25707 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.738282 26302 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.758066 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.817296 26326 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.794827 26250 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.812975 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:51.821799 23462 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:51.822266 26265 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.766016 26199 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:51.823175 26199 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:51.824127 26199 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.817824 22906 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:51.711067 26322 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:51.771941 26039 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:51.827116 26039 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:51.827452 26322 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.772929 26259 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:51.766536 26290 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:51.775066 26188 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.830025 26290 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:51.831063 26290 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:51.739298 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45132.5us
I20260709 13:12:51.834512 22912 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:51.834972 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:51.835830 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1446) took 1419 ms. Trace:
I20260709 13:12:51.789070 26323 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.790026 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:51.843109 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53228.2us
I20260709 13:12:51.762264 26327 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:51.796782 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 56278.7us
I20260709 13:12:51.799995 26332 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.801862 26307 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:51.803697 26394 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:51.756971 26311 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:51.813473 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:51.836221 22833 rpcz_store.cc:276] 0709 13:12:50.413925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.528059 (+114134us) service_pool.cc:224] Handling call
0709 13:12:51.053646 (+525587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:51.351247 (+297601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.724570 (+373323us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54001bc454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:51.833326 (+108756us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54001bc450. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:51.833474 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839603840}
I20260709 13:12:51.843704 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:51.845383 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:51.881469 23462 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:51.849650 22837 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:12:51.882398 22837 pending_rounds.cc:85] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:12:51.882704 22837 pending_rounds.cc:107] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:12:51.882992 22837 raft_consensus.cc:2889] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:51.885313 23464 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:51.892064 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:51.892345 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:51.907039 23464 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:51.887646 22879 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:51.856333 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:51.911789 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.913727 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:51.917357 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 230123us
I20260709 13:12:51.946154 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:51.847306 26328 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:51.952795 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:51.855099 26148 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:51.958318 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.960216 23464 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:51.854637 26327 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:51.854418 26307 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:51.943190 25707 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:51.945688 26356 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:51.946233 26259 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:51.883761 26149 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.873291 26232 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:12:51.970765 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:51.972792 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:51.887966 26153 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.854957 26169 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:12:52.005775 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:51.986876 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 357302us
I20260709 13:12:51.868367 26394 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.873471 26298 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:12:51.970626 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:52.003506 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:52.004277 23464 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:51.959398 26148 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.983103 26307 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:51.952891 26265 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:12:51.969167 26343 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:51.955817 26270 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:12:51.959496 26396 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:12:51.969813 25999 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:12:52.005734 25707 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:12:52.032112 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.009377 26356 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:52.018831 26326 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:52.005059 26301 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:12:52.020994 26289 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.042246 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:52.005293 26149 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.034406 26250 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.045576 26410 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:52.052853 26148 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:52.112253 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 224700us
I20260709 13:12:52.072652 26326 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:52.120504 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2258) took 1509 ms. Trace:
I20260709 13:12:52.064327 26302 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:12:52.066713 26134 consensus_queue.cc:1243] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:52.085475 26251 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:52.115091 26169 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.120294 26410 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.117769 26328 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.112092 26338 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:52.124504 26220 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:52.145676 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.085942 26188 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.063829 25707 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:52.149973 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 179288us
I20260709 13:12:52.176798 22835 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:52.179423 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 323952us
I20260709 13:12:52.126565 26270 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:52.205135 23462 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:52.151163 26410 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:52.205286 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:52.188053 26406 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:12:52.211066 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:52.210268 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:52.216952 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:52.217761 23462 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:52.145378 26302 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:52.183784 26134 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.177350 26327 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:52.199783 26383 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:52.149698 26169 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:52.244750 26335 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.576s	user 0.007s	sys 0.005s
I20260709 13:12:52.188777 26270 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:52.237537 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 252259us
I20260709 13:12:52.273541 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:52.177623 26328 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.209954 26415 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:52.185364 26188 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.300274 26413 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.300911 26414 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:52.302470 26408 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:52.194749 26356 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.337584 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.287705 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:52.311165 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:52.233435 26232 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:52.318348 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 276262us
I20260709 13:12:52.339563 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:52.285494 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68625.5us
I20260709 13:12:52.333458 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:52.283829 25999 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:52.305409 26298 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:12:52.321137 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.353237 22832 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:52.305003 26160 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:52.262349 26134 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.350457 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.286998 26250 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:12:52.354372 22914 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:12:52.349090 22834 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [8.4-9.5]
I20260709 13:12:52.365736 22914 raft_consensus.cc:606] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:12:52.366235 22914 raft_consensus.cc:614] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:52.357304 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.358126 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.347680 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:52.308597 26335 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:52.358812 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:52.364367 22839 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:52.370895 22833 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:52.368825 22836 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:52.215911 26326 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.282128 26302 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.357689 26163 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:52.332213 26237 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:12:52.358002 26160 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:52.358358 25999 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:52.381811 26332 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.007s
I20260709 13:12:52.394939 22839 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:52.382360 25955 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.614s	user 0.014s	sys 0.000s
I20260709 13:12:52.390831 26429 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:12:52.352654 26415 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:12:52.311870 26338 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.330837 26414 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.394048 26428 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:12:52.418624 26049 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:52.392877 26250 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.419674 26188 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:52.383708 26270 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.419948 26322 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:52.393726 26327 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:52.420688 26134 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:52.418741 26434 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:52.418771 26328 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:52.423401 26323 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.000s
I20260709 13:12:52.379112 26395 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:52.382158 26039 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.005s
I20260709 13:12:52.327121 26413 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:52.411688 26411 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 13:12:52.412505 26416 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 13:12:52.413335 26422 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:52.396072 26160 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:52.430080 26163 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:52.428017 26335 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:52.452124 26250 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.386498 26424 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.463502 26049 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:52.537838 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 160532us
I20260709 13:12:52.379200 26289 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 13:12:52.464944 26290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.016s	sys 0.033s
W20260709 13:12:52.481632 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 124348us
W20260709 13:12:52.485216 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 145749us
I20260709 13:12:52.556653 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:52.431131 26332 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.455375 26232 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:52.417291 26311 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.389241 22836 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:12:52.482753 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 171685us
I20260709 13:12:52.581003 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:52.582423 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:52.474718 26285 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:12:52.506909 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 301713us
I20260709 13:12:52.515573 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.479040 26434 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.485584 26270 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:52.432459 26262 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:52.559767 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.479267 26327 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.568818 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:52.572890 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.573724 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.583772 22761 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:52.490832 26039 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.591903 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:52.555469 26290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.016s	sys 0.033s
I20260709 13:12:52.595095 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:52.595216 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 181096us
I20260709 13:12:52.601388 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:52.602535 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.554685 26435 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:52.599731 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 219180us
I20260709 13:12:52.607056 22840 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:52.610477 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:52.618238 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:52.540427 26335 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.580340 26169 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:52.631063 22757 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:52.621702 22839 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:52.627384 22765 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:52.628427 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2275) took 1017 ms. Trace:
I20260709 13:12:52.621333 22913 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:52.620934 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:52.597157 26434 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:52.591919 26415 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.599825 26323 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:52.655689 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68529.1us
I20260709 13:12:52.646229 22759 rpcz_store.cc:276] 0709 13:12:51.610777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.630433 (+ 19656us) service_pool.cc:224] Handling call
0709 13:12:51.630738 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.658716 (+ 27978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.658767 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.681710 (+ 22943us) log.cc:844] Serialized 26 byte log entry
0709 13:12:52.046566 (+364856us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:52.330094 (+283528us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.461846 (+131752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.461937 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.628207 (+166270us) raft_consensus.cc:1685] finished
0709 13:12:52.628231 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.628365 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47950,"prepare.queue_time_us":192140,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":89030,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":32928,"thread_start_us":727464,"threads_started":3,"wal-append.queue_time_us":442055}
I20260709 13:12:52.657995 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:52.663839 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.673978 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:52.585798 26232 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:52.675197 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74054.4us
W20260709 13:12:52.676381 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.679270 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:52.680862 22833 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:52.682652 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2278) took 1071 ms. Trace:
I20260709 13:12:52.694322 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:52.614447 26446 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:12:52.683082 22764 rpcz_store.cc:276] 0709 13:12:51.611514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.658353 (+ 46839us) service_pool.cc:224] Handling call
0709 13:12:51.679222 (+ 20869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.686129 (+  6907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.686161 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.691871 (+  5710us) log.cc:844] Serialized 26 byte log entry
0709 13:12:52.031598 (+339727us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:52.178151 (+146553us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.178367 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.178452 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.682387 (+503935us) raft_consensus.cc:1685] finished
0709 13:12:52.682424 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.682571 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6465,"prepare.queue_time_us":206647,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":39686,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":19294,"thread_start_us":460106,"threads_started":2,"wal-append.queue_time_us":481881}
I20260709 13:12:52.680235 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:52.679205 26153 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.001s
I20260709 13:12:52.620184 26451 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:12:52.670480 26395 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:52.596222 26285 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.716024 25999 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:52.717578 26302 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:52.719458 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 88958.6us
I20260709 13:12:52.724934 26151 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:52.705641 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:52.655371 26343 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:12:52.725616 26153 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:52.712817 26322 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 13:12:52.702950 26436 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:12:52.727797 26188 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:52.728569 26332 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:52.753440 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132596us
I20260709 13:12:52.648994 26163 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.675215 26323 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.634155 26435 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.703747 26454 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:52.767180 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:52.725638 26433 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
W20260709 13:12:52.768072 26199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.014s	sys 0.018s
I20260709 13:12:52.729310 26415 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
W20260709 13:12:52.768256 26307 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.703s	user 0.015s	sys 0.000s
I20260709 13:12:52.766096 26188 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:52.764803 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 223867us
W20260709 13:12:52.821422 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 147530us
I20260709 13:12:52.827957 23465 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:52.751606 26446 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:52.827402 26049 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:52.813143 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 133775us
I20260709 13:12:52.768760 26153 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.839862 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:52.833961 26420 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:12:52.818500 26413 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:52.864337 26039 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:52.843861 26148 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.713s	user 0.015s	sys 0.000s
I20260709 13:12:52.831655 26424 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:52.839250 26379 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:52.838626 26199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.016s	sys 0.018s
I20260709 13:12:52.843957 26307 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:52.842482 26188 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:52.866132 26448 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:52.868280 26153 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:52.868454 26410 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.003s
I20260709 13:12:52.869298 26391 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:52.864806 26473 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.836442 26327 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:52.862246 26311 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:52.925082 22908 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:52.897873 26148 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.916237 26410 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.920009 26039 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:52.908617 26391 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:52.827170 26395 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.946286 22915 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:52.948047 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:52.948980 22913 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:52.952652 26343 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:52.924408 26285 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:52.953290 26343 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:52.952732 26391 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.953904 26322 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:52.954394 26322 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:52.954704 26391 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:52.955359 26322 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:52.956507 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.956763 26322 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:52.952805 26435 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:52.985347 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:52.986510 22909 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:52.989791 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:52.927066 26327 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:53.003670 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1514) took 1795 ms. Trace:
I20260709 13:12:53.005546 22763 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:53.005985 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.004138 22910 rpcz_store.cc:276] 0709 13:12:51.208074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.210727 (+  2653us) service_pool.cc:224] Handling call
0709 13:12:51.211134 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.224311 (+ 13177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.224345 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.806101 (+581756us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b2c00051d28. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:51.806557 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:12:52.081899 (+275342us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:52.081962 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.082685 (+   723us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.982507 (+899822us) raft_consensus.cc:1685] finished
0709 13:12:52.982538 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.003578 (+ 21040us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":290045,"raft.queue_time_us":1902,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":8414,"spinlock_wait_cycles":15465344,"thread_start_us":274639,"threads_started":1,"wal-append.queue_time_us":679129}
I20260709 13:12:53.007886 22915 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:53.008620 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 168863us
I20260709 13:12:53.007316 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:52.907667 26424 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.992667 26153 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.002689 26435 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.004501 26251 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.041162 26237 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:53.041174 26445 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:53.038726 26332 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:52.954409 26343 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.012387 25999 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:12:52.921766 26379 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:52.923126 26473 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:53.071794 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 304689us
I20260709 13:12:53.082448 26237 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:53.011655 26327 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.105024 26327 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.052986 26328 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:53.061271 26417 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.984017 26039 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:53.148880 26443 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.614s	user 0.000s	sys 0.008s
W20260709 13:12:53.150290 26458 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.334s	user 0.002s	sys 0.000s
W20260709 13:12:53.151027 26463 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
W20260709 13:12:53.107144 26468 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:12:53.107435 26160 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
I20260709 13:12:53.080783 26251 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.109925 26466 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.118371 26237 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:53.083832 26445 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:53.079568 26232 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:53.127455 26409 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.933s	user 0.004s	sys 0.000s
I20260709 13:12:53.055495 26153 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.185159 26153 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:53.136472 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 284879us
I20260709 13:12:53.185412 26445 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.186774 26445 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:53.190922 26307 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:53.173801 26356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.020s	sys 0.030s
W20260709 13:12:53.193845 26356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.020s	sys 0.030s
I20260709 13:12:52.126106 22766 rpcz_store.cc:276] 0709 13:12:50.611666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.919467 (+307801us) service_pool.cc:224] Handling call
0709 13:12:50.923253 (+  3786us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:51.567426 (+644173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.567458 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:51.568042 (+   584us) log.cc:844] Serialized 52 byte log entry
0709 13:12:51.693024 (+124982us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:52.054172 (+361148us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.076172 (+ 22000us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.120070 (+ 43898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.120169 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.120216 (+    47us) raft_consensus.cc:1685] finished
0709 13:12:52.120229 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.120426 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":658908,"raft.queue_time_us":657165,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":8105,"thread_start_us":470317,"threads_started":2,"wal-append.queue_time_us":297508}
I20260709 13:12:53.196861 26473 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:53.146147 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 127193us
I20260709 13:12:53.197893 26410 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.198262 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:53.168721 26338 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:12:53.200301 26338 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:53.200418 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2277) took 1589 ms. Trace:
I20260709 13:12:53.201062 26338 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:53.201455 26338 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:53.153259 26328 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.202306 26338 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.048688 26323 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.204329 26328 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:53.162173 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.169852 26160 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.212806 22832 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:53.214591 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.174351 26447 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:53.179577 26251 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:53.182531 26449 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:12:53.229768 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.229156 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:53.200855 22758 rpcz_store.cc:276] 0709 13:12:51.611263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.630604 (+ 19341us) service_pool.cc:224] Handling call
0709 13:12:51.630843 (+   239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.662799 (+ 31956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.662840 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.663365 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:12:51.982404 (+319039us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:52.089658 (+107254us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.124410 (+ 34752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.124505 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:53.126578 (+1002073us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:53.200085 (+ 73507us) raft_consensus.cc:1685] finished
0709 13:12:53.200210 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.200349 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71487,"prepare.queue_time_us":255656,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":131412,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":17545,"spinlock_wait_cycles":11513088,"thread_start_us":390430,"threads_started":2,"wal-append.queue_time_us":450512}
W20260709 13:12:53.157821 26270 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.006s
I20260709 13:12:53.209992 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.214679 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.215641 23462 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:53.216361 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:53.235227 22832 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:53.091789 26435 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.216941 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:53.157002 26039 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:53.225673 26434 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.004s
I20260709 13:12:53.225808 26408 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.226491 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:53.221416 26410 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:12:53.227909 26160 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.228915 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:53.275813 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:53.229871 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.218750 26338 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.218971 26447 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:53.268887 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:53.267390 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1558) took 1647 ms. Trace:
I20260709 13:12:53.248803 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:53.247838 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:53.276024 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:53.143018 25993 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:12:53.230952 23464 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:53.276769 23459 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:53.243757 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:53.279970 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:53.212747 26384 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:53.280594 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.225736 26332 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.283385 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:53.304446 22765 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 6: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:53.283473 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:53.287137 22838 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.006s	sys 0.026s
W20260709 13:12:53.305011 22838 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.006s	sys 0.026s
I20260709 13:12:53.284669 22837 rpcz_store.cc:276] 0709 13:12:51.620171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.631593 (+ 11422us) service_pool.cc:224] Handling call
0709 13:12:51.631850 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:51.883590 (+251740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.267035 (+1383445us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54000c62d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:53.267316 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499601152}
I20260709 13:12:53.306269 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1563) took 1175 ms. Trace:
I20260709 13:12:53.267742 26251 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.153677 26417 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.305209 22757 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:53.246217 25964 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:53.246230 26151 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:53.307225 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:53.267035 26311 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:53.303236 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59581.7us
I20260709 13:12:53.306772 22838 rpcz_store.cc:276] 0709 13:12:52.130417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.130666 (+   249us) service_pool.cc:224] Handling call
0709 13:12:52.131619 (+   953us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:52.132161 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:52.384459 (+252298us) log.cc:844] Serialized 14 byte log entry
0709 13:12:52.651914 (+267455us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:52.668792 (+ 16878us) log.cc:844] Serialized 52 byte log entry
0709 13:12:52.668975 (+   183us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:52.669225 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 13:12:53.149688 (+480463us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:53.182611 (+ 32923us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b18003b6d40. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:53.305783 (+123172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.305901 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:53.305966 (+    65us) raft_consensus.cc:1685] finished
0709 13:12:53.305986 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.306193 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12577,"prepare.queue_time_us":317053,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":104,"raft.queue_time_us":59260,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":14745,"spinlock_wait_cycles":242753152,"thread_start_us":943411,"threads_started":3,"wal-append.queue_time_us":203816}
I20260709 13:12:53.231288 26356 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:12:53.320066 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 220200us
I20260709 13:12:53.283524 25999 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.337857 26421 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:12:53.357993 23459 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:53.279568 26270 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:53.275205 26434 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.315969 26251 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.311450 26384 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.280871 26039 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:53.351164 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:53.304219 26410 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:53.177469 26466 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.359258 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:53.347183 22832 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:12:53.281738 26408 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.313887 26332 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.367008 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:53.518177 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:53.571358 26442 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:53.365808 26311 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.573436 26442 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:53.573908 26442 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.515348 26267 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:53.574776 26267 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:53.575752 26311 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.575683 26267 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.355827 26153 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.223s
I20260709 13:12:53.576828 26267 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:53.354875 26160 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.578054 26356 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:53.578751 26356 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:53.578516 26160 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.579282 26356 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:53.581709 26408 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:53.383778 26447 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.583675 26149 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:53.578051 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 59961.2us
I20260709 13:12:53.584419 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:53.367786 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:53.584936 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:53.586216 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 218657us
W20260709 13:12:53.430735 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.439284 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:53.591590 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 240549us
I20260709 13:12:53.591794 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:53.592651 22914 raft_consensus.cc:2376] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11. Re-sending same reply.
I20260709 13:12:53.435312 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:53.593425 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.593967 22761 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:53.595125 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.595125 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:53.597415 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:53.598227 22761 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 11.
I20260709 13:12:53.599066 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:53.600956 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.329931 26385 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:53.599133 26328 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.612473 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.609879 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.613787 23459 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:53.444321 26482 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:12:53.616654 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.620728 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 575) took 1024 ms. Trace:
I20260709 13:12:53.616180 22835 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:53.622355 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.624382 23466 rpcz_store.cc:276] 0709 13:12:52.596739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.597132 (+   393us) service_pool.cc:224] Handling call
0709 13:12:52.625654 (+ 28522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.203209 (+577555us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b18003d4260. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:12:53.203360 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:53.212278 (+  8918us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.240022 (+ 27744us) log.cc:844] Serialized 26 byte log entry
0709 13:12:53.592084 (+352062us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:53.592141 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.592214 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:53.592258 (+    44us) raft_consensus.cc:1685] finished
0709 13:12:53.592269 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.620657 (+ 28388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44671,"raft.queue_time_us":189028,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9247,"spinlock_wait_cycles":6224256,"thread_start_us":452591,"threads_started":2,"wal-append.queue_time_us":311388}
I20260709 13:12:53.515941 26433 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:53.517490 26460 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.672s	user 0.004s	sys 0.000s
I20260709 13:12:53.605096 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:53.628291 22762 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:53.628633 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.628903 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.633630 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
W20260709 13:12:53.629236 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 270040us
I20260709 13:12:53.636118 26250 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.567539 26417 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:53.627138 26385 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:53.603360 26428 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:53.637136 26251 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.612208 26415 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.637740 26385 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.638239 26415 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.566821 26379 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:53.637283 26454 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.800s	user 0.013s	sys 0.000s
I20260709 13:12:53.636058 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.421573 26475 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:53.515064 26148 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:53.600167 26440 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:53.641536 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:53.664036 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.443621 26260 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:53.600322 26487 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:53.641448 26220 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:53.645395 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.695043 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:53.599758 26410 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.650775 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:53.646670 26301 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.354768 25897 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.276s
I20260709 13:12:53.346359 25964 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.622859 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:53.655665 23464 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:53.664975 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.385002 25999 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.667433 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:53.717303 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:53.719058 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 116363us
W20260709 13:12:53.668532 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63515.8us
I20260709 13:12:53.391426 26470 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:53.654006 26322 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.008s
I20260709 13:12:53.388914 26459 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.631784 26433 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:53.612097 25707 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:12:53.388543 26270 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.648228 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:53.600008 26039 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.696627 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:53.650045 22910 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
W20260709 13:12:53.700268 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.667630 22838 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:53.701149 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.711639 26326 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:53.703896 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53217.2us
I20260709 13:12:53.768163 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.654134 26421 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:53.770346 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.771587 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.771827 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.772938 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.713152 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.662766 26149 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:53.667459 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.721683 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:53.661592 26311 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:53.669318 26483 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:53.727274 23461 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:53.637773 26237 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.686615 26260 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:53.630879 26328 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.690299 26435 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.667832 25993 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.731835 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:53.734356 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:53.695807 26487 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:53.665341 26454 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:53.705090 26415 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:53.738294 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2390) took 1226 ms. Trace:
I20260709 13:12:53.713900 26148 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:53.755658 23466 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:53.712018 26220 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:53.695508 26425 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:53.719704 26484 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:53.765952 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.769932 26421 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:53.774394 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.774292 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.688197 26475 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.669492 26440 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:53.774632 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.726400 26301 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.059445 26323 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.779687 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:12:54.064039 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 284459us
I20260709 13:12:54.064898 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:54.062276 26323 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:53.785862 22910 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:53.981168 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1467) took 1321 ms. Trace:
I20260709 13:12:54.068773 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:53.952032 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.070214 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.729204 26289 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:54.070539 22911 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:53.901427 22912 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:12:53.765756 26459 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.868474 22766 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:53.863140 26326 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:53.861207 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:54.062319 26265 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:53.769939 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:54.064993 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 333284us
I20260709 13:12:53.983103 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:54.086520 22838 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:54.085947 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:54.069051 22839 rpcz_store.cc:276] 0709 13:12:52.660052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.660672 (+   620us) service_pool.cc:224] Handling call
0709 13:12:52.661138 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:52.758936 (+ 97798us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:52.798425 (+ 39489us) log.cc:844] Serialized 14 byte log entry
0709 13:12:53.282086 (+483661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:53.282152 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:53.282199 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.980859 (+698660us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b54003205d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:53.980917 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.981091 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15243,"prepare.queue_time_us":48639,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":46991,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148,"spinlock_wait_cycles":224249088,"thread_start_us":467196,"threads_started":2,"wal-append.queue_time_us":368586}
I20260709 13:12:54.088007 26232 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:54.088296 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:54.088418 26232 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:12:54.091866 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 370237us
I20260709 13:12:54.092876 22806 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 11.
I20260709 13:12:54.086198 22907 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:53.772964 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:54.096901 22906 rpcz_store.cc:276] 0709 13:12:52.511546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.511689 (+   143us) service_pool.cc:224] Handling call
0709 13:12:52.511965 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:52.512325 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:52.512350 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:52.520298 (+  7948us) log.cc:844] Serialized 26 byte log entry
0709 13:12:52.683402 (+163104us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:52.683443 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:52.683538 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:53.516523 (+832985us) spinlock_profiling.cc:243] Waited 78.6 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:53.728516 (+211993us) raft_consensus.cc:1685] finished
0709 13:12:53.728544 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:53.738135 (+  9591us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33279,"raft.queue_time_us":2032,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":8324,"spinlock_wait_cycles":180785408,"thread_start_us":100746,"threads_started":1,"wal-append.queue_time_us":303107}
W20260709 13:12:53.869656 26489 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:12:54.108608 23459 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:53.942875 26492 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
I20260709 13:12:54.109788 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:53.861304 26391 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:54.097267 26188 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.183s	user 0.013s	sys 0.002s
I20260709 13:12:54.083583 25897 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:54.088986 26232 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:54.063239 26434 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:54.114218 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:53.767278 26335 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.116739 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.138176 22906 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:12:54.112061 22839 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:54.063009 26270 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:53.888983 26466 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.085458 26440 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:54.129436 22766 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:54.090152 26160 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:54.001642 26495 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:12:54.137279 22914 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:54.097961 26435 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:54.140465 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:54.184074 22906 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:54.063144 26311 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.104328 26385 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:54.091159 26487 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:53.752758 26470 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:54.146324 22732 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items
I20260709 13:12:54.165846 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:54.090852 25993 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.090600 26149 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.097306 26265 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.187157 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.208644 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.193450 22838 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:54.090432 26483 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.211437 26483 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.209337 22914 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:54.212749 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.213486 22914 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:54.129652 26395 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:12:54.098057 26484 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.181192 26498 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:54.183694 26435 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.130496 26266 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:12:53.948478 26433 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.134696 26391 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:54.241070 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 129246us
I20260709 13:12:54.142355 26188 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.246184 22906 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:54.183521 26302 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:54.252560 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 150597us
I20260709 13:12:54.224277 26410 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:54.187316 22732 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1608)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 416)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1619)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
I20260709 13:12:54.167475 26260 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.301254 23461 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:54.319346 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:54.318041 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:54.323330 26395 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:54.323989 26395 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:54.256397 26496 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 13:12:54.246834 26435 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:54.223258 26220 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.249555 26289 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:54.339401 22906 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:54.306846 26391 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:54.262887 26039 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:54.353905 22915 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:54.312657 26484 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.321472 26473 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.308693 26393 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.379242 22732 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1608)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 416)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1619)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
I20260709 13:12:54.307271 26486 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:54.338553 25955 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:54.351719 26260 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:54.289846 25999 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.381990 26503 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.305032 26421 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.392237 26475 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.387598 22732 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1608)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 416)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1619)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
W20260709 13:12:54.418210 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 353381us
I20260709 13:12:54.419816 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:54.326460 26302 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.394044 25955 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.325184 26395 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.394778 26328 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:54.419137 26496 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:12:54.389940 26516 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:54.388986 26039 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:54.394189 26484 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.389339 26270 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:54.407998 22732 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1608)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 416)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1619)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
I20260709 13:12:54.388609 26473 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.432471 26237 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.421008 25999 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:54.439878 25955 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:54.459379 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72552.5us
I20260709 13:12:54.418345 26323 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:54.467592 22915 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:54.455160 22732 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1608)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 416)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1619)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2302)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 420)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1607)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1610)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
I20260709 13:12:54.478106 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.454751 26328 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:54.460872 26484 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:54.483718 26356 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.904s	user 0.007s	sys 0.007s
I20260709 13:12:54.486028 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.460948 26270 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.454952 26512 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:54.448835 26511 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:54.460845 25993 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.396451 26486 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.394392 26393 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.462591 26265 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.494035 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:54.474872 26311 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:54.492264 26501 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:12:54.507774 26514 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.522398 26188 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.522697 26414 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:54.525578 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:54.526368 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:54.553355 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:54.489459 26328 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:54.417232 26503 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.463371 26039 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.484236 26356 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.491549 26484 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.471638 25999 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.543972 22838 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:54.558293 22832 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:12:54.559255 26473 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:54.546017 26442 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.969s	user 0.014s	sys 0.000s
I20260709 13:12:54.510190 26429 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:12:54.451377 26289 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.559967 26328 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:54.453905 26516 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.595275 22837 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 7: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:54.601811 26505 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.202s	user 0.002s	sys 0.000s
I20260709 13:12:54.601928 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:54.602924 22764 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:54.604147 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:54.604637 26149 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:54.605291 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:54.605952 22764 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:54.573297 26414 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.568935 26470 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.615844 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:54.616959 22764 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:54.617803 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2290) took 1362 ms. Trace:
I20260709 13:12:54.618217 22764 rpcz_store.cc:276] 0709 13:12:53.255153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.606579 (+1351426us) service_pool.cc:224] Handling call
0709 13:12:54.617721 (+ 11142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:54.588259 26442 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.695991 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:54.727514 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.599s (SENT)
I20260709 13:12:54.765993 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1599) took 1114 ms. Trace:
I20260709 13:12:54.766938 22764 rpcz_store.cc:276] 0709 13:12:53.651792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.619359 (+967567us) service_pool.cc:224] Handling call
0709 13:12:54.765893 (+146534us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:54.806102 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:54.896306 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 195822us
I20260709 13:12:54.526455 26511 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.559410 26514 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.613643 26414 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:54.934350 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:54.692071 26302 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:54.508791 26512 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:54.886447 26393 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:55.191653 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 464170us
W20260709 13:12:55.139513 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.227494 22764 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:12:55.233490 26338 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.944s	user 0.006s	sys 0.011s
I20260709 13:12:55.191957 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 630) took 1471 ms. Trace:
W20260709 13:12:55.251518 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1614) took 1486 ms (client timeout 1755 ms). Trace:
W20260709 13:12:55.252002 22764 rpcz_store.cc:269] 0709 13:12:53.765038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.767902 (+1002864us) service_pool.cc:224] Handling call
0709 13:12:55.251455 (+483553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.237324 23460 rpcz_store.cc:276] 0709 13:12:53.720656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.720790 (+   134us) service_pool.cc:224] Handling call
0709 13:12:53.731496 (+ 10706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.736960 (+  5464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.736990 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.750969 (+ 13979us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.169012 (+418043us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:54.169072 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.169175 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:54.613374 (+444199us) raft_consensus.cc:1685] finished
0709 13:12:54.613394 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.191882 (+578488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4787,"raft.queue_time_us":5104,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":11596,"thread_start_us":281097,"threads_started":1,"wal-append.queue_time_us":525432}
I20260709 13:12:55.256678 22759 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:12:55.257776 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:54.916487 26503 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.155619 26270 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.243326 26149 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:55.263844 26267 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.686s	user 0.017s	sys 0.001s
I20260709 13:12:55.276855 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.283587 22764 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
W20260709 13:12:55.284416 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1609) took 1559 ms (client timeout 1822 ms). Trace:
W20260709 13:12:55.284971 22764 rpcz_store.cc:269] 0709 13:12:53.725190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.252676 (+1527486us) service_pool.cc:224] Handling call
0709 13:12:55.284323 (+ 31647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:54.901945 26483 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.359s
I20260709 13:12:55.287420 22763 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:55.314358 26332 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:55.314580 26260 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.002s
W20260709 13:12:55.301131 26232 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.134s	user 0.012s	sys 0.001s
I20260709 13:12:55.286970 26149 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.294539 26486 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.297260 26509 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:55.299916 26039 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:55.319341 25999 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:55.331538 26521 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.542s	user 0.001s	sys 0.000s
W20260709 13:12:55.335034 26440 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.126s	user 0.007s	sys 0.007s
I20260709 13:12:55.334596 26356 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.345921 26511 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.351846 26440 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.358963 26394 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:55.366699 26424 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.367049 26428 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:55.337344 26509 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:55.372937 26498 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.120s	user 0.017s	sys 0.000s
I20260709 13:12:55.373422 26498 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:55.369988 26514 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:55.421417 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 320202us
I20260709 13:12:55.421731 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:55.423977 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 419) took 1655 ms (client timeout 1798 ms). Trace:
W20260709 13:12:55.426154 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2281) took 2178 ms (client timeout 1556 ms). Trace:
W20260709 13:12:55.426937 22766 rpcz_store.cc:269] 0709 13:12:53.247491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.247611 (+   120us) service_pool.cc:224] Handling call
0709 13:12:55.425643 (+2178032us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b540002e4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.425907 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1342878080}
I20260709 13:12:55.429019 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:55.430151 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1581) took 1754 ms. Trace:
W20260709 13:12:55.430913 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.920s (SENT)
I20260709 13:12:55.439198 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2392) took 2861 ms. Trace:
W20260709 13:12:55.444022 26487 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.228s	user 0.015s	sys 0.002s
I20260709 13:12:55.440747 22915 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:12:55.424503 22764 rpcz_store.cc:269] 0709 13:12:53.769038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.286016 (+1516978us) service_pool.cc:224] Handling call
0709 13:12:55.423842 (+137826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.362679 26394 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:55.456035 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2294) took 2199 ms (client timeout 1920 ms). Trace:
I20260709 13:12:55.326006 26232 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.459829 26511 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:55.460269 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 556) took 7164 ms (client timeout 1761 ms). Trace:
I20260709 13:12:55.450080 22838 rpcz_store.cc:276] 0709 13:12:53.675112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.326241 (+651129us) service_pool.cc:224] Handling call
0709 13:12:54.353746 (+ 27505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.574994 (+221248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.429244 (+854250us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54001bbf54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:55.429764 (+   520us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436316544}
I20260709 13:12:55.461203 26511 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.462390 26527 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.463783 26527 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.424624 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.466130 22760 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:55.466583 26462 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.620s	user 0.015s	sys 0.014s
W20260709 13:12:55.466719 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1619) took 1382 ms (client timeout 1529 ms). Trace:
I20260709 13:12:55.467268 26149 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.467758 26498 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.468922 26498 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.469762 26326 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.367579 26428 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:55.472565 26232 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.474941 26473 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:55.475432 26473 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:55.467864 26424 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:55.478335 26424 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.483410 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:55.450296 22908 rpcz_store.cc:276] 0709 13:12:52.577051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.577209 (+   158us) service_pool.cc:224] Handling call
0709 13:12:52.606020 (+ 28811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:52.925853 (+319833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.438348 (+2512495us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000cf654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:55.438859 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733203712}
W20260709 13:12:55.485158 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54589.1us
W20260709 13:12:55.467285 22760 rpcz_store.cc:269] 0709 13:12:54.084337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.423962 (+1339625us) service_pool.cc:224] Handling call
0709 13:12:55.466633 (+ 42671us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.436924 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:55.486583 26487 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.513286 26487 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:55.460947 23465 rpcz_store.cc:269] 0709 13:12:48.296028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.354123 (+ 58095us) service_pool.cc:224] Handling call
0709 13:12:49.852895 (+1498772us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b5800178b98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:12:55.459587 (+5606692us) spinlock_profiling.cc:243] Waited 2.56 s on lock 0x7b5400740254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.459860 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5901858432}
W20260709 13:12:55.493651 22907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1529) took 1717 ms (client timeout 2117 ms). Trace:
W20260709 13:12:55.456887 22761 rpcz_store.cc:269] 0709 13:12:53.256431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.646341 (+389910us) service_pool.cc:224] Handling call
0709 13:12:55.455321 (+1808980us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54000028d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.455666 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3926851840}
I20260709 13:12:55.455228 26423 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:55.455380 26440 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:55.485263 26509 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:55.523267 26465 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:55.523867 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.525252 26424 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.534304 26343 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:55.534688 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:55.536176 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1594) took 1885 ms (client timeout 2013 ms). Trace:
W20260709 13:12:55.515611 22907 rpcz_store.cc:269] 0709 13:12:53.776313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.792802 (+ 16489us) service_pool.cc:224] Handling call
0709 13:12:55.492960 (+1700158us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b54002107d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.493289 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34960640}
I20260709 13:12:55.538865 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:55.532225 26465 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:55.536782 22764 rpcz_store.cc:269] 0709 13:12:53.650571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.516388 (+1865817us) service_pool.cc:224] Handling call
0709 13:12:55.536047 (+ 19659us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.544399 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:55.547817 26499 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.459s	user 0.000s	sys 0.011s
W20260709 13:12:55.551798 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 115551us
I20260709 13:12:55.554349 22760 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:55.554533 26423 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:55.555234 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1601) took 1903 ms (client timeout 2033 ms). Trace:
W20260709 13:12:55.556139 22760 rpcz_store.cc:269] 0709 13:12:53.652278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.523311 (+1871033us) service_pool.cc:224] Handling call
0709 13:12:55.555140 (+ 31829us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.558102 26423 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:55.558641 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.559293 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 423) took 1893 ms. Trace:
I20260709 13:12:55.559756 22761 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:55.557929 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:55.560483 26301 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.559867 22909 rpcz_store.cc:276] 0709 13:12:53.665427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.665584 (+   157us) service_pool.cc:224] Handling call
0709 13:12:53.665968 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:53.687523 (+ 21555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.687562 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:53.700019 (+ 12457us) log.cc:844] Serialized 52 byte log entry
0709 13:12:53.928471 (+228452us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:53.928538 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.928652 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.546551 (+1617899us) spinlock_profiling.cc:243] Waited 9.01 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:55.558495 (+ 11944us) raft_consensus.cc:1685] finished
0709 13:12:55.558518 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.559003 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21082,"raft.queue_time_us":38546,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":6578,"spinlock_wait_cycles":20726656,"thread_start_us":192701,"threads_started":1,"wal-append.queue_time_us":362961}
I20260709 13:12:55.563911 26384 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.565676 22766 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:55.568960 26433 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:55.516713 26473 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.574662 26442 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:55.575655 26473 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:55.576676 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:55.577291 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:55.554343 26440 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:55.578142 26440 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:55.578214 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2283) took 2329 ms (client timeout 1670 ms). Trace:
W20260709 13:12:55.560606 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2302) took 1451 ms (client timeout 1586 ms). Trace:
I20260709 13:12:55.486259 26394 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:55.579133 22761 rpcz_store.cc:269] 0709 13:12:54.108870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.552417 (+1443547us) service_pool.cc:224] Handling call
0709 13:12:55.560520 (+  8103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:55.553155 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47753.2us
I20260709 13:12:55.591645 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:55.582086 26426 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.552510 26465 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.531361 26232 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.595544 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:55.596350 26426 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.603727 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:55.605307 22834 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:12:55.606875 22837 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1579) took 1947 ms (client timeout 2218 ms). Trace:
W20260709 13:12:55.578855 22759 rpcz_store.cc:269] 0709 13:12:53.248336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.248486 (+   150us) service_pool.cc:224] Handling call
0709 13:12:55.577748 (+2329262us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.577982 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4478059904}
I20260709 13:12:55.608451 26465 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:55.608966 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.607213 26426 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.610023 26450 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:55.610266 22836 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:55.596875 23458 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:55.612866 26500 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.524s	user 0.009s	sys 0.012s
I20260709 13:12:55.624782 22906 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:55.625701 22759 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:55.607378 22837 rpcz_store.cc:269] 0709 13:12:53.659773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.066365 (+406592us) service_pool.cc:224] Handling call
0709 13:12:55.606383 (+1540018us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54001bc954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.606654 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1101823744}
W20260709 13:12:55.626312 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1610) took 1897 ms (client timeout 1991 ms). Trace:
I20260709 13:12:55.592175 26440 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:55.623471 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:55.621752 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:55.626703 22759 rpcz_store.cc:269] 0709 13:12:53.728811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.608454 (+1879643us) service_pool.cc:224] Handling call
0709 13:12:55.626221 (+ 17767us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.632984 26251 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:55.634745 22732 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items [suppressed 4 similar messages]
W20260709 13:12:55.625401 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1573) took 2016 ms (client timeout 1991 ms). Trace:
W20260709 13:12:55.635541 22835 rpcz_store.cc:269] 0709 13:12:53.609308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.609518 (+   210us) service_pool.cc:224] Handling call
0709 13:12:55.624895 (+2015377us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b54000a5854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.625170 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4217600}
I20260709 13:12:55.636979 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:55.658748 22835 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:55.635135 22732 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 423)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
W20260709 13:12:55.660777 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49381.7us
I20260709 13:12:55.661156 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:55.662465 22837 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:12:55.663206 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:55.664268 26435 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.294s	user 0.014s	sys 0.010s
I20260709 13:12:55.664387 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:55.664915 26435 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.666549 26459 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.667333 26450 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:55.668111 26435 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.669706 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:55.669473 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44317.4us
I20260709 13:12:55.679652 22766 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
W20260709 13:12:55.680531 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1612) took 1930 ms (client timeout 1909 ms). Trace:
W20260709 13:12:55.681205 22766 rpcz_store.cc:269] 0709 13:12:53.750236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.428525 (+1678289us) service_pool.cc:224] Handling call
0709 13:12:55.680435 (+251910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.696192 26428 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.698738 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:55.699745 23458 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:55.718142 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:55.721210 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:55.721735 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:55.721050 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:55.723024 22909 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:55.732201 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.733352 22906 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:12:55.738365 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2296) took 1643 ms (client timeout 1657 ms). Trace:
W20260709 13:12:55.739017 22766 rpcz_store.cc:269] 0709 13:12:54.095094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.690833 (+1595739us) service_pool.cc:224] Handling call
0709 13:12:55.738265 (+ 47432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.740396 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.705081 26232 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.750344 23465 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:55.751655 26232 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:55.765378 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48370.1us
W20260709 13:12:55.774811 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1175082679999377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
I20260709 13:12:55.775305 26503 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.776208 26503 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.776681 26188 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.777278 26188 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.777946 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.778784 22911 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:55.780485 26188 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.782006 26322 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.782976 26322 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:55.784721 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:55.775475 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0743082679999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
W20260709 13:12:55.768501 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.658s (SENT)
W20260709 13:12:55.790428 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.779642 26511 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:55.786901 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 420) took 2017 ms (client timeout 1937 ms). Trace:
W20260709 13:12:55.794601 22764 rpcz_store.cc:269] 0709 13:12:53.769628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.786615 (+2016987us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:55.786793 (+   178us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:12:55.791225 25964 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.799804 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:55.808112 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.809253 22909 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:55.811010 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.812713 26511 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:55.812945 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:55.814342 22907 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:55.821302 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.825423 22907 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:55.858585 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:55.836938 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1593) took 2186 ms (client timeout 2087 ms). Trace:
W20260709 13:12:55.859531 22759 rpcz_store.cc:269] 0709 13:12:53.650311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.836435 (+2186124us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:55.836849 (+   414us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:12:55.857478 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58028.4us
W20260709 13:12:55.826833 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1603) took 2164 ms (client timeout 2048 ms). Trace:
W20260709 13:12:55.861349 22760 rpcz_store.cc:269] 0709 13:12:53.662131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.826583 (+2164452us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:55.826735 (+   152us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:12:55.862916 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1607) took 2147 ms (client timeout 2002 ms). Trace:
W20260709 13:12:55.863368 22761 rpcz_store.cc:269] 0709 13:12:53.715346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.858645 (+2143299us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:55.862822 (+  4177us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:12:55.825340 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 410) took 3242 ms (client timeout 1937 ms). Trace:
I20260709 13:12:55.828671 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 515) took 2181 ms. Trace:
I20260709 13:12:55.828408 26511 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.035s
W20260709 13:12:55.864029 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1621) took 1730 ms (client timeout 1720 ms). Trace:
W20260709 13:12:55.867637 22764 rpcz_store.cc:269] 0709 13:12:54.109731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.837462 (+1727731us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:55.840273 (+  2811us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:12:55.835376 22915 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1543) took 1370 ms (client timeout 1635 ms). Trace:
W20260709 13:12:55.832266 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1598) took 2180 ms (client timeout 2170 ms). Trace:
W20260709 13:12:55.872081 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 82605.4us
I20260709 13:12:55.787228 26503 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:55.869422 22915 rpcz_store.cc:269] 0709 13:12:54.465022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.490299 (+ 25277us) service_pool.cc:224] Handling call
0709 13:12:55.835157 (+1344858us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b540020b7d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.835314 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120610048}
W20260709 13:12:55.864961 22762 rpcz_store.cc:269] 0709 13:12:52.582321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.594736 (+ 12415us) service_pool.cc:224] Handling call
0709 13:12:55.824848 (+3230112us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002dcdd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.825129 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2319023872}
W20260709 13:12:55.870848 22765 rpcz_store.cc:269] 0709 13:12:53.651539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.831829 (+2180290us) service_pool.cc:211] Skipping call since client already timed out
0709 13:12:55.832010 (+   181us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:12:55.831116 26493 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 2.068s	user 0.004s	sys 0.007s
W20260709 13:12:55.826692 25955 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.346s	user 0.011s	sys 0.013s
I20260709 13:12:55.866180 22836 rpcz_store.cc:276] 0709 13:12:53.647515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.188573 (+541058us) service_pool.cc:224] Handling call
0709 13:12:55.609615 (+1421042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.623994 (+ 14379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.828412 (+204418us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001bc950. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:55.828587 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3172312832}
I20260709 13:12:55.875368 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.878950 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.880620 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 638) took 1554 ms. Trace:
I20260709 13:12:55.880954 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 636) took 2149 ms. Trace:
I20260709 13:12:55.881475 23457 rpcz_store.cc:276] 0709 13:12:53.731324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.746328 (+ 15004us) service_pool.cc:224] Handling call
0709 13:12:53.746571 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.115969 (+369398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.116006 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:54.116053 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:54.116086 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.873250 (+1757164us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400603454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:55.880522 (+  7272us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.880735 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849017216}
I20260709 13:12:55.883201 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.785477 26322 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.830950 26428 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:55.856449 26188 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.884781 22731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1622) took 1458 ms. Trace:
I20260709 13:12:55.885838 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:55.888983 26298 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:55.890010 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:55.891067 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 417) took 2164 ms (client timeout 2156 ms). Trace:
W20260709 13:12:55.891511 22759 rpcz_store.cc:269] 0709 13:12:53.727055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.871267 (+2144212us) service_pool.cc:224] Handling call
0709 13:12:55.890963 (+ 19696us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.826828 26435 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:12:55.874975 25964 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:55.895763 22907 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:12:55.888278 22731 rpcz_store.cc:276] 0709 13:12:54.424424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.882777 (+1458353us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:12:55.886413 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:55.897862 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.904371 26260 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.904723 26148 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:55.904930 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 191806us
I20260709 13:12:55.905242 26260 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.905722 26511 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:55.905967 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.798s (SENT)
I20260709 13:12:55.905488 26148 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.906379 26511 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.906869 26511 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:55.907509 23431 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.937s (SENT)
I20260709 13:12:55.908020 26260 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:55.881131 23462 rpcz_store.cc:276] 0709 13:12:54.322605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.322848 (+   243us) service_pool.cc:224] Handling call
0709 13:12:54.323411 (+   563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:54.482155 (+158744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.482199 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:54.491376 (+  9177us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.611273 (+119897us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:54.611319 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.873079 (+1261760us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540062f554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:55.874418 (+  1339us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.874511 (+    93us) raft_consensus.cc:1685] finished
0709 13:12:55.874531 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.877331 (+  2800us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":566,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3019,"spinlock_wait_cycles":364097280,"thread_start_us":71802,"threads_started":1,"wal-append.queue_time_us":768116}
I20260709 13:12:55.908087 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:55.905433 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47388.6us
I20260709 13:12:55.910614 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.916244 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.924871 22653 maintenance_manager.cc:419] P 6e189717484b49e8bb20c98667e6d294: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033357
I20260709 13:12:55.929175 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:55.932261 26413 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:55.927918 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.938740 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:55.939782 22759 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:55.917780 26322 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:55.940297 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 415) took 2222 ms (client timeout 2754 ms). Trace:
I20260709 13:12:55.940533 26447 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:55.943575 22759 rpcz_store.cc:269] 0709 13:12:53.718317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.895789 (+2177472us) service_pool.cc:224] Handling call
0709 13:12:55.940217 (+ 44428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.942605 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:55.949096 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1617) took 2042 ms (client timeout 2136 ms). Trace:
W20260709 13:12:55.949927 22880 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.218s (SENT) [suppressed 37 similar messages]
W20260709 13:12:55.949580 22764 rpcz_store.cc:269] 0709 13:12:53.903592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.875201 (+1971609us) service_pool.cc:224] Handling call
0709 13:12:55.946122 (+ 70921us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408}
W20260709 13:12:55.951726 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86752.2us
I20260709 13:12:55.895967 26188 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:55.912220 22731 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46956 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2310)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620)
I20260709 13:12:55.893379 25955 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:55.955865 22908 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:55.960918 22762 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
W20260709 13:12:55.964407 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1616) took 2072 ms (client timeout 2218 ms). Trace:
W20260709 13:12:55.964808 22762 rpcz_store.cc:269] 0709 13:12:53.892123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.884619 (+1992496us) service_pool.cc:224] Handling call
0709 13:12:55.964347 (+ 79728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.969308 26379 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:55.969720 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.972851 22760 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:55.973389 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1606) took 2285 ms (client timeout 2223 ms). Trace:
I20260709 13:12:55.973925 22765 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:12:55.974416 22760 rpcz_store.cc:269] 0709 13:12:53.687929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.865864 (+2177935us) service_pool.cc:224] Handling call
0709 13:12:55.973328 (+107464us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24448}
I20260709 13:12:55.976321 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:55.962463 26428 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:55.985672 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:55.989048 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:55.974234 26148 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:55.988487 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:56.001413 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.002242 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.002593 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.004760 22908 raft_consensus.cc:2368] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:56.004160 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:56.006099 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98691.4us
I20260709 13:12:56.009370 22760 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:56.013558 26434 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:56.014814 26434 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:56.013067 26428 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.021466 22907 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:12:56.023820 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.022812 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.025914 26516 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:56.026721 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1615) took 2164 ms (client timeout 1924 ms). Trace:
W20260709 13:12:56.027187 22766 rpcz_store.cc:269] 0709 13:12:53.861856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.739939 (+1878083us) service_pool.cc:224] Handling call
0709 13:12:56.024742 (+284803us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54003689a8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78ce924 00007f68a6ab7227 00007f68a6ab6ab8 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:56.026612 (+  1870us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33622272}
I20260709 13:12:56.028098 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:56.033824 22761 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:12:56.034343 22761 pending_rounds.cc:85] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 3
I20260709 13:12:56.035791 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.017402 26434 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.034701 22761 pending_rounds.cc:107] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:56.045650 22915 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 7: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:56.044189 22761 raft_consensus.cc:2889] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:56.047290 22911 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:56.049000 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1591) took 2399 ms. Trace:
I20260709 13:12:56.049618 22761 rpcz_store.cc:276] 0709 13:12:53.649290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.030631 (+2381341us) service_pool.cc:224] Handling call
0709 13:12:56.031216 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.048135 (+ 16919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.048364 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.048617 26413 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.053207 26413 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.055837 26413 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.056157 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2287) took 2802 ms. Trace:
I20260709 13:12:56.058053 22762 rpcz_store.cc:276] 0709 13:12:53.254196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.989858 (+2735662us) service_pool.cc:224] Handling call
0709 13:12:56.044857 (+ 54999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.055748 (+ 10891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.055781 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.055830 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:56.055865 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.055936 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.056069 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.062125 22913 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.063593 22838 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:56.019932 26440 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:56.045535 26328 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.356s	user 0.014s	sys 0.011s
I20260709 13:12:56.069254 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.051627 26322 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.074179 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:56.075739 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.060006 26503 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:12:56.080441 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2280) took 2828 ms (client timeout 1556 ms). Trace:
W20260709 13:12:56.083688 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104635us
W20260709 13:12:56.078333 26473 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.502s	user 0.016s	sys 0.001s
I20260709 13:12:56.086031 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:56.080941 22758 rpcz_store.cc:269] 0709 13:12:53.247163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.247434 (+   271us) service_pool.cc:224] Handling call
0709 13:12:56.075526 (+2828092us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400222ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:56.075736 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684124416}
I20260709 13:12:56.091413 22762 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:56.094605 26512 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:56.083145 26516 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.091302 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.101995 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.115439 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:56.112756 22807 connection.cc:256] RPC call timeout handler was delayed by -1.3158103420000771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 13:12:56.119648 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.073307 26328 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.133808 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.134198 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.139530 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.140390 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:56.141042 26328 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:56.117190 26503 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.091089 26322 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.148950 23458 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:56.149466 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.178937 26503 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.136023 26335 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:12:56.157449 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.154088 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:56.144610 22758 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.181272 22759 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:56.148285 23457 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:56.181818 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2301) took 2074 ms (client timeout 2748 ms). Trace:
W20260709 13:12:56.182251 22759 rpcz_store.cc:269] 0709 13:12:54.107872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.101348 (+1993476us) service_pool.cc:224] Handling call
0709 13:12:56.181753 (+ 80405us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:56.157393 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2289) took 2902 ms (client timeout 1639 ms). Trace:
I20260709 13:12:56.183446 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.154810 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:56.176183 26544 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:12:56.185021 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:56.141408 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.133944 26188 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.186033 22762 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:56.187104 26440 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:56.161280 26498 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:56.187685 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.188809 22762 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:56.146910 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:56.190182 23462 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.176497 22761 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:56.170958 26527 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:56.193389 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 143450us
I20260709 13:12:56.148635 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.195710 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.195825 23465 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:12:56.196149 26394 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:56.197125 26527 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:56.174961 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.198195 26527 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:56.198756 26527 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:56.199604 26298 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:56.138577 26428 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.201036 26379 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:12:56.202224 26546 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:12:56.203049 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:56.202183 26298 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:56.205181 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:56.205591 26298 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:56.203742 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.176396 26311 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:56.137060 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.160719 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.164502 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.211419 22764 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:56.151010 26414 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:56.183202 22763 rpcz_store.cc:269] 0709 13:12:53.254852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.438956 (+184104us) service_pool.cc:224] Handling call
0709 13:12:56.156863 (+2717907us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400311854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:56.157147 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3902220672}
I20260709 13:12:56.179998 26322 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.221925 26414 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:56.224226 26289 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:56.224771 26289 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.225143 26289 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:56.225010 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.225782 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:56.226523 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:56.237537 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76973.4us
W20260709 13:12:56.208280 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69211.1us
W20260709 13:12:56.249361 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2300) took 2142 ms (client timeout 1966 ms). Trace:
W20260709 13:12:56.250051 22765 rpcz_store.cc:269] 0709 13:12:54.106964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.883664 (+1776700us) service_pool.cc:224] Handling call
0709 13:12:56.249055 (+365391us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b540037f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:56.249234 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25743616}
W20260709 13:12:56.251904 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 82426.6us
I20260709 13:12:56.253235 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 412) took 2949 ms. Trace:
I20260709 13:12:56.253749 22766 rpcz_store.cc:276] 0709 13:12:53.277260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.028137 (+2750877us) service_pool.cc:224] Handling call
0709 13:12:56.028645 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.029055 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.029082 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:56.034348 (+  5266us) log.cc:844] Serialized 26 byte log entry
0709 13:12:56.077684 (+ 43336us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:56.088691 (+ 11007us) log.cc:844] Serialized 14 byte log entry
0709 13:12:56.088939 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.088998 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:56.226717 (+137719us) raft_consensus.cc:1685] finished
0709 13:12:56.226740 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.226898 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11459,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":5660,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3385,"spinlock_wait_cycles":2517888,"thread_start_us":20253,"threads_started":1,"wal-append.queue_time_us":48991}
I20260709 13:12:56.261057 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.263886 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.131886 26250 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:56.263931 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.266216 22762 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:56.270917 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.282495 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.283488 23466 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:12:56.292192 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.297919 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:56.300931 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 54149.6us
I20260709 13:12:56.305703 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.307163 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.310391 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.312065 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.290484 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:56.323923 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:56.324823 22832 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.334492 22913 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:12:56.337313 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.316851 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.340925 22763 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:56.342198 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:56.340220 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:56.348531 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61273us
W20260709 13:12:56.348799 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 299084us
W20260709 13:12:56.350289 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.304580 26498 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.351790 22909 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:56.354004 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.356329 22764 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:56.342829 22906 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:56.362035 22766 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:56.362222 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:56.361009 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57409.1us
W20260709 13:12:56.361758 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.365561 23463 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:56.339967 25955 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.369220 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:56.378887 26413 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.379426 26413 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:56.379817 26413 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.378898 26285 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.381616 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.382490 26285 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.385179 26285 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.389710 26039 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.390514 26039 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.391026 26039 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:56.394901 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 95526.6us
I20260709 13:12:56.392690 26511 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.411794 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.414378 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.418856 22832 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:56.369210 26343 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.430330 26148 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:56.431097 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.359692 26516 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:12:56.432085 23460 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:56.402871 25955 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:56.353408 26459 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:12:56.441325 22910 raft_consensus.cc:2368] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8.
W20260709 13:12:56.441710 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74400.8us
I20260709 13:12:56.445883 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.454704 22915 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:56.436842 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.457804 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.458738 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.465957 22763 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:56.466176 23462 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:56.454032 26434 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.470317 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:56.472088 26434 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:56.473901 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 114334us
I20260709 13:12:56.474129 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.475266 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:56.475598 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.476266 22765 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:56.476394 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:56.476997 22839 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:56.477106 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.477670 22759 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:56.478667 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.479523 26498 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.479383 26413 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.479943 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.481626 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.481940 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.481104 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.482786 26413 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:56.481190 22834 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:56.483086 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.483850 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:56.484560 23463 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:56.477280 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.485737 26413 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.485903 26440 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:56.486289 26413 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.486797 26413 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.488256 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 411) took 3211 ms. Trace:
I20260709 13:12:56.488700 22760 rpcz_store.cc:276] 0709 13:12:53.276963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.010819 (+2733856us) service_pool.cc:224] Handling call
0709 13:12:56.040680 (+ 29861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.064897 (+ 24217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.064929 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:56.088561 (+ 23632us) log.cc:844] Serialized 26 byte log entry
0709 13:12:56.356320 (+267759us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:56.356371 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.356454 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:56.487983 (+131529us) raft_consensus.cc:1685] finished
0709 13:12:56.488008 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.488170 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5137,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3709,"thread_start_us":115292,"threads_started":1,"wal-append.queue_time_us":349667}
I20260709 13:12:56.490398 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.490733 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.491674 22760 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:56.492529 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.492538 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.493484 22760 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:56.461767 22764 raft_consensus.cc:2376] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7. Re-sending same reply.
I20260709 13:12:56.494493 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.494626 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.495277 22764 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:56.495361 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.495666 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.496382 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.496796 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.497623 22765 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:56.498276 22662 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 133) took 30041 ms (client timeout 15000 ms). Trace:
I20260709 13:12:56.496882 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:56.503516 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47964us
I20260709 13:12:56.505188 26466 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:56.508049 26466 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.508469 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:56.509728 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:56.510565 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.511368 22765 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:56.511166 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:56.512554 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.512971 26260 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.513350 22834 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:56.513398 26260 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.513763 26394 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:56.514139 26260 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.514621 26394 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:56.515902 26394 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.432062 26343 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:56.517731 26343 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.514003 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:56.506554 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.518801 22915 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:12:56.520184 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45053.7us
I20260709 13:12:56.512387 26396 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:56.480002 26498 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:56.513307 22765 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:12:56.482522 22837 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:56.530400 22758 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:56.497242 22764 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 6.
I20260709 13:12:56.545054 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:56.507561 22662 rpcz_store.cc:269] 0709 13:12:26.414189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.414506 (+   317us) service_pool.cc:224] Handling call
0709 13:12:56.449465 (+30034959us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:48.978813 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:48.979575 (+   762us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:48.979634 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:12:48.979654 (+    20us) tablet.cc:662] Decoding operations
0709 13:12:48.980698 (+  1044us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:48.980804 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:48.980829 (+    25us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:12:49.038682 (+ 57853us) tablet.cc:701] Row locks acquired
0709 13:12:49.072386 (+ 33704us) write_op.cc:260] PREPARE: finished
0709 13:12:49.108741 (+ 36355us) write_op.cc:270] Start()
0709 13:12:49.183080 (+ 74339us) write_op.cc:276] Timestamp: P: 1783602769098681 usec, L: 0
0709 13:12:49.183107 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:12:49.292487 (+109380us) log.cc:844] Serialized 3540 byte log entry
0709 13:12:50.474150 (+1181663us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:12:51.039833 (+565683us) op_driver.cc:464] REPLICATION: finished
0709 13:12:51.345885 (+306052us) write_op.cc:301] APPLY: starting
0709 13:12:51.350028 (+  4143us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:51.376097 (+ 26069us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:51.376125 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:51.426388 (+ 50263us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:51.426616 (+   228us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:12:51.426661 (+    45us) write_op.cc:312] APPLY: finished
0709 13:12:51.426943 (+   282us) log.cc:844] Serialized 65 byte log entry
0709 13:12:51.427136 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:51.427278 (+   142us) write_op.cc:454] Released schema lock
0709 13:12:51.427323 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:51.429827 (+  2504us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:51.429860 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1118984,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":291685,"apply.run_cpu_time_us":3733,"apply.run_wall_time_us":97945,"num_ops":8,"prepare.queue_time_us":292051,"prepare.run_cpu_time_us":9195,"prepare.run_wall_time_us":523526,"raft.queue_time_us":343101,"raft.run_cpu_time_us":35236,"raft.run_wall_time_us":193614,"replication_time_us":1854211,"spinlock_wait_cycles":11667712,"thread_start_us":663759,"threads_started":4,"wal-append.queue_time_us":266120,"wal-append.run_cpu_time_us":98058,"wal-append.run_wall_time_us":2843904}]]}
I20260709 13:12:56.537967 22760 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:56.531721 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:56.538076 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.547040 22762 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:56.537261 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:56.540019 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.555289 22838 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:56.511657 26466 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:56.544651 26447 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.008s
I20260709 13:12:56.560815 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:56.531984 26396 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.583612 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.554196 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.569957 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:56.571064 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.585806 23463 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:56.546125 26498 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.570833 26413 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.571941 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:56.590219 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 171966us
I20260709 13:12:56.574280 26298 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.590140 23460 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:56.575911 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.577690 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:56.580539 26553 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:12:56.595914 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:56.597838 26160 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:56.598724 26160 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.599357 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:56.600008 26289 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:56.581851 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68333.9us
I20260709 13:12:56.580848 26266 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:56.588790 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:56.553745 25955 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.620352 22834 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:56.608486 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1592) took 2944 ms. Trace:
I20260709 13:12:56.622820 22764 rpcz_store.cc:276] 0709 13:12:53.649839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.559476 (+2909637us) service_pool.cc:224] Handling call
0709 13:12:56.587200 (+ 27724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.593372 (+  6172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.593401 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.593459 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:56.593533 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.593600 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.593712 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.594867 22757 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:56.601091 26160 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:56.586805 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:56.603551 22758 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:56.604017 22765 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:56.609293 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1595) took 2958 ms. Trace:
I20260709 13:12:56.586189 26396 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:56.628260 22763 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:56.635351 22833 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.635829 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1596) took 2984 ms. Trace:
I20260709 13:12:56.633390 22762 rpcz_store.cc:276] 0709 13:12:53.650825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.565872 (+2915047us) service_pool.cc:224] Handling call
0709 13:12:56.608657 (+ 42785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.608879 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.608933 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.608989 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:56.609031 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.609101 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.609220 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.636235 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.636373 22763 rpcz_store.cc:276] 0709 13:12:53.651079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.562526 (+2911447us) service_pool.cc:224] Handling call
0709 13:12:56.628008 (+ 65482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.635511 (+  7503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.635766 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.637189 23460 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:56.555809 26232 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 13:12:56.643006 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.607334 26266 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:56.649714 26302 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.650243 26302 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.650475 26266 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.619612 26473 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:12:56.651971 26266 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:56.626652 26160 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.657892 26160 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:56.657173 26527 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.660684 26527 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:56.661702 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 113413us
I20260709 13:12:56.619318 26413 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.664933 26413 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:56.665318 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1602) took 3012 ms. Trace:
I20260709 13:12:56.665946 22758 rpcz_store.cc:276] 0709 13:12:53.652525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.640634 (+2988109us) service_pool.cc:224] Handling call
0709 13:12:56.664633 (+ 23999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.664911 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.664940 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.665011 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:56.665056 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.665115 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.665248 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.628983 26466 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.667101 26466 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:56.640026 26250 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:56.669672 26470 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:56.670519 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:12:56.671540 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 95716.3us
I20260709 13:12:56.619254 26289 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:56.693437 22839 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:56.631122 26326 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:56.696686 22760 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:12:56.695700 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:56.694880 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.728018 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:56.728484 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:56.729590 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 65480.7us
I20260709 13:12:56.731123 22760 pending_rounds.cc:85] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:12:56.731986 22760 pending_rounds.cc:107] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:12:56.732364 22760 raft_consensus.cc:2889] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:56.735076 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:56.739151 23461 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:56.739748 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:56.745222 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44799.5us
I20260709 13:12:56.748435 23462 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:56.759059 22765 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:56.759999 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1613) took 3001 ms. Trace:
I20260709 13:12:56.764951 22765 rpcz_store.cc:276] 0709 13:12:53.758386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.644610 (+2886224us) service_pool.cc:224] Handling call
0709 13:12:56.727721 (+ 83111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.759630 (+ 31909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.759911 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.767614 22762 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:56.768494 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1605) took 3099 ms. Trace:
I20260709 13:12:56.768887 22762 rpcz_store.cc:276] 0709 13:12:53.669164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.636810 (+2967646us) service_pool.cc:224] Handling call
0709 13:12:56.757285 (+120475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.768133 (+ 10848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.768422 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.770957 22762 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:12:56.771396 22762 pending_rounds.cc:85] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 3
I20260709 13:12:56.705087 26470 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:56.648238 25955 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:56.812951 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 66744.3us
I20260709 13:12:56.771659 22762 pending_rounds.cc:107] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:56.787288 22732 leader_election.cc:400] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:56.764566 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1611) took 3017 ms. Trace:
I20260709 13:12:56.816568 22762 raft_consensus.cc:2889] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:56.816848 22760 rpcz_store.cc:276] 0709 13:12:53.747446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.640591 (+2893145us) service_pool.cc:224] Handling call
0709 13:12:56.688910 (+ 48319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.764257 (+ 75347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.764507 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.819756 22760 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:56.830518 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2297) took 2730 ms. Trace:
I20260709 13:12:56.833739 22760 rpcz_store.cc:276] 0709 13:12:54.099658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.818894 (+2719236us) service_pool.cc:224] Handling call
0709 13:12:56.819474 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.830091 (+ 10617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.830391 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.831995 26470 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.816569 22732 leader_election.cc:403] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:56.809820 22833 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:56.840231 25955 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:56.823467 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2298) took 2720 ms. Trace:
I20260709 13:12:56.850430 22766 rpcz_store.cc:276] 0709 13:12:54.102576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.821980 (+2719404us) service_pool.cc:224] Handling call
0709 13:12:56.822482 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.822899 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.822940 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.823077 (+   137us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:56.823126 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.823201 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.823369 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:56.853859 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 222396us
I20260709 13:12:56.854876 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:56.855168 23463 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:12:56.860814 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73814.6us
W20260709 13:12:56.863231 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48972.3us
I20260709 13:12:56.757004 26527 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:56.865077 25955 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:56.763551 26511 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:56.879659 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 261568us
W20260709 13:12:56.885680 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 213153us
I20260709 13:12:56.886948 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:12:56.889417 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.884585 22807 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:56.891350 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:56.892624 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 422) took 3122 ms. Trace:
I20260709 13:12:56.893477 22765 rpcz_store.cc:276] 0709 13:12:53.770318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.815167 (+3044849us) service_pool.cc:224] Handling call
0709 13:12:56.862927 (+ 47760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.892125 (+ 29198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.892170 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.892238 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:56.892288 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.892376 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.892537 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.872537 26413 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:56.913997 26434 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.914685 26434 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.915134 26434 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.916987 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.918381 22839 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:56.904670 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:56.889281 26447 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:12:56.911676 26384 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:56.881826 25955 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:56.930732 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.931625 26384 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.932065 26384 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.931406 25955 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.932701 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:56.913866 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44246.9us
I20260709 13:12:56.934665 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.934866 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:56.936299 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.937336 22832 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:56.930011 26447 consensus_queue.cc:1243] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:56.937709 26413 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:56.946853 22766 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:12:56.947589 22766 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 3
I20260709 13:12:56.947876 22766 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:12:56.948170 22766 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:56.949659 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.950582 22832 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 6: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:56.952261 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.953962 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.954310 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.954682 22838 raft_consensus.cc:2368] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:56.954391 22834 pending_rounds.cc:85] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:12:56.955478 22834 pending_rounds.cc:107] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:12:56.955811 22834 raft_consensus.cc:2889] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:56.956100 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:56.956925 22838 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:56.957850 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.960996 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1618) took 2934 ms. Trace:
I20260709 13:12:56.962010 22762 rpcz_store.cc:276] 0709 13:12:54.026682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.769891 (+2743209us) service_pool.cc:224] Handling call
0709 13:12:56.770394 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.960573 (+190179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.960921 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:56.963765 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 363074us
I20260709 13:12:56.922245 26527 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:56.996035 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1620) took 2892 ms. Trace:
I20260709 13:12:56.996459 22766 rpcz_store.cc:276] 0709 13:12:54.103720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.852079 (+2748359us) service_pool.cc:224] Handling call
0709 13:12:56.946280 (+ 94201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.995784 (+ 49504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.995966 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:56.975611 22832 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:56.978812 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.979434 26298 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:56.980767 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.982064 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.001940 22836 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:56.983618 26559 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:56.985191 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:57.006654 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.986750 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:56.986601 26343 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:56.987315 26250 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:56.991708 26466 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.009676 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:57.009770 22761 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:57.029320 23462 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:57.030025 22839 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:56.995245 26527 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.056766 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:57.058243 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 352705us
I20260709 13:12:57.068346 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1600) took 3416 ms. Trace:
W20260709 13:12:57.048045 26260 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.007s
I20260709 13:12:57.082827 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:57.085371 22839 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:57.096376 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 215766us
W20260709 13:12:57.099846 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 211862us
I20260709 13:12:57.100904 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:57.050643 26516 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:57.056624 26527 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:57.088778 26039 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.697s	user 0.015s	sys 0.001s
I20260709 13:12:57.076604 22764 rpcz_store.cc:276] 0709 13:12:53.652034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.624152 (+2972118us) service_pool.cc:224] Handling call
0709 13:12:56.649020 (+ 24868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.653088 (+  4068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.653120 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:56.655035 (+  1915us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.067810 (+412775us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:57.067876 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.067986 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:57.068053 (+    67us) raft_consensus.cc:1685] finished
0709 13:12:57.068071 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.068256 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":702,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":116,"thread_start_us":412214,"threads_started":1,"wal-append.queue_time_us":407167}
I20260709 13:12:57.119293 22763 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:57.124178 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.104803 26326 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:57.139575 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1597) took 3473 ms. Trace:
I20260709 13:12:57.141863 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.099494 26413 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:57.134289 26250 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:57.119592 26557 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:57.151132 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.081648 26396 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.091776 26260 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.106863 26498 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.167809 22839 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:57.166074 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 421) took 3396 ms. Trace:
W20260709 13:12:57.161309 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2292) took 3905 ms (client timeout 1599 ms). Trace:
I20260709 13:12:57.173249 26326 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.186970 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:57.178828 22758 rpcz_store.cc:276] 0709 13:12:53.769995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.667011 (+2897016us) service_pool.cc:224] Handling call
0709 13:12:56.667482 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.667928 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.667967 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:56.668583 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.053860 (+385277us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:57.053907 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.053986 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:57.165856 (+111870us) raft_consensus.cc:1685] finished
0709 13:12:57.165876 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.166011 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15566,"raft.queue_time_us":2518,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":13905,"thread_start_us":384815,"threads_started":1,"wal-append.queue_time_us":495210}
W20260709 13:12:57.178928 22757 rpcz_store.cc:269] 0709 13:12:53.255798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.348540 (+ 92742us) service_pool.cc:224] Handling call
0709 13:12:57.141181 (+3792641us) spinlock_profiling.cc:243] Waited 3.01 s on lock 0x7b5400221fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:57.161213 (+ 20032us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6929722240}
I20260709 13:12:57.138576 26466 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:12:57.197391 22837 raft_consensus.cc:2368] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8.
I20260709 13:12:57.156559 26557 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:57.198297 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.203715 22832 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:57.135246 26039 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.198202 22835 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:57.146257 26160 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.202075 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:57.219630 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99424.9us
I20260709 13:12:57.234154 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:57.234935 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1641) took 1074 ms. Trace:
I20260709 13:12:57.235482 22761 rpcz_store.cc:276] 0709 13:12:56.143710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.351338 (+207628us) service_pool.cc:224] Handling call
0709 13:12:57.218047 (+866709us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54002233d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:57.218201 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487075072}
I20260709 13:12:57.239519 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.240463 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:57.238513 22806 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.556s (SENT)
I20260709 13:12:57.241844 22836 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12.
I20260709 13:12:57.243876 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:57.245056 22839 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:57.254912 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:57.255733 22909 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:12:57.247674 22832 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:12:57.261072 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.256524 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.241837 26343 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:57.275978 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:57.279471 22913 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:57.286270 26570 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:57.211011 26413 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.297930 22911 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:57.296783 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:57.274499 26266 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.007s
I20260709 13:12:57.273247 26584 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:12:57.307483 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.308341 22838 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:57.309440 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.310443 22664 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 4 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:57.301642 26579 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:57.314731 26558 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:12:57.314172 26579 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.312641 22664 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:57.334082 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.296511 26039 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:57.318290 22840 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:12:57.361639 22912 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:57.363194 26260 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.293125 26569 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:57.368208 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 418) took 3634 ms. Trace:
I20260709 13:12:57.368659 22763 rpcz_store.cc:276] 0709 13:12:53.733919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.637379 (+2903460us) service_pool.cc:224] Handling call
0709 13:12:56.652466 (+ 15087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.120072 (+467606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.366584 (+246512us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54002dafd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:12:57.368131 (+  1547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073099520}
I20260709 13:12:57.293323 26302 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.356376 22664 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 6, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:57.317404 26579 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.352165 26584 consensus_queue.cc:1243] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:12:57.390533 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 225814us
I20260709 13:12:57.376147 26266 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:12:57.402549 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 301757us
I20260709 13:12:57.396006 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:57.305787 26250 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:57.437682 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 286636us
W20260709 13:12:57.439622 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 218974us
W20260709 13:12:57.440506 22806 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.556s (SENT)
I20260709 13:12:57.444417 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.400779 26498 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:57.424386 26396 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.438273 26148 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:57.487071 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83129.5us
I20260709 13:12:57.443864 22664 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:57.504333 26567 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.260s	user 0.000s	sys 0.002s
I20260709 13:12:57.505828 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2299) took 3402 ms. Trace:
W20260709 13:12:57.506757 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 417488us
I20260709 13:12:57.508292 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:57.510205 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69748.2us
I20260709 13:12:57.506242 22760 rpcz_store.cc:276] 0709 13:12:54.103212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.835230 (+2732018us) service_pool.cc:224] Handling call
0709 13:12:56.835862 (+   632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.858168 (+ 22306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.858222 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:56.865280 (+  7058us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.148995 (+283715us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:57.213933 (+ 64938us) log.cc:844] Serialized 14 byte log entry
0709 13:12:57.229873 (+ 15940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.229968 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:57.309914 (+ 79946us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b18003333a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:57.505443 (+195529us) raft_consensus.cc:1685] finished
0709 13:12:57.505590 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.505756 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3136,"prepare.queue_time_us":106949,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":35053,"raft.run_cpu_time_us":551,"raft.run_wall_time_us":27060,"spinlock_wait_cycles":12881664,"thread_start_us":330401,"threads_started":2,"wal-append.queue_time_us":328127}
I20260709 13:12:57.510793 26569 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:57.485647 26560 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:57.512769 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74135.9us
W20260709 13:12:57.531205 26434 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.615s	user 0.015s	sys 0.006s
I20260709 13:12:57.531737 26434 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.533944 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:57.533833 26573 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:57.518065 26569 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.535604 26573 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:57.536512 26573 raft_consensus.cc:2764] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:57.537242 26573 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:57.537650 26573 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:57.538223 26343 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:57.538635 26343 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:57.538960 26434 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.538575 26573 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.539774 26343 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.540349 26573 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:57.539947 26384 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.001s
W20260709 13:12:57.543015 26588 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:12:57.541931 26384 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.512534 26148 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:12:57.545979 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54366.4us
I20260709 13:12:57.528115 26302 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:57.541147 26343 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:57.546633 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:57.591688 22880 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:57.587715 22664 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:12:57.604740 26585 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.605690 26593 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:57.606297 26593 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:57.607209 26593 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.608929 26039 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:57.609333 26039 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.609687 26039 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:57.604830 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58572.9us
I20260709 13:12:57.616417 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:57.618831 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:57.619874 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:57.620592 23460 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:57.621060 26557 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.621621 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:57.622236 23461 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.622718 26557 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:57.615615 26384 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:57.637063 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 85280.1us
I20260709 13:12:57.646395 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:57.592530 26160 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:57.651954 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60373.7us
I20260709 13:12:57.654246 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.650874 22664 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 6, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:12:57.644192 26298 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.643s	user 0.014s	sys 0.009s
W20260709 13:12:57.662621 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 127245us
I20260709 13:12:57.645522 26498 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:57.636062 26585 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:57.673420 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.142495 22759 rpcz_store.cc:276] 0709 13:12:53.651319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.592630 (+2941311us) service_pool.cc:224] Handling call
0709 13:12:56.618226 (+ 25596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.618485 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:12:56.890292 (+271807us) log.cc:844] Serialized 14 byte log entry
0709 13:12:57.123953 (+233661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:57.124029 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:57.124090 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.124171 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.124311 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":468635,"raft.queue_time_us":2311,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5354,"thread_start_us":478353,"threads_started":2}
I20260709 13:12:57.668527 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:57.667094 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:57.670015 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.639s (SENT)
I20260709 13:12:57.686954 22806 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.651783 26578 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:57.696373 26302 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:57.703307 26447 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 13:12:57.708151 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53999.3us
W20260709 13:12:57.709120 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:57.725987 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.728773 23460 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:12:57.731285 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.670s (SENT)
I20260709 13:12:57.731843 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.732098 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:57.697232 26578 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.708544 26466 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.745395 22664 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:57.761832 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.726219 26302 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:57.706343 26600 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:57.722823 26447 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:57.717679 26148 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.712954 26585 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.736752 26396 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:57.743978 26298 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.745869 26578 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:57.799981 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.784379 26600 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.807492 26600 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.793995 26396 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:57.818490 26302 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.820688 26600 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.823653 26160 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:57.824903 26160 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.826339 26260 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:57.826822 26260 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:57.834204 26569 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.835461 26600 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:57.835381 26260 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.836647 26569 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:12:57.838325 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47389.7us
I20260709 13:12:57.840543 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:57.795979 26527 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.003s
W20260709 13:12:57.805719 26570 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.000s
I20260709 13:12:57.841809 26413 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.842229 26570 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:57.843852 26396 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.841809 26527 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.847853 26527 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:57.849532 23461 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:57.803642 26298 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.783390 26584 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:57.783638 26447 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:57.855795 26413 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:57.860800 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76634.4us
W20260709 13:12:57.863524 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 224228us
I20260709 13:12:57.789811 26039 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.864233 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.864334 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:57.868150 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 141716us
I20260709 13:12:57.870440 26426 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:57.871023 26426 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:57.872237 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.872530 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:57.873301 22906 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:57.873246 22764 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:57.786123 26609 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:57.871403 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.568s (SENT)
I20260709 13:12:57.878790 26426 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.879803 22909 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:57.880155 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.890270 22759 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:57.880831 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:57.881038 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:57.881718 26447 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.881982 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:57.894742 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:57.894972 22763 raft_consensus.cc:2376] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10. Re-sending same reply.
I20260709 13:12:57.881052 26589 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:57.880867 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:57.911399 22914 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.911015 22912 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:57.884825 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.912449 22765 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:57.872423 22664 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:57.919512 22911 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:12:57.922019 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50643.5us
W20260709 13:12:57.922814 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.586s (SENT)
I20260709 13:12:57.886286 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.882079 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:57.892129 22760 raft_consensus.cc:2376] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6. Re-sending same reply.
I20260709 13:12:57.923739 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.924131 22915 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:57.924499 22766 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:57.924636 22764 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:57.895131 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:57.881398 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:57.925779 22838 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:57.934052 26562 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:12:57.934127 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.937738 22757 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:12:57.939054 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72175.8us
I20260709 13:12:57.939769 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:57.940925 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:57.894433 26570 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:57.946269 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:57.945431 22806 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.657s (SENT)
I20260709 13:12:57.888666 26498 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:57.942765 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:57.950579 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:57.910768 26511 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:57.955066 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:57.958591 26498 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.945878 26585 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:12:57.962807 26585 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:57.963225 26585 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:57.964119 26585 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:57.965721 26447 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:57.969660 26511 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.973766 26511 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:57.978098 26562 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:57.981599 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1478) took 1068 ms. Trace:
I20260709 13:12:57.942755 26298 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:57.996220 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.002658 22914 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:12:58.004854 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64035.7us
I20260709 13:12:57.951714 22664 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.982030 22833 rpcz_store.cc:276] 0709 13:12:56.913712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.913862 (+   150us) service_pool.cc:224] Handling call
0709 13:12:56.914236 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:56.914732 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:57.090680 (+175948us) log.cc:844] Serialized 14 byte log entry
0709 13:12:57.383435 (+292755us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:57.507807 (+124372us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b180048a3c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:12:57.947286 (+439479us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:12:57.947728 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.947923 (+   195us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:57.947959 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.948035 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:57.981293 (+ 33258us) raft_consensus.cc:1685] finished
0709 13:12:57.981356 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.981523 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55975,"prepare.queue_time_us":228458,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":137634,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":10447,"spinlock_wait_cycles":49609344,"thread_start_us":410617,"threads_started":3,"wal-append.queue_time_us":192098}
I20260709 13:12:58.007534 22836 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:58.007622 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:58.007783 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:58.008857 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:12:58.009317 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63963us
W20260709 13:12:58.010283 22806 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.739s (SENT)
I20260709 13:12:58.003729 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:58.011648 23462 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.010759 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.009433 23461 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:58.075238 22837 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:58.075940 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.079439 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69206.7us
I20260709 13:12:58.076035 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:12:58.085701 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 143018us
I20260709 13:12:58.087466 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.081444 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:57.965464 26585 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.092487 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.107219 22764 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:58.108592 22730 leader_election.cc:400] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
W20260709 13:12:58.094362 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.206s (SENT)
W20260709 13:12:58.130299 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54601.5us
I20260709 13:12:58.135362 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.151670 23461 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:58.079972 26584 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:58.075335 26413 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:58.158630 22764 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:58.159374 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.169109 22763 raft_consensus.cc:2368] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:12:58.128780 26613 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:58.177306 26613 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.093044 26434 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:58.088106 26570 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:58.185842 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 98995us
I20260709 13:12:58.129007 26578 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.216609 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.222751 23462 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:58.223855 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:58.224323 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 129977us
I20260709 13:12:58.228308 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.230646 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.231514 22763 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 6.
I20260709 13:12:58.194684 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.229396 22909 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:58.190897 26250 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:58.197398 26557 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.241122 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:58.247462 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60756.9us
I20260709 13:12:58.248203 22909 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.246078 22906 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:58.249969 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.251348 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:58.245954 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.252949 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.254766 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.248550 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:58.270864 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:58.271713 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.272567 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.272574 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.278415 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.232656 26613 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:58.282711 22764 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:12:58.190480 26434 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.276437 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:58.288642 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.281638 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:58.289688 22806 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.966s (SENT)
I20260709 13:12:58.290247 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.279816 26250 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.292199 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:58.292480 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:58.292908 22913 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:58.293936 26562 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:58.294981 26434 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.295569 26434 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:58.295907 26434 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.294021 22913 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:58.291487 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.293399 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.297613 22907 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 9. Ops: []
W20260709 13:12:58.301644 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.306617 22906 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:12:58.311902 22912 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:58.312677 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:58.313885 22836 raft_consensus.cc:2410] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:12:58.272118 26593 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:58.321403 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126989us
I20260709 13:12:58.310900 26434 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:58.297008 26413 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:58.312685 26250 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:58.328987 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47473.2us
I20260709 13:12:58.363210 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.334707 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:58.373900 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.350116 22915 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:58.374949 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.375200 22764 raft_consensus.cc:2410] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:12:58.363595 22766 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:58.367865 22765 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:58.365914 22836 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:58.362526 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.373709 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:12:58.339232 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49589us
I20260709 13:12:58.378010 22763 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:58.379179 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.228s (SENT)
I20260709 13:12:58.328588 22908 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:58.379595 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.376744 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:58.374486 22757 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:58.390969 22909 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:12:58.333849 26302 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.421696 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:58.329363 26607 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:12:58.430627 22759 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:12:58.387296 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.386401 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:58.422482 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45785.2us
W20260709 13:12:58.330089 26618 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 13:12:58.429860 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50717.9us
I20260709 13:12:58.432446 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:58.433096 22912 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:58.346246 26426 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:58.427548 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54428.3us
I20260709 13:12:58.343354 26578 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:58.328374 26516 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:58.437342 22764 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:58.338323 26617 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:12:58.367465 26413 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:58.333261 26570 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.338748 26396 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:58.385682 26343 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:58.448390 22759 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:58.425835 26600 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.429179 26285 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:58.454926 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81441us
I20260709 13:12:58.489570 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.457700 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.462534 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.467365 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.492230 22760 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:58.468856 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.469700 26581 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:58.462895 26440 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:12:58.478552 26573 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.481405 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.491730 22764 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:58.447285 26039 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:58.386109 26560 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
W20260709 13:12:58.499996 26511 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.000s
I20260709 13:12:58.497674 22763 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:58.459594 26498 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:58.495230 26447 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.000s
I20260709 13:12:58.501993 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.511679 22765 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:58.460564 26413 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.480911 26589 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:58.520663 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.521718 22760 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:58.494652 26148 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:58.458549 26302 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.480914 26610 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:12:58.506584 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 531) took 1478 ms. Trace:
I20260709 13:12:58.488204 26343 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:58.516757 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 141880us
I20260709 13:12:58.524923 22764 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:58.527531 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.530416 22763 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.501042 26285 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:58.509413 26581 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:58.532028 22764 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:58.534824 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:12:58.536665 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47288.4us
I20260709 13:12:58.537310 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.510605 26498 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:58.523764 26039 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.524314 26589 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:58.541965 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.544274 22757 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:58.529014 22841 rpcz_store.cc:276] 0709 13:12:57.028165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.438003 (+409838us) service_pool.cc:224] Handling call
0709 13:12:57.489569 (+ 51566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:57.551625 (+ 62056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.551660 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:57.600617 (+ 48957us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.861825 (+261208us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:57.861890 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.862919 (+  1029us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:58.506326 (+643407us) raft_consensus.cc:1685] finished
0709 13:12:58.506352 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:58.506518 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18424,"raft.queue_time_us":47830,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"thread_start_us":197216,"threads_started":1,"wal-append.queue_time_us":435839}
I20260709 13:12:58.516449 26511 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.545567 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.531517 26447 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.506271 26573 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.547320 22730 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12.
W20260709 13:12:58.546105 22806 leader_election.cc:336] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.558s (SENT)
I20260709 13:12:58.553412 22730 leader_election.cc:403] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:12:58.548448 22759 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:58.536064 26343 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.541960 22764 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.555289 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.552531 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:58.525740 26250 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:12:58.519495 26622 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:12:58.558706 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.543514 26498 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:58.542472 26623 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.564072 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.565145 22759 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.534829 26516 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:12:58.530733 26148 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.565796 26285 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.569387 26609 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:58.569754 26148 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.569916 26609 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.570377 26609 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.575100 22764 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:58.575846 22757 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:58.577193 22759 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:58.582505 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.582962 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.556711 26613 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:58.584256 22763 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:12:58.584911 22761 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:12:58.559089 26302 raft_consensus.cc:2764] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:12:58.590425 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.591748 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.592098 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:12:58.596917 26616 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.342s	user 0.001s	sys 0.001s
I20260709 13:12:58.598479 26434 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:58.600034 26621 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:12:58.601902 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:58.556684 26447 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.607932 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:58.611343 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64111.4us
I20260709 13:12:58.588714 26039 raft_consensus.cc:2764] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:58.630627 26302 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:58.614540 26298 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:12:58.606864 26581 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.632053 22758 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:58.632287 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.592774 26623 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.635289 22766 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9. Re-sending same reply.
I20260709 13:12:58.603099 26570 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:58.600780 26434 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.641536 26631 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:58.653770 26631 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:58.654793 26631 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.657748 22764 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:58.643208 26466 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.874s	user 0.008s	sys 0.005s
I20260709 13:12:58.661428 26466 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.647289 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:58.665926 26623 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:58.666651 26623 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:58.667104 26623 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.642364 26573 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.650885 26557 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.659125 22765 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:58.649150 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.673789 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:58.678587 22908 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:58.678782 22910 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.675595 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.679776 22764 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:58.680998 22759 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:58.680913 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.683219 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.683959 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.684695 22764 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.688520 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:58.689020 22762 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:58.697036 22764 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:58.698688 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:58.700232 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 232677us
W20260709 13:12:58.700961 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.702023 22766 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:12:58.693221 26585 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.581s	user 0.003s	sys 0.012s
I20260709 13:12:58.699272 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.736292 22757 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:58.718865 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.713061 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.140s (SENT)
I20260709 13:12:58.727283 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.737112 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.737299 22765 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:58.738272 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.738974 22765 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:58.741030 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:58.726907 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.732060 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.741936 22762 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:58.742087 22759 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:58.733778 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 118349us
I20260709 13:12:58.743253 22730 leader_election.cc:400] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8.
I20260709 13:12:58.743561 22730 leader_election.cc:403] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:58.744498 26573 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8.
I20260709 13:12:58.744832 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:58.746516 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 139220us
I20260709 13:12:58.749706 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:58.750522 23462 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:58.751305 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:58.753400 22764 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:58.708905 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:58.758221 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 148554us
I20260709 13:12:58.758451 22840 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:58.758708 22908 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:58.759375 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:58.759790 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:58.760119 22833 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:12:58.735201 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 302833us
I20260709 13:12:58.760339 22839 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.760591 22840 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:58.757195 22765 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
W20260709 13:12:58.757108 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.757402 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.761896 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:58.762425 22765 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.764215 23462 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:58.698088 26610 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:58.769610 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.770433 26638 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.772686 22839 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:12:58.773171 22765 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:58.773065 26638 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:12:58.774387 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61395us
W20260709 13:12:58.814673 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.775238 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:58.815268 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62099.3us
I20260709 13:12:58.775878 26640 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.777113 26623 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:12:58.777943 26626 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:12:58.782763 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:58.785681 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.824739 22759 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:58.786242 26624 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:58.789245 22762 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:12:58.789769 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:58.828461 23462 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:12:58.791359 22760 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:12:58.797264 22835 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:58.798488 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:12:58.802130 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3701094060000969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
I20260709 13:12:58.820770 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:58.834630 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.835788 22732 leader_election.cc:400] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 7. Current term is 8.
I20260709 13:12:58.837530 22760 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:12:58.835888 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75188.6us
I20260709 13:12:58.726826 26250 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:12:58.844702 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:12:58.846324 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:58.850827 23458 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:58.850557 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:58.851890 22766 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:58.854018 23462 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:58.817245 26610 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:58.857880 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.859745 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:58.860656 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:12:58.821933 26632 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:58.837951 26623 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:58.830827 26624 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:58.834344 26630 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12.
I20260709 13:12:58.851657 26511 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:58.862447 26579 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:58.889187 22910 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:58.890914 23458 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:12:58.892138 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54010.5us
I20260709 13:12:58.847689 26622 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:12:58.863893 26039 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:12:58.881915 26624 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.906528 26426 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:58.907213 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.884351 26637 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.916203 26637 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.917922 26396 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:58.920871 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:58.895548 26579 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:12:58.908859 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 110474us
I20260709 13:12:58.887143 26632 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:58.914387 26610 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.918347 26396 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:58.936856 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104561us
I20260709 13:12:58.921270 26562 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:58.897114 26413 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.899724 26581 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:12:58.945612 26581 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:58.945663 26605 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:58.946170 26581 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:58.946569 26637 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.947067 26581 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.954649 26343 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.955330 26413 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.956373 26413 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.957201 26343 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:58.925606 26584 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:58.927246 26579 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.944474 22763 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:12:58.945276 26632 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.934428 26560 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:58.920475 26447 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:58.933605 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75829.9us
I20260709 13:12:58.941380 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:58.959563 26498 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:58.974941 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:58.939970 26562 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.988125 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:58.991921 26654 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:58.999914 26654 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:58.971096 26581 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.001984 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:59.001619 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.011300 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.012743 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.014837 26654 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:58.992892 26285 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:58.993449 26562 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.000537 26447 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.039364 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:59.023937 26600 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:59.051340 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1622) took 1056 ms. Trace:
I20260709 13:12:59.052043 22835 rpcz_store.cc:276] 0709 13:12:57.994393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.996814 (+  2421us) service_pool.cc:224] Handling call
0709 13:12:59.051029 (+1054215us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400038ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:12:59.051241 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529917696}
I20260709 13:12:59.026942 26426 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:59.023953 26285 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:59.024121 26584 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:59.069401 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67495.4us
I20260709 13:12:59.071182 22732 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:59.071676 22732 leader_election.cc:403] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:12:59.072575 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.531s (SENT)
I20260709 13:12:59.055894 26600 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:59.087308 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.087636 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:59.089913 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.069537 26285 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.043581 26636 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 13:12:59.096652 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 107455us
I20260709 13:12:59.069728 26640 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:59.075444 26645 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:59.094492 26661 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.109s	user 0.000s	sys 0.002s
I20260709 13:12:59.041618 26589 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.115751 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.074504 26584 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:59.109764 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:59.093822 22757 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:59.117039 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:12:59.118585 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.112421 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:59.112634 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.119982 22766 raft_consensus.cc:2376] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9. Re-sending same reply.
I20260709 13:12:59.120446 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.089300 26600 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.113472 22909 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 13:12:59.122602 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:12:59.137477 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64958.4us
I20260709 13:12:59.109602 26434 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.067128 26396 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.110589 26651 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:59.156061 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.156810 22766 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 6.
I20260709 13:12:59.153015 22906 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:59.138098 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:12:59.139070 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.160024 22757 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:59.146410 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.161906 22763 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:59.139385 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.543s (SENT)
I20260709 13:12:59.164834 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:59.139932 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:59.154131 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.140467 22907 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:59.145411 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.145777 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.173748 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:59.180615 22836 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.172420 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:59.180198 22908 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:12:59.147229 22730 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
W20260709 13:12:59.184544 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67576.6us
I20260709 13:12:59.185600 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.185717 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:12:59.186717 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.187839 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:59.188496 22909 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:59.187877 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48661.3us
W20260709 13:12:59.189205 23431 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.582s (SENT)
I20260709 13:12:59.189638 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:59.194151 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47101.1us
I20260709 13:12:59.155642 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:59.207005 22915 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:59.207067 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 69086.8us
I20260709 13:12:59.208097 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:59.210394 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:59.148674 26466 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:59.141665 26631 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.216609 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:59.123273 26584 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.150935 26396 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.165668 26413 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:12:59.205678 26638 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:59.205892 26384 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:59.250370 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.115644 26645 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.219677 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:12:59.122656 26557 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:12:59.227002 22909 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:12:59.251976 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.239156 22763 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:12:59.181716 26148 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:59.137535 26659 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:59.258319 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.150079 26434 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.272253 22766 raft_consensus.cc:2376] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7. Re-sending same reply.
W20260709 13:12:59.272253 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52653.6us
W20260709 13:12:59.263751 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61021.8us
I20260709 13:12:59.218039 26654 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:59.219080 26039 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:59.271242 22835 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:59.218291 26653 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.269835 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.218773 26605 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:59.279299 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:59.249758 26466 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:12:59.319561 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.322196 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.266222 26384 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:12:59.341487 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:59.282228 26569 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:59.216202 26660 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
W20260709 13:12:59.348311 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65894us
I20260709 13:12:59.365901 22763 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:59.232111 26651 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:12:59.340404 26384 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.375116 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.378324 22763 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:59.341751 26039 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:59.381716 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.390352 22763 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
W20260709 13:12:59.392813 26667 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:12:59.393141 26656 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:12:59.394670 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.396234 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.396857 22763 raft_consensus.cc:2376] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9. Re-sending same reply.
I20260709 13:12:59.398016 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:12:59.401633 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125150us
I20260709 13:12:59.402029 22766 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:12:59.403404 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:12:59.405435 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:59.381433 26447 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:59.424543 22763 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:59.426564 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.427492 22766 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12.
I20260709 13:12:59.322806 26653 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.388772 26569 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:59.379101 26637 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:12:59.382045 26573 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:12:59.442590 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.369464 26624 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.373073 26655 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.309s
I20260709 13:12:59.406723 26650 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:59.470005 26650 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.472668 26650 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:59.156487 26560 consensus_queue.cc:1175] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "acfda9b350734def9b6c621f66a5f247" responder_term: 7 status { last_received { term: 6 index: 4 } last_committed_idx: 3 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:59.439182 26659 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:59.476161 26624 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:59.475888 26659 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.478137 26669 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:59.479387 26669 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:59.480019 26669 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.482025 26655 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:59.482656 26670 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:59.483331 26670 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:12:59.380088 26676 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:12:59.483690 26670 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.484210 26676 consensus_queue.cc:1243] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:59.484786 26676 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:59.485065 26645 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:59.485718 26674 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:12:59.485948 26645 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:59.486443 26645 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:12:59.487813 26675 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:12:59.469291 26039 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:59.489655 26673 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:12:59.489861 22765 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:12:59.482730 26655 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:59.447280 26557 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.463316 22763 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:12:59.483759 22761 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:12:59.483412 26627 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.925s	user 0.000s	sys 0.013s
W20260709 13:12:59.501189 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 95837.2us
I20260709 13:12:59.509899 26692 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.510885 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.510640 26692 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.511595 22908 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:12:59.511912 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.513381 22909 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:59.513876 26636 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:59.514513 26692 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:12:59.509891 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 230683us
I20260709 13:12:59.501636 26447 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:59.535202 26655 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.516189 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:59.466566 26589 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:59.561870 22836 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:12:59.485105 26676 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:12:59.487430 26645 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:59.570083 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53989.9us
W20260709 13:12:59.619333 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.623960 22761 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:12:59.624606 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 164786us
I20260709 13:12:59.439141 26466 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.636443 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:12:59.637066 22761 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:12:59.614710 26609 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.044s	user 0.013s	sys 0.002s
I20260709 13:12:59.564136 26426 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:12:59.638322 26663 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:12:59.629164 26692 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.635499 26683 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:12:59.642496 26434 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:59.645296 26645 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.660694 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2447) took 1789 ms. Trace:
W20260709 13:12:59.663502 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 110715us
I20260709 13:12:59.666733 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.672112 26584 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:59.674216 26671 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
I20260709 13:12:59.705437 26498 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:12:59.705768 26676 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.708292 26584 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:12:59.682315 26426 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:59.716593 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49937.5us
I20260709 13:12:59.718330 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:12:59.683311 26691 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:12:59.688838 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.694468 26260 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.664201 26683 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:59.661098 26579 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.684s	user 0.013s	sys 0.000s
I20260709 13:12:59.645745 26609 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.662432 22914 rpcz_store.cc:276] 0709 13:12:57.871440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.073866 (+202426us) service_pool.cc:224] Handling call
0709 13:12:58.074444 (+   578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:58.155886 (+ 81442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:58.155929 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:58.187455 (+ 31526us) log.cc:844] Serialized 26 byte log entry
0709 13:12:58.373991 (+186536us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:58.374036 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.374104 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:59.483076 (+1108972us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b2400205388. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea4309d 00007f68a167510e 00007f68a168c88c 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221
0709 13:12:59.655784 (+172708us) raft_consensus.cc:1685] finished
0709 13:12:59.655810 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:59.660609 (+  4799us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55985,"raft.queue_time_us":106149,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7364,"spinlock_wait_cycles":16711424,"thread_start_us":126767,"threads_started":1,"wal-append.queue_time_us":302334}
I20260709 13:12:59.760807 22832 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
W20260709 13:12:59.734894 22915 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
W20260709 13:12:59.736902 26665 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:12:59.738345 26653 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:12:59.742259 26148 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:59.632102 26569 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:12:59.758311 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.797366 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.755671 26631 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.003s
W20260709 13:12:59.679915 26562 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.000s
I20260709 13:12:59.761986 22915 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:12:59.800812 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:59.803869 22912 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:12:59.804313 22761 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:12:59.804839 22761 pending_rounds.cc:85] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 4
I20260709 13:12:59.805140 22761 pending_rounds.cc:107] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:12:59.805437 22761 raft_consensus.cc:2889] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:59.767266 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:59.772902 26298 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:59.810195 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1671) took 1095 ms. Trace:
I20260709 13:12:59.787154 26584 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:12:59.754585 26285 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 13:12:59.755177 26637 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:59.787326 26668 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.821357 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:59.821354 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:59.814004 26686 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.811069 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:59.810657 22760 rpcz_store.cc:276] 0709 13:12:58.714321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.848305 (+133984us) service_pool.cc:224] Handling call
0709 13:12:58.872214 (+ 23909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:59.167622 (+295408us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:59.232692 (+ 65070us) log.cc:844] Serialized 14 byte log entry
0709 13:12:59.617595 (+384903us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:59.618144 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:12:59.618350 (+   206us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:59.618669 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:12:59.803443 (+184774us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b18003ab280. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:12:59.803651 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:59.803736 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:59.809942 (+  6206us) raft_consensus.cc:1685] finished
0709 13:12:59.809965 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:59.810130 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84758,"prepare.queue_time_us":64984,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3832,"raft.queue_time_us":381765,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":12049,"spinlock_wait_cycles":3647232,"thread_start_us":599149,"threads_started":3,"wal-append.queue_time_us":136071}
I20260709 13:12:59.794755 26148 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:59.817005 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:59.817413 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:59.765599 26384 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:12:59.775632 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57408.5us
I20260709 13:12:59.786727 26579 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.806933 26343 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:59.833197 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:12:59.824903 26581 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.815s	user 0.010s	sys 0.003s
W20260709 13:12:59.834483 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 205602us
I20260709 13:12:59.839541 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.817183 26498 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:59.847270 22915 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:59.848817 22834 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:59.843513 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:59.846392 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:59.850070 22840 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.809633 26562 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.848742 22839 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:59.841794 23431 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:59.853574 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:12:59.855662 22914 raft_consensus.cc:2376] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9. Re-sending same reply.
I20260709 13:12:59.855114 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:12:59.857306 22910 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:59.816609 26250 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:12:59.834024 26637 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:12:59.857200 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.835450 26632 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:59.903419 22758 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:59.867691 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:12:59.894660 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52988.7us
I20260709 13:12:59.905730 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:12:59.906028 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:12:59.823807 26285 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.906713 22914 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:12:59.907182 23431 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12.
W20260709 13:12:59.895053 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 380111us
I20260709 13:12:59.907625 22913 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:12:59.908445 22838 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:59.821573 26584 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:12:59.908452 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:12:59.907534 23431 leader_election.cc:403] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:12:59.854029 26600 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.000s
I20260709 13:12:59.904738 22841 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:59.910493 22765 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:59.928596 22910 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:59.848398 26298 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:59.929113 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:12:59.935196 22835 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:12:59.894214 26581 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:59.826011 26668 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:12:59.894286 26498 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:59.944267 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:59.945086 22835 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.838618 26326 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.088s
W20260709 13:12:59.934492 26665 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:12:59.951139 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:12:59.956972 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:12:59.931666 26632 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:12:59.940140 26581 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.830265 26686 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:00.028203 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.936288 26600 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.057475 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.058733 22841 raft_consensus.cc:2368] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:00.063841 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:00.064759 22760 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:59.977576 26581 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.071797 26628 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:00.072321 26678 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:00.073288 26677 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.073863 26688 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:00.074141 26384 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:00.030314 26653 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12.
W20260709 13:13:00.050706 26665 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:13:00.135449 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 228384us
I20260709 13:13:00.080727 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.115748 22833 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:59.959645 26298 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 13:13:00.140089 22835 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
W20260709 13:13:00.143682 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 205164us
I20260709 13:13:00.084481 26589 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.147382 22760 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:12:59.957860 26148 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.151166 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:00.147969 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:00.153542 22760 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:00.139528 26670 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.003s
I20260709 13:13:00.122747 26655 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:00.137665 26688 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:00.122018 26628 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:00.221424 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.002s	sys 0.006s
I20260709 13:13:00.223968 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1669) took 1502 ms. Trace:
I20260709 13:13:00.218670 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:00.150002 26678 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:00.162213 26285 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.138010 26677 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.221928 26683 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:00.236444 26466 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.020s	sys 0.047s
W20260709 13:13:00.247113 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.242023 22762 rpcz_store.cc:276] 0709 13:12:58.713757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.827138 (+113381us) service_pool.cc:224] Handling call
0709 13:12:58.827544 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:59.003415 (+175871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:59.003454 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:59.004088 (+   634us) log.cc:844] Serialized 52 byte log entry
0709 13:12:59.154827 (+150739us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:59.217365 (+ 62538us) log.cc:844] Serialized 14 byte log entry
0709 13:12:59.336873 (+119508us) log.cc:844] Serialized 14 byte log entry
0709 13:12:59.345131 (+  8258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:59.345229 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:00.215611 (+870382us) raft_consensus.cc:1685] finished
0709 13:13:00.215644 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.215879 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28733,"prepare.queue_time_us":81565,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":361679,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":30902,"thread_start_us":395454,"threads_started":3,"wal-append.queue_time_us":404080}
I20260709 13:13:00.113271 26637 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.225486 26579 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:00.261005 26703 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:13:00.268347 22762 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:00.266589 26466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.020s	sys 0.047s
I20260709 13:13:00.166903 26589 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.238037 26148 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:00.285712 22762 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:00.287498 26653 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:00.288264 26664 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:00.288836 26664 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:00.288918 22906 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:00.289893 22906 raft_consensus.cc:606] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:00.290258 22906 raft_consensus.cc:614] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:00.290342 26664 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.292621 22763 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:00.294027 22763 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:00.266729 26708 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.294835 26589 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:00.295241 26708 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.302498 26678 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:00.311856 26697 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.250s	user 0.000s	sys 0.002s
I20260709 13:13:00.267362 26584 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.315047 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:00.316406 22907 raft_consensus.cc:606] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:00.218020 26670 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.319358 26670 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.321836 26706 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:00.322811 26706 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:00.323302 26706 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.326009 26628 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.342082 26600 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.344300 26677 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.259869 26578 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:00.226701 26688 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:00.148460 26498 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.150578 26632 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:00.362411 22840 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 13:13:00.386910 22840 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
W20260709 13:13:00.348161 26396 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.077s	user 0.008s	sys 0.006s
W20260709 13:13:00.355469 26669 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.875s	user 0.005s	sys 0.009s
I20260709 13:13:00.355593 26578 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.359246 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:00.400031 22910 raft_consensus.cc:606] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:00.363126 26708 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:00.321152 26686 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:00.226816 26655 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:00.286640 26700 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 13:13:00.246065 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.002s	sys 0.007s
I20260709 13:13:00.405438 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:00.405900 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 765892us
W20260709 13:13:00.406553 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 265684us
W20260709 13:13:00.408071 26557 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.905s	user 0.013s	sys 0.000s
I20260709 13:13:00.416971 26639 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.397867 26562 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:00.445142 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:00.446259 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.388562 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:00.461225 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:00.462459 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57259.8us
I20260709 13:13:00.463588 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:00.467489 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:00.468515 23461 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:00.472325 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:00.432304 26600 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.428226 26557 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:00.475806 23463 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:00.403678 26669 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.477938 23458 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:00.441856 26660 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:00.482477 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:00.476509 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:00.490149 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:00.491842 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:00.493974 26683 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:00.441709 26599 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.434123 26639 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.510980 22837 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:00.499119 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:00.516827 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:00.497836 26645 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:00.498140 26630 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:00.529681 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:00.531646 22910 raft_consensus.cc:606] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:00.474680 26562 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:13:00.536070 23461 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:00.504798 26677 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:00.509195 26637 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:00.487463 26600 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.525620 26683 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:00.582180 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 442) took 1915 ms. Trace:
I20260709 13:13:00.574237 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1646) took 1295 ms. Trace:
I20260709 13:13:00.584254 22833 rpcz_store.cc:276] 0709 13:12:59.278749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.278892 (+   143us) service_pool.cc:224] Handling call
0709 13:12:59.320657 (+ 41765us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:00.140291 (+819634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.573782 (+433491us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b54001ff4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:00.574125 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1761706880}
W20260709 13:13:00.586755 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 438870us
I20260709 13:13:00.583159 22912 rpcz_store.cc:276] 0709 13:12:58.667206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:58.668472 (+  1266us) service_pool.cc:224] Handling call
0709 13:13:00.569066 (+1900594us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001e3cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:00.582090 (+ 13024us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563166080}
I20260709 13:13:00.558162 26683 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.550246 26660 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:00.616091 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:00.550544 26677 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:00.617023 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.522169 26599 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.631449 26600 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.631594 26599 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.632325 26578 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:00.633102 26717 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:13:00.635819 26710 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:00.636438 26710 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:00.638584 26721 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:13:00.639695 22912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:00.640384 22912 raft_consensus.cc:606] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:00.639982 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:00.586242 26586 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:00.647828 26692 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.005s
I20260709 13:13:00.648339 26692 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:00.648092 26586 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:00.648806 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.650610 26586 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.652967 26669 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.564509 26384 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.654110 26384 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.654805 26639 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.656888 22840 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
W20260709 13:13:00.661334 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:00.662295 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.650549 26664 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:00.667394 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:00.668184 22836 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:00.672967 23461 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:00.671739 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:00.683001 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.618458 26683 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:00.681331 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:00.618784 26498 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:00.686194 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 174700us
I20260709 13:13:00.689689 22836 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:00.621901 26677 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:00.694520 26723 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:00.702656 26669 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:00.703419 26669 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.707141 26715 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.707868 26715 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.708811 26632 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:00.709395 26632 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.709844 26711 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.558420 26637 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.712196 22909 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:00.713562 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:00.714814 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 198143us
I20260709 13:13:00.718756 23466 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:00.720149 22730 leader_election.cc:400] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:00.720988 22730 leader_election.cc:403] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:00.719946 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.722136 22836 raft_consensus.cc:2410] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:13:00.730424 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:00.732707 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.762012 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:00.803117 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:00.729115 26527 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.804785 26527 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.806572 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:00.808166 23459 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:00.717331 26498 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.710413 26711 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.746771 26573 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:00.680045 26706 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.819898 26706 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.820588 26326 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.820211 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.821126 26326 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.823470 26326 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:00.827023 26632 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:00.827843 26655 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.688136 26710 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:00.803807 26609 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:00.831590 26637 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:00.763971 26600 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:13:00.838035 26498 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:00.839300 26669 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:00.839563 26581 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.002s
I20260709 13:13:00.813803 26686 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:00.841039 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:00.846350 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 126541us
I20260709 13:13:00.847502 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:00.855432 23460 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:00.840149 26581 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:00.868656 22730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.556s (SENT) [suppressed 44 similar messages]
W20260709 13:13:00.880564 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 88869us
I20260709 13:13:00.880764 22836 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:00.853302 26599 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:00.863765 26655 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:00.848703 26686 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.889528 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:00.864212 26586 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:00.909425 23460 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:13:00.913822 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1685) took 1002 ms. Trace:
I20260709 13:13:00.914928 26669 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:00.915814 26669 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.917707 26728 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.918291 26728 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:00.915125 26600 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.914307 22761 rpcz_store.cc:276] 0709 13:12:59.911715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.343325 (+431610us) service_pool.cc:224] Handling call
0709 13:13:00.343717 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:00.424403 (+ 80686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:00.424454 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:00.454246 (+ 29792us) log.cc:844] Serialized 26 byte log entry
0709 13:13:00.664634 (+210388us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:00.664683 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.664762 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:00.913580 (+248818us) raft_consensus.cc:1685] finished
0709 13:13:00.913603 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:00.913755 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96776,"raft.queue_time_us":79139,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":13385,"thread_start_us":156554,"threads_started":1,"wal-append.queue_time_us":257479}
I20260709 13:13:00.900794 26715 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:00.892349 26711 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:00.984264 26731 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.115s	user 0.002s	sys 0.003s
I20260709 13:13:00.986469 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1502) took 1187 ms. Trace:
W20260709 13:13:00.956004 26688 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.599s	user 0.014s	sys 0.000s
I20260709 13:13:00.990103 22834 rpcz_store.cc:276] 0709 13:12:59.799219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.800403 (+  1184us) service_pool.cc:224] Handling call
0709 13:13:00.986210 (+1185807us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b54000ec4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:00.986394 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96561408}
I20260709 13:13:00.992799 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:00.994297 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:00.996515 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:00.997444 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1515) took 1156 ms. Trace:
I20260709 13:13:01.001701 22836 rpcz_store.cc:276] 0709 13:12:59.840803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.727495 (+886692us) service_pool.cc:224] Handling call
0709 13:13:00.997218 (+269723us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b54000c76d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:00.997371 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134646016}
I20260709 13:13:01.002875 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.004021 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.004799 22836 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:00.998773 26669 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:00.999253 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.006250 22835 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:01.007668 22835 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:13:01.007995 22835 pending_rounds.cc:85] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 7
I20260709 13:13:01.008217 22835 pending_rounds.cc:107] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:13:01.008462 22835 raft_consensus.cc:2889] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.009126 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1512) took 1200 ms. Trace:
I20260709 13:13:01.000701 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.009864 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:01.010135 26711 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.010910 26711 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.012531 22906 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:01.013321 22836 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.013597 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:01.014132 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1510) took 1208 ms. Trace:
I20260709 13:13:01.014695 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:01.016181 22909 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:01.016273 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.015045 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.017230 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.011091 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:01.018124 22912 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:01.017390 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:00.956225 26696 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:01.014451 22836 rpcz_store.cc:276] 0709 13:12:59.806130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.005353 (+1199223us) service_pool.cc:224] Handling call
0709 13:13:01.013152 (+  7799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.013912 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.014081 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:01.013597 22835 rpcz_store.cc:276] 0709 13:12:59.808413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.006899 (+1198486us) service_pool.cc:224] Handling call
0709 13:13:01.007200 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.008906 (+  1706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.009067 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:01.021109 22909 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:01.019960 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.024526 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:01.025799 26640 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:01.026315 26640 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:01.027185 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 451) took 1883 ms. Trace:
I20260709 13:13:01.027503 26640 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.027614 22765 rpcz_store.cc:276] 0709 13:12:59.143321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.490668 (+347347us) service_pool.cc:224] Handling call
0709 13:12:59.490993 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:59.931095 (+440102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.026887 (+1095792us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54001b9c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.027118 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964295040}
I20260709 13:13:01.032066 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.032615 22760 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:13:01.032730 22763 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:01.032882 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.033797 22762 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:01.035643 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 167019us
I20260709 13:13:01.035598 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.039350 22836 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:13:01.039712 22836 pending_rounds.cc:85] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 7
I20260709 13:13:01.039903 22836 pending_rounds.cc:107] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:13:01.040107 22836 raft_consensus.cc:2889] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.043458 22834 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.046880 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1519) took 1076 ms. Trace:
I20260709 13:13:01.047223 22836 rpcz_store.cc:276] 0709 13:12:59.970739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.031392 (+1060653us) service_pool.cc:224] Handling call
0709 13:13:01.039089 (+  7697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.046693 (+  7604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.046834 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:01.047276 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:00.922055 26586 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.046252 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:00.979012 26599 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.061275 22906 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:01.074608 22763 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:01.064867 22836 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:00.958153 26600 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.075826 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.078598 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.080188 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.082178 22833 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:01.087472 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.091504 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:01.092558 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:01.094425 26730 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:01.094699 26660 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:01.095247 26660 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.095589 26660 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:01.096745 26639 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:01.097368 26639 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.097831 26639 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.103031 22906 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:13:01.048236 26696 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:01.107314 26696 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.108974 22762 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:01.110080 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.111261 22762 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:01.115542 26586 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.120553 26677 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.123435 26599 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.124606 26581 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.045747 26413 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.149304 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.131501 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.153147 22762 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:01.116904 26660 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.069245 26715 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.162864 26715 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.164182 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.165112 26715 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:01.148309 26396 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:01.166522 26396 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:01.174870 22912 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:01.142424 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.023064 26688 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.148937 26413 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.178913 26413 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.045217 26655 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:01.179672 26655 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.150287 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:01.180467 23465 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.074770 26711 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:01.161744 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.161551 26683 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.181599 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.182358 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.183866 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:01.185962 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75235.1us
I20260709 13:13:01.185380 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:01.139302 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 176337us
I20260709 13:13:01.156102 26650 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.188246 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.187593 22763 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:01.191725 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:01.198422 22762 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:13:01.185914 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1518) took 1237 ms. Trace:
I20260709 13:13:01.203296 22841 rpcz_store.cc:276] 0709 13:12:59.948132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.010783 (+1062651us) service_pool.cc:224] Handling call
0709 13:13:01.011093 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.011310 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:01.011902 (+   592us) log.cc:844] Serialized 14 byte log entry
0709 13:13:01.185543 (+173641us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:01.185603 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:01.185655 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.185729 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:01.185848 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":856,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":1931,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":4477,"thread_start_us":172486,"threads_started":1}
W20260709 13:13:01.206362 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 113883us
I20260709 13:13:01.286595 26676 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:01.270656 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.420572 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:01.271340 26384 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.422541 22841 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.271692 26560 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.152s
W20260709 13:13:01.377714 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.213732 26660 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.273732 26686 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.261122 22836 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:01.223333 26728 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.273267 26396 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.244069 26688 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.213299 23457 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:01.428468 26396 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:01.416509 26613 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:01.428764 23457 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:01.428871 26688 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.303267 26586 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:01.414968 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.417271 26039 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:01.429199 26613 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.213116 26599 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.213s
I20260709 13:13:01.292339 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.286724 23466 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:01.195473 26650 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.414965 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:01.274394 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1618) took 3307 ms (client timeout 1544 ms). Trace:
I20260709 13:13:01.199909 26737 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:01.430020 26039 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:01.437712 22832 rpcz_store.cc:269] 0709 13:12:57.950145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.950288 (+   143us) service_pool.cc:224] Handling call
0709 13:13:01.256990 (+3306702us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54000a4e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:01.257213 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4140081408}
I20260709 13:13:01.456067 22832 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:01.430704 23457 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.455565 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.458714 23457 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:01.467887 22832 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:01.470475 22832 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.290048 26581 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.433224 26599 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:01.431529 26613 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.455335 26737 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.458648 26728 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:01.453297 26655 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.483474 22832 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:01.481587 26599 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:01.473258 26711 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.473253 26660 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:01.430104 26586 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.429481 26688 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.273105 26637 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.209s
I20260709 13:13:01.479652 26677 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.486114 26737 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:01.483309 26581 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.504906 26581 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.492604 26728 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:01.512151 26728 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.517364 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.495715 26683 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.522826 26640 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:01.525321 26650 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.495684 26711 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.528229 26711 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.529644 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1645) took 2254 ms. Trace:
I20260709 13:13:01.529940 26600 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.530241 22838 rpcz_store.cc:276] 0709 13:12:59.275159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.275366 (+   207us) service_pool.cc:224] Handling call
0709 13:12:59.321907 (+ 46541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:59.908839 (+586932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.529146 (+1620307us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54000c80d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.529487 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348405632}
I20260709 13:13:01.531787 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:01.532866 26701 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.472s	user 0.005s	sys 0.000s
I20260709 13:13:01.533341 26639 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:01.535868 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.536161 26639 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.537104 22757 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:01.501154 26660 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.541479 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 452) took 1701 ms. Trace:
I20260709 13:13:01.541980 26660 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.542661 26660 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.544063 22760 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:01.545442 26599 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.546705 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2367) took 1924 ms. Trace:
I20260709 13:13:01.547236 26599 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.547154 22766 rpcz_store.cc:276] 0709 13:12:59.622440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.623884 (+  1444us) service_pool.cc:224] Handling call
0709 13:12:59.638633 (+ 14749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:59.796958 (+158325us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:59.812724 (+ 15766us) log.cc:844] Serialized 14 byte log entry
0709 13:12:59.945864 (+133140us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:59.945922 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:59.945980 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.545569 (+1599589us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b1800483100. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419
0709 13:13:01.546421 (+   852us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540033fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.546478 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:01.546628 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18153,"prepare.queue_time_us":18223,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":18545,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":8612,"spinlock_wait_cycles":366835328,"thread_start_us":114586,"threads_started":1,"wal-append.queue_time_us":237175,"wal-append.run_cpu_time_us":2581,"wal-append.run_wall_time_us":1274732}
I20260709 13:13:01.542115 22911 rpcz_store.cc:276] 0709 13:12:59.839932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.840387 (+   455us) service_pool.cc:224] Handling call
0709 13:12:59.857185 (+ 16798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.581803 (+724618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:00.581835 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:00.581878 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:00.581906 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.541144 (+959238us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54002111d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.541202 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:01.541379 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649533696}
I20260709 13:13:01.528035 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:01.566233 23457 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:01.568768 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:01.574761 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:01.577811 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.581108 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.579463 26613 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:13:01.582049 22838 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:01.578166 26440 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:01.579679 26581 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:13:01.582970 26613 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:13:01.580901 26664 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.912s	user 0.015s	sys 0.000s
I20260709 13:13:01.543155 26660 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.577565 26711 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:01.585937 26581 consensus_queue.cc:1243] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:01.586195 26613 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:01.572456 26694 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:13:01.594597 26570 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:13:01.596115 26728 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.580596 26715 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:13:01.583781 26516 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:13:01.597024 26696 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:01.607324 26711 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:01.599116 26578 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:01.615751 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.597186 26557 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:13:01.598773 26384 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:13:01.622216 26696 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.628105 26578 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.631460 26711 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.640501 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1509) took 1835 ms. Trace:
I20260709 13:13:01.640955 22833 rpcz_store.cc:276] 0709 13:12:59.805571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.767605 (+962034us) service_pool.cc:224] Handling call
0709 13:13:00.831159 (+ 63554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.082699 (+251540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.640240 (+557541us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540028a5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.640434 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576642304}
I20260709 13:13:01.638408 26586 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.642226 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1641) took 2466 ms. Trace:
I20260709 13:13:01.640516 26660 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.642753 22839 rpcz_store.cc:276] 0709 13:12:59.176220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.176702 (+   482us) service_pool.cc:224] Handling call
0709 13:12:59.177680 (+   978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:59.855242 (+677562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.641797 (+1786555us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.642051 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1543978112}
I20260709 13:13:01.648864 26668 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.649225 22838 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:13:01.649748 22838 pending_rounds.cc:85] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:13:01.649973 22838 pending_rounds.cc:107] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:13:01.650199 22838 raft_consensus.cc:2889] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.650444 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.651548 22835 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:01.652022 26688 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.652456 22838 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:01.649153 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.636677 26696 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.654393 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.636612 26039 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.657061 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.660781 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:01.661392 26702 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.442s	user 0.000s	sys 0.003s
I20260709 13:13:01.662253 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.663338 26260 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:01.655593 22833 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
W20260709 13:13:01.665083 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:01.665696 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.653818 26650 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:01.666887 26343 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:01.667112 26650 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:01.667490 26343 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.668144 26650 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.665604 22836 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:01.669440 26650 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:01.676612 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.670292 26343 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.666870 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.663370 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.680639 23466 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:01.681870 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:01.682154 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.672145 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:01.683763 22839 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:01.690784 22914 pending_rounds.cc:85] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 3
I20260709 13:13:01.694568 22914 pending_rounds.cc:107] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:13:01.694842 22914 raft_consensus.cc:2889] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.669267 26578 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:01.696838 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:01.622859 26728 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.714097 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:01.721724 26668 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:01.693461 26636 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:01.693811 26613 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.730590 26636 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:01.747159 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:01.762315 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 103057us
W20260709 13:13:01.766177 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 187559us
I20260709 13:13:01.747856 26636 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:01.766264 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:01.768028 23464 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.767052 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:01.769268 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.770632 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.770392 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:01.773116 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.774857 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.775630 23458 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:01.769910 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.782414 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1648) took 2314 ms. Trace:
I20260709 13:13:01.787197 22837 rpcz_store.cc:276] 0709 13:12:59.468298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.468544 (+   246us) service_pool.cc:224] Handling call
0709 13:12:59.478105 (+  9561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.513994 (+1035889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.782023 (+1268029us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001bce54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.782331 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374880000}
I20260709 13:13:01.768843 26343 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.797061 26527 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.773476 26573 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.777113 26613 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:01.804906 23461 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:01.796270 26748 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:01.827389 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.831307 22837 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:01.774477 26737 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.842265 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:01.850597 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1520) took 1781 ms (client timeout 1639 ms). Trace:
W20260709 13:13:01.852108 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.853430 22760 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.856918 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.858124 22764 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.858817 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:01.808506 26660 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:01.866276 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 255793us
I20260709 13:13:01.822278 26570 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.861261 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.825829 26758 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.862973 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.873864 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.874516 22766 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:01.866489 22765 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:13:01.851075 22840 rpcz_store.cc:269] 0709 13:13:00.069217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.388201 (+318984us) service_pool.cc:224] Handling call
0709 13:13:01.850413 (+1462212us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000a5354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:01.850528 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460726016}
I20260709 13:13:01.867628 22760 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:01.827059 26748 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.883602 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:01.877375 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:01.877781 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.877940 26711 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:01.878095 26578 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:01.886911 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 686) took 1034 ms. Trace:
I20260709 13:13:01.807178 26343 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.827839 26557 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:13:01.819010 26573 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.893630 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:01.899708 22765 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.865478 26676 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:01.887310 23465 rpcz_store.cc:276] 0709 13:13:00.852735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.852904 (+   169us) service_pool.cc:224] Handling call
0709 13:13:00.865271 (+ 12367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.181024 (+315753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.886640 (+705616us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000b6654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:01.886834 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724423936}
I20260709 13:13:01.900887 22762 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:01.895809 26343 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.873808 26570 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:01.906173 22760 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:13:01.906641 26760 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:01.907671 26570 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.912755 26715 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:01.903750 26039 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:13:01.914964 26766 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:13:01.914002 26715 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:01.917824 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.918744 26516 raft_consensus.cc:993] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:01.919216 26516 raft_consensus.cc:1081] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:01.872773 26660 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.842087 26708 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:13:01.903975 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:01.903240 26707 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.677s	user 0.000s	sys 0.003s
W20260709 13:13:01.924404 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.931123 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.964318 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.928687 26715 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:01.899263 26573 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:02.021061 22761 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:01.980458 26440 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:01.887354 26748 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:01.933609 22764 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:13:02.028888 22764 pending_rounds.cc:85] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:13:02.029172 22764 pending_rounds.cc:107] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:13:02.029457 22764 raft_consensus.cc:2889] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:02.030651 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.948680 26753 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:13:02.030917 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2366) took 2409 ms. Trace:
I20260709 13:13:02.031780 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:01.988889 22765 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:01.899374 26683 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:01.996925 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67130.1us
I20260709 13:13:01.887274 26756 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:13:01.998664 26728 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:01.999720 26660 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:01.901234 26677 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:02.014544 22760 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:02.015535 22757 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:02.013288 23464 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:01.985186 22766 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:01.929948 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:02.026044 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:02.026361 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.026580 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.026978 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:02.014226 26765 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:13:02.049400 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1654) took 1136 ms. Trace:
I20260709 13:13:02.049808 22841 rpcz_store.cc:276] 0709 13:13:00.913285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.442297 (+529012us) service_pool.cc:224] Handling call
0709 13:13:01.442749 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:01.552298 (+109549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.552347 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:01.552399 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:01.552442 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.049170 (+496728us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540003a2d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:02.049217 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:02.049342 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251369728}
I20260709 13:13:02.024705 26440 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:02.054374 26711 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:02.054827 26711 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:02.054265 26440 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.055734 26440 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:02.055699 26711 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.057089 26711 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:02.059460 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.060639 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.063683 26710 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:02.063877 26640 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.002s
I20260709 13:13:02.065213 26570 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:02.066152 26570 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:02.066298 26756 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:02.067426 26593 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:02.067871 26593 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:02.068676 26737 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:02.068789 26593 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.070151 26593 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:02.070402 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:02.071503 26654 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:02.072484 26708 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:02.046046 22912 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:02.031359 22759 rpcz_store.cc:276] 0709 13:12:59.622041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.630893 (+  8852us) service_pool.cc:224] Handling call
0709 13:12:59.631572 (+   679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:59.835836 (+204264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:59.835872 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:59.841105 (+  5233us) log.cc:844] Serialized 26 byte log entry
0709 13:13:00.146024 (+304919us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:00.225300 (+ 79276us) log.cc:844] Serialized 14 byte log entry
0709 13:13:00.225594 (+   294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.225687 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:00.337221 (+111534us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b18003f5c20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:01.514314 (+1177093us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b18003f5c20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:01.902602 (+388288us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:02.030624 (+128022us) raft_consensus.cc:1685] finished
0709 13:13:02.030652 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:02.030795 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":243257,"prepare.queue_time_us":79799,"prepare.run_cpu_time_us":276,"prepare.run_wall_time_us":2898,"raft.queue_time_us":196263,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":3858,"spinlock_wait_cycles":27624832,"thread_start_us":211979,"threads_started":1,"wal-append.queue_time_us":243002}
I20260709 13:13:01.710820 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:02.061388 22757 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:13:02.073465 26654 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:13:02.090548 26654 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:02.090796 26654 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:02.091388 26654 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.092355 26654 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:02.073714 26039 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:02.074923 26768 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:13:02.095031 26516 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:02.083405 22760 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:02.035480 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:02.096725 22906 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:02.023437 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 130608us
I20260709 13:13:02.036669 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:02.083763 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.037446 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.098558 22762 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:01.929077 26676 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.013022 26639 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:02.009991 26696 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:02.041164 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 110508us
I20260709 13:13:02.044477 22913 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:02.047320 26677 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.039038 26683 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:02.048147 22915 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:02.094228 26039 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.095789 26516 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:02.035676 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.081332 26715 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:02.089744 26757 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:01.882978 26758 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:02.087045 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 157262us
I20260709 13:13:02.033854 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.106685 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:02.084906 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.080303 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.169557 22759 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:13:02.101478 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.115577 22912 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:02.174712 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.175981 22766 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:02.099957 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.176996 22760 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:02.177922 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.178503 22760 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:02.179239 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.046815 22761 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:02.135424 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.140023 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:02.186988 22762 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:02.084702 22757 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:02.149044 26516 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.150256 22764 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:02.154456 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.157109 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:02.191591 22760 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:02.158293 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.198504 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.202003 22909 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:02.202864 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.203984 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.204617 22759 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:02.159605 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:02.163702 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:02.210641 22840 raft_consensus.cc:2376] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10. Re-sending same reply.
I20260709 13:13:02.164695 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.211927 22908 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:02.153417 26676 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:02.146023 26039 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:02.190068 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.164989 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.212756 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.139680 26623 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.213671 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:02.248031 22834 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:02.214123 22761 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:02.215201 22760 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:02.218873 22764 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:02.156239 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.221596 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.222774 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:02.225360 22913 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:02.249557 22757 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:02.083621 26636 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:02.226044 22906 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:02.228012 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:02.238134 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:02.240574 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:02.134686 26639 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.247138 22839 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:02.255663 26640 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.256256 26640 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.257213 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:02.258080 26623 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.258889 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:02.260416 26678 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:02.261535 26678 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:02.261920 26678 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:02.262027 26763 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:13:02.262615 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.263927 22908 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:02.143008 26669 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:13:02.144577 26573 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:13:02.272025 22759 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:13:02.275245 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:02.275779 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.262333 26678 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:02.233747 26668 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.299671 22762 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:02.303489 22906 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:13:02.306231 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:02.249433 26676 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:02.309904 22915 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:02.308106 22765 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:02.260492 26623 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:02.312058 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.253890 26516 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:02.221582 26585 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:13:02.268656 26757 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:02.327625 22766 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:13:02.309688 26695 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:02.302831 26298 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:13:02.338728 26696 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:02.328903 26623 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:02.312119 26637 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:02.280818 26762 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:13:02.355832 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.282341 26650 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:13:02.358381 22913 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:02.357124 22915 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:02.360757 22909 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:13:02.365828 26623 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:02.368517 26623 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:02.380935 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:02.381843 26637 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:02.382012 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.385810 22762 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:02.383908 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.386961 22766 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:02.384055 22915 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:02.317556 26668 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:02.423707 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:02.426203 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:02.432090 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 272547us
I20260709 13:13:02.438119 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.440795 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:02.443095 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:02.460049 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.461055 22835 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:02.457751 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:02.464952 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:02.478096 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.492309 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:02.496845 22835 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:13:02.463490 26573 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:02.432792 26683 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:02.463698 26728 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:02.465058 26640 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:02.503414 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.548439 22909 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:13:02.483628 26660 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:02.558010 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:02.558911 22909 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:02.567957 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:02.568739 22759 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:13:02.577131 26639 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:02.558627 26660 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.588881 26660 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:02.555490 26677 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:02.519207 26637 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:02.509878 26688 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:13:02.519946 26683 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 13:13:02.572853 26678 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:02.501442 22915 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:02.501863 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:02.507755 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:02.518081 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 294459us
W20260709 13:13:02.512210 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 238569us
I20260709 13:13:03.127146 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:03.143018 22913 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:02.989421 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:03.130237 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.050s
W20260709 13:13:03.146080 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.000s	sys 0.052s
W20260709 13:13:02.732254 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:03.146725 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:02.664199 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 355690us
W20260709 13:13:02.650120 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 543506us
W20260709 13:13:03.150205 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.151522 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:03.129537 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:03.154610 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:02.639943 26654 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.004s
W20260709 13:13:02.727385 26748 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.000s
W20260709 13:13:02.696297 26711 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.639s	user 0.016s	sys 0.001s
I20260709 13:13:03.163146 26748 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:02.769763 26695 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:02.637465 26636 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:03.164981 26695 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.165820 26636 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:03.172029 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.157465 22915 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:03.175562 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:03.182760 22915 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:13:03.126245 26715 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.960s	user 0.009s	sys 0.005s
I20260709 13:13:03.187866 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:02.782860 26394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.001s	sys 0.010s
I20260709 13:13:03.188650 22841 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:13:03.238480 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:03.146525 26593 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.076s	user 0.014s	sys 0.001s
W20260709 13:13:03.196794 22840 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.008s	sys 0.033s
W20260709 13:13:03.249730 22840 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.009s	sys 0.033s
I20260709 13:13:02.620258 26527 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:02.771848 26560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.008s	sys 0.003s
I20260709 13:13:03.213868 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:03.002534 26650 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.593389 26758 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:03.252054 22914 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:03.207283 22915 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:03.147210 26668 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:03.252465 26394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.003s	sys 0.021s
W20260709 13:13:03.212462 26599 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.006s	sys 0.017s
W20260709 13:13:03.195549 26776 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:13:03.282555 26039 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.028s	user 0.006s	sys 0.007s
I20260709 13:13:03.265530 22912 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:13:03.192210 26696 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.012s	sys 0.008s
I20260709 13:13:03.178345 26343 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:03.169162 26636 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.252272 26654 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:03.262933 26560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.008s	sys 0.017s
I20260709 13:13:03.277889 22766 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:03.294046 26599 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.006s	sys 0.018s
W20260709 13:13:03.306591 26696 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.012s	sys 0.009s
I20260709 13:13:03.281777 26527 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.285746 26650 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.332692 26039 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.380079 26748 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:03.657442 22908 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:13:03.657486 22765 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:03.659237 22907 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:03.660317 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1585) took 1492 ms. Trace:
I20260709 13:13:03.672650 22912 rpcz_store.cc:276] 0709 13:13:02.171747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.174400 (+  2653us) service_pool.cc:224] Handling call
0709 13:13:03.660091 (+1485691us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000447d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:03.660235 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506834304}
W20260709 13:13:03.626171 26676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.284s	user 0.014s	sys 0.014s
W20260709 13:13:03.627698 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.010s
W20260709 13:13:03.378456 22835 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.010s	sys 0.075s
I20260709 13:13:03.633623 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 469) took 1321 ms. Trace:
W20260709 13:13:03.693255 22835 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.010s	sys 0.123s
W20260709 13:13:03.635926 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 484460us
W20260709 13:13:03.693236 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.000s	sys 0.011s
I20260709 13:13:03.695116 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1684) took 1246 ms. Trace:
W20260709 13:13:03.636889 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.17612e+06us
W20260709 13:13:03.310705 22762 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.017s	sys 0.023s
W20260709 13:13:03.645882 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.658977 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1685) took 1209 ms. Trace:
I20260709 13:13:03.695549 22834 rpcz_store.cc:276] 0709 13:13:02.448817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.637897 (+1189080us) service_pool.cc:224] Handling call
0709 13:13:03.694626 (+ 56729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.694803 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.694825 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.694866 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:03.694901 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.694953 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.695058 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:03.693405 22906 rpcz_store.cc:276] 0709 13:13:02.312194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.312365 (+   171us) service_pool.cc:224] Handling call
0709 13:13:02.342580 (+ 30215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:02.342799 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:02.386079 (+ 43280us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.603697 (+1217618us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.632927 (+ 29230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.632989 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:03.633046 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.633121 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.633547 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28897,"prepare.queue_time_us":49663,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":418047,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":24227,"thread_start_us":807022,"threads_started":2,"wal-append.queue_time_us":1304906}
I20260709 13:13:03.700021 22841 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:03.694489 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:03.706210 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:03.708635 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:03.696177 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:03.710923 22762 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.024s	sys 0.029s
I20260709 13:13:03.714324 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:03.715911 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:03.696894 22837 rpcz_store.cc:276] 0709 13:13:02.449070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.658019 (+1208949us) service_pool.cc:224] Handling call
0709 13:13:03.658363 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.658581 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.658615 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.658665 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:03.658695 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.658742 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.658840 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:03.721618 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:03.724152 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:03.725354 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:03.320597 26669 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.009s	sys 0.058s
I20260709 13:13:03.733316 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:03.678918 26710 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.615s	user 0.015s	sys 0.011s
W20260709 13:13:03.751124 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:03.757527 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.758575 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:03.674747 26756 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:03.762809 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:03.680016 26688 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.684638 26748 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:03.659106 26758 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:03.692746 26676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.351s	user 0.014s	sys 0.014s
I20260709 13:13:03.740653 26650 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
W20260709 13:13:03.766100 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.009s	sys 0.021s
W20260709 13:13:03.738708 26669 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.278s	user 0.011s	sys 0.059s
I20260709 13:13:03.693933 26516 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.713034 26783 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.768405 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1686) took 1319 ms. Trace:
W20260709 13:13:03.754730 26695 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.010s	sys 0.019s
I20260709 13:13:03.659044 26560 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:13:03.717296 26039 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:03.775640 22879 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.002s	sys 0.029s
I20260709 13:13:03.745803 26593 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:03.741042 26715 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:13:03.814317 22834 rpcz_store.cc:276] 0709 13:13:02.449306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.741123 (+1291817us) service_pool.cc:224] Handling call
0709 13:13:03.767810 (+ 26687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.768013 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.768038 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.768081 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:03.768108 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.768169 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.768338 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:03.650241 26343 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.759161 26711 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:03.760644 26774 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.676015 26654 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:03.834945 22879 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.003s	sys 0.029s
W20260709 13:13:03.859699 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.006s	sys 0.015s
W20260709 13:13:03.883821 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 685213us
W20260709 13:13:03.884238 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.008s	sys 0.016s
W20260709 13:13:03.812456 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.009s	sys 0.022s
I20260709 13:13:03.763240 26756 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:03.776189 26788 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.791052 26748 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:03.854944 22840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.015s	sys 0.003s
W20260709 13:13:03.829185 26695 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.010s	sys 0.019s
I20260709 13:13:03.790673 26817 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:03.888154 22840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.016s	sys 0.004s
I20260709 13:13:03.815382 26758 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.821061 26653 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:03.912777 22834 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:03.834283 26516 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.932555 26650 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.933445 26650 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.934525 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1681) took 1486 ms. Trace:
I20260709 13:13:03.934702 26516 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:03.934974 22833 rpcz_store.cc:276] 0709 13:13:02.448036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.463681 (+ 15645us) service_pool.cc:224] Handling call
0709 13:13:02.464056 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:02.773415 (+309359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.773449 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:02.773986 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.636419 (+862433us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:03.636466 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.636536 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.934264 (+297728us) raft_consensus.cc:1685] finished
0709 13:13:03.934285 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.934456 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":348548,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":54,"thread_start_us":754108,"threads_started":2,"wal-append.queue_time_us":1140646}
I20260709 13:13:03.936621 26650 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:03.937232 26758 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:03.938043 26756 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.794729 26688 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.948966 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:03.888334 26737 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.950121 23459 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:03.950526 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.015s	sys 0.015s
W20260709 13:13:03.950944 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.015s	sys 0.015s
I20260709 13:13:03.861987 26039 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:03.901474 26678 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:03.966112 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:03.971300 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:03.973634 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.892401 26774 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.947976 26710 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:03.972141 23465 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8. Re-sending same reply.
I20260709 13:13:03.949000 26817 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.976047 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:03.953158 26557 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:04.013332 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 317242us
I20260709 13:13:03.915509 26695 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:03.939114 26650 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:03.966308 26715 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:13:03.952142 26696 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.011356 26790 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.015098 23466 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:04.015616 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.021603 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:04.054147 26660 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.465s	user 0.013s	sys 0.012s
I20260709 13:13:03.977741 26756 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:04.127795 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.028295 23461 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:13:03.940295 26653 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:03.892793 26788 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:03.951946 26516 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:13:03.992425 26811 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:03.991679 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 275869us
I20260709 13:13:04.079617 26637 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.082267 26676 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.987632 26039 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.010049 26678 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:03.830155 26793 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.049479 26560 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:03.829047 26783 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.121166 23464 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:04.027562 26819 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.046586 26812 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:04.103538 26650 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.094012 26695 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.106012 26557 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:04.137346 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1680) took 1689 ms. Trace:
I20260709 13:13:04.140417 26688 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.140962 26696 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.115921 26636 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.094447 26800 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:04.262516 22908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1595) took 1890 ms (client timeout 2155 ms). Trace:
I20260709 13:13:04.262835 26800 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:04.263182 22908 rpcz_store.cc:269] 0709 13:13:02.372139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.459518 (+ 87379us) service_pool.cc:224] Handling call
0709 13:13:04.262266 (+1802748us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54000eae54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:04.262431 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179343104}
I20260709 13:13:04.265159 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1690) took 1094 ms. Trace:
I20260709 13:13:04.265702 22841 rpcz_store.cc:276] 0709 13:13:03.170441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.238103 (+ 67662us) service_pool.cc:224] Handling call
0709 13:13:04.264930 (+1026827us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:04.265080 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059174656}
W20260709 13:13:04.268414 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.269979 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2495) took 1125 ms. Trace:
I20260709 13:13:04.270481 22915 rpcz_store.cc:276] 0709 13:13:03.144387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.283870 (+139483us) service_pool.cc:224] Handling call
0709 13:13:04.268147 (+984277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.268386 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:04.268416 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:04.268462 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:04.268499 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.268558 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.269733 (+  1175us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54000eae50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:04.269897 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2262764544}
W20260709 13:13:04.145607 26794 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.368s	user 0.001s	sys 0.000s
I20260709 13:13:04.130056 26660 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.147433 26653 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.148967 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:04.243643 26786 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 13:13:04.318481 26688 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:04.087930 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 202397us
I20260709 13:13:04.360989 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.015669 26710 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.371503 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:03.967618 26737 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.021597 26817 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:04.174896 22914 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1599) took 1801 ms (client timeout 2117 ms). Trace:
W20260709 13:13:04.164144 26585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.823s	user 0.014s	sys 0.059s
I20260709 13:13:04.187781 22759 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:04.188932 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1713) took 1670 ms. Trace:
W20260709 13:13:04.169754 26816 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:13:04.136288 26343 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.375689 26516 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:04.404510 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:04.404817 26343 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:13:04.173388 26799 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 13:13:03.987387 26748 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.240278 22838 rpcz_store.cc:276] 0709 13:13:02.447735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.447885 (+   150us) service_pool.cc:224] Handling call
0709 13:13:02.448300 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:02.448726 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.448758 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:02.449277 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:13:02.786279 (+337002us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:02.786333 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.786436 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.137050 (+1350614us) raft_consensus.cc:1685] finished
0709 13:13:04.137081 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.137258 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":363941,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":52,"thread_start_us":300792,"threads_started":1,"wal-append.queue_time_us":1537542}
I20260709 13:13:04.239321 26819 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.411144 26819 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.407068 26650 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.405539 22836 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:04.390275 26585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.049s	user 0.014s	sys 0.059s
I20260709 13:13:04.180769 26678 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.420848 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 472) took 2001 ms. Trace:
I20260709 13:13:04.421224 22913 rpcz_store.cc:276] 0709 13:13:02.419738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.154716 (+734978us) service_pool.cc:224] Handling call
0709 13:13:03.246773 (+ 92057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:03.247163 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.247191 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:03.247685 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.762839 (+515154us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:03.762901 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.762976 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.420620 (+657644us) raft_consensus.cc:1685] finished
0709 13:13:04.420647 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.420786 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10531,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":21057,"thread_start_us":457515,"threads_started":1,"wal-append.queue_time_us":1028438}
I20260709 13:13:04.422506 26678 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:04.255955 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.011s	sys 0.002s
W20260709 13:13:04.257129 26825 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 13:13:04.423220 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.011s	sys 0.002s
I20260709 13:13:04.424067 26788 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.257582 26557 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.257186 26728 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:04.147277 26815 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:04.425655 26728 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.425535 26815 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.426203 26728 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.426833 26715 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:04.427947 26815 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.304138 26394 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.428416 26728 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.428879 26660 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.428838 26394 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.429370 26728 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.430387 26728 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.431159 26394 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.432852 26728 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.419938 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1584) took 2277 ms. Trace:
W20260709 13:13:04.270021 26824 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:13:04.170313 26676 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.434170 22911 rpcz_store.cc:276] 0709 13:13:02.142441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.142591 (+   150us) service_pool.cc:224] Handling call
0709 13:13:02.168950 (+ 26359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:02.201019 (+ 32069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.201066 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:02.237719 (+ 36653us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.137023 (+899304us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:03.137094 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.137241 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.419740 (+1282499us) raft_consensus.cc:1685] finished
0709 13:13:04.419760 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.419871 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43180,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":20525,"thread_start_us":303784,"threads_started":1,"wal-append.queue_time_us":1589405}
I20260709 13:13:04.438208 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.273532 26812 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:04.439009 26817 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.439142 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.439924 26812 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.440565 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:04.441777 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.170192 26637 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.444008 26637 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:04.377974 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 229090us
I20260709 13:13:04.444862 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.265440 26800 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:04.317926 26668 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.012s
I20260709 13:13:04.346046 26653 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:04.445819 26668 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.002s	sys 0.012s
W20260709 13:13:04.384810 22914 rpcz_store.cc:269] 0709 13:13:02.373441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.429644 (+ 56203us) service_pool.cc:224] Handling call
0709 13:13:04.174621 (+1744977us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54001d1d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:04.174811 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866570112}
I20260709 13:13:04.446113 26653 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.241650 26711 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:04.447705 26711 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.448320 26653 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.164503 26811 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.390710 22759 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:04.450814 26811 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.392467 26826 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.451676 26826 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.306248 26654 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.313482 26039 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.456681 26676 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.397138 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:04.460050 26593 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.604s	user 0.008s	sys 0.001s
W20260709 13:13:04.460335 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 63353.8us
I20260709 13:13:04.460564 26593 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.463670 26710 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.390719 22762 rpcz_store.cc:276] 0709 13:13:02.518127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.518305 (+   178us) service_pool.cc:224] Handling call
0709 13:13:02.561661 (+ 43356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:02.561970 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:03.711581 (+1149611us) log.cc:844] Serialized 14 byte log entry
0709 13:13:04.187188 (+475607us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:04.187524 (+   336us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:04.187602 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.188643 (+  1041us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.188849 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":593058,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":9784,"thread_start_us":1564904,"threads_started":3}
I20260709 13:13:04.419938 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1682) took 1971 ms. Trace:
I20260709 13:13:04.465973 26737 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.378626 26688 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.465858 22835 rpcz_store.cc:276] 0709 13:13:02.448319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.556671 (+108352us) service_pool.cc:224] Handling call
0709 13:13:02.557144 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:02.699737 (+142593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.699798 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:02.738617 (+ 38819us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.694013 (+955396us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:03.694060 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.694138 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.419679 (+725541us) raft_consensus.cc:1685] finished
0709 13:13:04.419702 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.419859 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":255576,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":52,"thread_start_us":910755,"threads_started":1,"wal-append.queue_time_us":1634808}
I20260709 13:13:04.468626 26688 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:04.242798 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 344396us
I20260709 13:13:04.438294 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.469391 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.470566 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 473) took 1909 ms. Trace:
I20260709 13:13:04.470777 22841 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:04.470978 22909 rpcz_store.cc:276] 0709 13:13:02.561411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.283851 (+722440us) service_pool.cc:224] Handling call
0709 13:13:03.284280 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:03.288081 (+  3801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.288134 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:03.288843 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.559075 (+270232us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:03.559131 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.559232 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.470346 (+911114us) raft_consensus.cc:1685] finished
0709 13:13:04.470368 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.470503 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":557,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":8205,"spinlock_wait_cycles":47616,"thread_start_us":269772,"threads_started":1,"wal-append.queue_time_us":952751}
I20260709 13:13:04.473836 22841 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:04.474151 26841 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.474794 26841 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.288193 26696 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.478264 26793 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.478840 26793 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.479341 26696 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.479950 26696 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.480428 26696 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.481125 26793 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.541544 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1678) took 2094 ms. Trace:
W20260709 13:13:04.422911 26778 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.958s	user 0.003s	sys 0.020s
I20260709 13:13:04.486459 26838 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:04.573316 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 699) took 2647 ms. Trace:
W20260709 13:13:04.519706 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.484193 26527 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.488629 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1691) took 1255 ms. Trace:
I20260709 13:13:04.486377 26599 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:04.599311 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.508497 26748 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:04.492660 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1581) took 2420 ms. Trace:
I20260709 13:13:04.551807 26737 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:13:04.546427 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76294.3us
I20260709 13:13:04.545812 26636 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:04.544942 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100180us
I20260709 13:13:04.608349 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:04.542546 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 103488us
I20260709 13:13:04.485538 26826 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.484045 26783 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:04.455739 26794 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:13:04.513000 26830 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.534003 22661 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 7, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:04.482653 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.544267 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:04.614507 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.523715 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 458) took 2669 ms. Trace:
W20260709 13:13:04.615239 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71086.9us
I20260709 13:13:04.615438 22835 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:04.616397 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:04.524255 26683 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:04.489444 26696 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.461272 26593 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:04.513099 22732 connection.cc:256] RPC call timeout handler was delayed by -1.57726133999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
I20260709 13:13:04.563405 26695 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.526342 26787 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:04.491479 26774 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.637965 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.606134 26599 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:04.586088 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.587234 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.159325 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1679) took 1712 ms. Trace:
I20260709 13:13:04.589908 22759 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.573804 23460 rpcz_store.cc:276] 0709 13:13:01.925992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.926158 (+   166us) service_pool.cc:224] Handling call
0709 13:13:01.965180 (+ 39022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.982431 (+ 17251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.982467 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:02.012222 (+ 29755us) log.cc:844] Serialized 26 byte log entry
0709 13:13:02.282333 (+270111us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:02.282386 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.282472 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.422188 (+2139716us) spinlock_profiling.cc:243] Waited 8.51 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:04.572926 (+150738us) raft_consensus.cc:1685] finished
0709 13:13:04.572956 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.573236 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50053,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":19572992,"thread_start_us":269678,"threads_started":1,"wal-append.queue_time_us":406249}
I20260709 13:13:04.575830 22832 rpcz_store.cc:276] 0709 13:13:02.446980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.451204 (+  4224us) service_pool.cc:224] Handling call
0709 13:13:02.451688 (+   484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:02.452178 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.452222 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:02.490074 (+ 37852us) log.cc:844] Serialized 52 byte log entry
0709 13:13:03.001822 (+511748us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:03.558723 (+556901us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.640470 (+ 81747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.640570 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.303451 (+662881us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b180014db80. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:04.538710 (+235259us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b180014db80. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:04.541199 (+  2489us) raft_consensus.cc:1685] finished
0709 13:13:04.541223 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.541439 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2238,"prepare.queue_time_us":833562,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":662825,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":277,"spinlock_wait_cycles":31559680,"thread_start_us":865171,"threads_started":2,"wal-append.queue_time_us":1387688}
I20260709 13:13:04.643392 22838 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:04.644788 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.575440 22834 rpcz_store.cc:276] 0709 13:13:03.233215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.913910 (+680695us) service_pool.cc:224] Handling call
0709 13:13:03.953990 (+ 40080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.280053 (+326063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:04.280105 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:04.280170 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:04.280220 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.488296 (+208076us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540027ea54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:04.488363 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.488565 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748980608}
I20260709 13:13:04.602100 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.603168 22907 rpcz_store.cc:276] 0709 13:13:02.072235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.100762 (+ 28527us) service_pool.cc:224] Handling call
0709 13:13:02.101075 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.672239 (+1571164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.492345 (+820106us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400125cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:04.492580 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3583091584}
I20260709 13:13:04.614837 22839 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:04.643349 22841 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:04.647070 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.647742 22839 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:04.648547 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.500486 26841 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.617688 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:04.619338 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:04.619920 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.615398 22758 rpcz_store.cc:276] 0709 13:13:01.854492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.854623 (+   131us) service_pool.cc:224] Handling call
0709 13:13:01.871953 (+ 17330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.872321 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.872352 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:01.889921 (+ 17569us) log.cc:844] Serialized 26 byte log entry
0709 13:13:02.266421 (+376500us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:02.720462 (+454041us) log.cc:844] Serialized 14 byte log entry
0709 13:13:04.523161 (+1802699us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b54002b6ea0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6c733 00007f68a1af1dbf 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c
0709 13:13:04.523223 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.523334 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.523384 (+    50us) raft_consensus.cc:1685] finished
0709 13:13:04.523498 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.523644 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":506057,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":9365,"raft.queue_time_us":5799,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":38020,"spinlock_wait_cycles":141753216,"thread_start_us":698010,"threads_started":2,"wal-append.queue_time_us":609674,"wal-append.run_cpu_time_us":23482,"wal-append.run_wall_time_us":1729275}
W20260709 13:13:04.626008 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4414441080000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 13:13:04.626736 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.572547 26838 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.583994 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:04.638907 23461 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:04.640592 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.607239 26636 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.645015 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:04.657027 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:04.651132 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3098170790000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 13:13:04.658131 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3079836069999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260709 13:13:04.658811 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0937836069999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 13:13:04.659142 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:04.659286 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0415058289999024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 13:13:04.659705 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.660084 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:04.651693 22837 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.654930 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.652431 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.656083 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.662310 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:04.643333 22840 rpcz_store.cc:276] 0709 13:13:02.447371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.449644 (+  2273us) service_pool.cc:224] Handling call
0709 13:13:02.450035 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:02.450306 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:03.250289 (+799983us) log.cc:844] Serialized 14 byte log entry
0709 13:13:04.158785 (+908496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:04.158871 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:04.158940 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.159040 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.159240 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1308358,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":253961,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":8732,"thread_start_us":1707172,"threads_started":3,"wal-append.queue_time_us":660793}
I20260709 13:13:04.662437 23458 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:04.651248 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:04.657840 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.076s	sys 0.183s
W20260709 13:13:04.665283 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.077s	sys 0.183s
W20260709 13:13:04.665644 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 613671us
I20260709 13:13:04.666386 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:04.664404 22661 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 5 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 7 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:04.628961 26527 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:04.673350 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:04.673305 22661 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 6, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:04.626924 26815 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.638363 26787 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:04.677053 26756 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.677588 26593 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:04.678324 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60756.4us
I20260709 13:13:04.679028 26817 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.708149 26654 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:04.708000 26787 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.679652 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.663655 26636 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:04.709671 26787 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:04.680809 22664 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 244) took 38364 ms (client timeout 15000 ms). Trace:
I20260709 13:13:04.711344 26819 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:04.711794 26819 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.712160 26819 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:04.680887 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.681367 26845 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.682860 22759 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.684162 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.684914 26846 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:04.625413 26833 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.639321 26039 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:04.685595 26573 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:04.649580 26851 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:13:04.686028 26676 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:04.687374 26669 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:13:04.726949 23431 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.014s	sys 0.014s
W20260709 13:13:04.727350 23431 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.014s	sys 0.014s
I20260709 13:13:04.705070 26343 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:04.691661 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.731338 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.731930 22913 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:04.731963 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.733182 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.734289 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.692871 26841 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:04.658267 26842 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:13:04.693424 26599 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.629031 26683 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.694962 26696 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.639626 26826 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.696456 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.620632 26830 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.628628 26783 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.674346 26774 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:04.703912 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:04.717866 26593 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.706439 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:04.717813 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.718858 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.687415 26838 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.720480 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.721740 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.721997 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.722741 22910 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:04.722971 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.725865 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46956 (request call id 1714) took 2150 ms. Trace:
I20260709 13:13:04.722843 22661 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 2 to 4, term changed from 5 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to <none>, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 6 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:04.726059 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:04.727679 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 777326us
I20260709 13:13:04.691259 22837 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:04.724020 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.715850 26756 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.738029 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:04.711138 22664 rpcz_store.cc:269] 0709 13:12:26.326768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.327228 (+   460us) service_pool.cc:224] Handling call
0709 13:13:04.680697 (+38353469us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:58.755130 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:58.756011 (+   881us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:58.756090 (+    79us) write_op.cc:435] Acquired schema lock
0709 13:12:58.756124 (+    34us) tablet.cc:662] Decoding operations
0709 13:12:58.815554 (+ 59430us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:58.835840 (+ 20286us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:58.844279 (+  8439us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:12:59.015946 (+171667us) tablet.cc:701] Row locks acquired
0709 13:12:59.030455 (+ 14509us) write_op.cc:260] PREPARE: finished
0709 13:12:59.030694 (+   239us) write_op.cc:270] Start()
0709 13:12:59.030855 (+   161us) write_op.cc:276] Timestamp: P: 1783602779030613 usec, L: 0
0709 13:12:59.030885 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:12:59.031701 (+   816us) log.cc:844] Serialized 3587 byte log entry
0709 13:12:59.641392 (+609691us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b18002a4b40. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:00.028210 (+386818us) op_driver.cc:464] REPLICATION: finished
0709 13:13:00.566809 (+538599us) write_op.cc:301] APPLY: starting
0709 13:13:00.618298 (+ 51489us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:00.698339 (+ 80041us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:00.698422 (+    83us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:00.699503 (+  1081us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:00.699856 (+   353us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:13:00.699910 (+    54us) write_op.cc:312] APPLY: finished
0709 13:13:00.700354 (+   444us) log.cc:844] Serialized 71 byte log entry
0709 13:13:00.700595 (+   241us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:00.700783 (+   188us) write_op.cc:454] Released schema lock
0709 13:13:00.700859 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:00.701369 (+   510us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:00.701413 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":281151,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":517912,"apply.run_cpu_time_us":4547,"apply.run_wall_time_us":134541,"num_ops":9,"prepare.queue_time_us":174081,"prepare.run_cpu_time_us":41754,"prepare.run_wall_time_us":350387,"raft.queue_time_us":261337,"raft.run_cpu_time_us":225530,"raft.run_wall_time_us":563705,"replication_time_us":927169,"spinlock_wait_cycles":44253056,"thread_start_us":727730,"threads_started":4,"wal-append.queue_time_us":105291,"wal-append.run_cpu_time_us":20367,"wal-append.run_wall_time_us":2068823}]]}
I20260709 13:13:04.719679 26846 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.724571 26800 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:04.752964 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56620.1us
I20260709 13:13:04.729902 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.739094 26788 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.739393 26394 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.730885 26573 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:04.779315 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72995.8us
I20260709 13:13:04.736469 26841 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:04.741441 26785 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:04.738986 26829 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:04.781219 22912 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:04.778708 22909 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:04.782670 22907 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:04.780251 22759 rpcz_store.cc:276] 0709 13:13:02.568271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.568554 (+   283us) service_pool.cc:224] Handling call
0709 13:13:04.681559 (+2113005us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540036f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1ae8a3d 00007f68a6ab8cd0 00007f68a003d254 00007f68a003d1e2 00007f68a003d111 00007f68a003d08c 00007f68a003c342 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:04.718732 (+ 37173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3721709824}
I20260709 13:13:04.785709 22908 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:04.787822 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:04.788316 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 64386.5us
W20260709 13:13:04.788678 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50761.9us
I20260709 13:13:04.781050 22661 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 3 to 7, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 7 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:04.817327 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.726547 26039 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:04.791801 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87981.3us
W20260709 13:13:04.792282 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 71845.3us
I20260709 13:13:04.794109 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.819512 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.820120 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.795303 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:04.820851 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:04.797031 26573 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.798147 26838 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:04.798579 26841 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:04.800992 26830 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:04.795188 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:04.724038 26845 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:04.795186 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.745110 26696 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.804299 22915 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:04.806257 26829 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.784201 26833 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.755074 26774 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:04.811009 26593 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:04.802212 26557 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.812188 26683 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.870540 22807 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.780186 26527 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:04.793135 26394 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.823782 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.827896 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:04.830938 26838 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.829578 22837 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:13:04.785354 26846 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:04.791999 26788 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.793828 26827 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.835083 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:04.830680 26343 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:13:04.828354 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.803398 26854 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:04.865088 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.802871 26785 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.866115 22914 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:04.752620 26826 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.830257 26800 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.885622 26838 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:04.777438 26783 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.878631 22661 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 7, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 7 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:04.821301 22911 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:04.881407 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:04.881938 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58261.4us
I20260709 13:13:04.833643 26839 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.830417 26844 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:04.813797 26711 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:13:04.864387 26696 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.863222 26516 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.863273 26829 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:04.888612 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60815.8us
I20260709 13:13:04.828568 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:04.862661 26830 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.865733 26774 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:04.892031 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57044.3us
I20260709 13:13:04.885779 26683 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.893992 26845 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.911167 26854 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:04.927588 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.928188 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.928587 26789 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:04.928241 26854 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.929159 26789 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:04.929963 26844 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.930420 26844 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:04.930154 26789 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.930838 26839 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.929185 26827 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:04.930896 26830 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.931545 26839 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.907202 26654 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.932255 26654 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:04.932677 26654 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.901084 26640 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.934479 26711 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:04.930475 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:04.936102 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:04.930631 22661 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 1 to 7, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 7 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:04.887511 26788 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.875445 26527 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.941636 26527 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:04.943228 26842 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:04.943668 26842 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.944017 26842 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:04.901937 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74807.3us
I20260709 13:13:04.865726 26840 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:13:04.947796 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.950138 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.884986 26394 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.873781 26557 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:04.907145 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.904641 26817 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:04.826058 26676 raft_consensus.cc:2764] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:13:04.937006 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:04.938370 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 109903us
I20260709 13:13:04.938349 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:04.940207 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58961.1us
I20260709 13:13:04.967789 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:04.909013 26785 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:04.889060 26668 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 13:13:04.986932 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50003.8us
I20260709 13:13:04.959154 23461 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:04.886301 26737 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:13:04.991060 23431 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.748s (SENT)
I20260709 13:13:04.996457 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:04.997140 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.955552 26827 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:04.933297 26830 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.971726 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:04.921032 26826 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.972208 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:04.934854 26711 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:04.918467 26039 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:04.961740 26845 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:04.980516 26696 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:04.912789 26650 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:13:04.984839 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.985441 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.030943 23459 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:05.030696 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.031639 22762 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:05.004369 26850 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:04.992530 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:04.951151 26640 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:04.911870 26862 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:13:04.965342 22661 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 5 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 7 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:04.997177 23463 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:05.005407 22766 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:13:04.965384 26557 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.965147 26593 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:04.965740 26817 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:05.013834 23457 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:05.015874 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.018651 23465 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:13:04.995123 26737 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.999274 26683 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:04.989848 26838 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:04.991433 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:05.128896 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 137542us
I20260709 13:13:05.130609 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:05.031118 22760 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:05.021687 26845 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:05.139609 22762 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:05.045657 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.037123 26850 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.046516 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.148627 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:05.047819 26696 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.050387 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.052433 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:04.980520 26819 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:05.173928 22764 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:13:05.083992 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 145711us
W20260709 13:13:05.066524 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 116450us
I20260709 13:13:05.022527 26756 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:05.132468 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:05.154269 22661 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:05.046080 26864 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:13:05.142665 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.036044 26039 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:05.144970 22730 connection.cc:256] RPC call timeout handler was delayed by -2.9732594039999185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:05.049860 26640 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:05.146704 22761 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:05.013656 26823 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:13:05.174000 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.175784 22757 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:13:05.117074 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.074242 26593 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.112207 26737 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:05.187322 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.177790 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.203087 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.177817 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.137219 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:05.182986 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.358317 23458 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:05.073336 26817 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:05.112399 26527 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.146405 26599 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:05.154736 26695 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:05.293871 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:05.328063 26593 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.266943 22661 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 8, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 8 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:05.361335 26864 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:13:05.212677 26573 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:05.202466 26866 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:13:05.194787 26819 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.186947 26838 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:05.216486 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.454008 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:05.225677 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 173300us
I20260709 13:13:05.324280 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.455371 22757 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:05.442602 22661 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index -1 to 6, term changed from 6 to 7, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to <none>, VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added, no longer has a pending config: VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1), VOTER 776baf137f8745dd866379ca15ab8cee (127.21.33.2), VOTER 6545453e74344bb2957dd3729c5b3870 (127.21.33.3), NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 7 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:05.350656 22766 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:05.199299 26683 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.459084 26683 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.459581 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.460304 22766 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:05.144693 26845 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:05.154556 26854 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.298156 26039 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.267053 26756 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:05.267271 26676 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.266825 26343 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.292611 26640 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:05.264673 26876 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.353871 22762 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:05.307808 26688 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.357113 23461 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:13:05.361912 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 224074us
I20260709 13:13:05.351404 26829 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.366086 26789 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:05.531613 26039 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:05.179437 26774 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:05.196062 26846 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:05.174428 26636 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.358376 26654 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:05.200912 26841 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.154345 26850 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:05.145838 26711 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:05.440122 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.268210 26696 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:05.431905 26695 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:05.461295 26683 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.461591 26868 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:05.565203 26839 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.002s
I20260709 13:13:05.562459 26756 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:05.463900 26573 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.573328 26869 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 13:13:05.726272 26774 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:05.609324 26842 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.003s
I20260709 13:13:05.461088 26394 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.730471 23460 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:05.731768 26715 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.592835 26789 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.461812 26854 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:05.715842 26844 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.785s	user 0.007s	sys 0.007s
I20260709 13:13:05.573611 26676 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.566504 26343 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.595031 26688 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:05.736575 26872 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.742527 26695 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.591231 26876 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.824604 26876 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:05.765633 26756 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:05.830245 26882 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
W20260709 13:13:05.834192 26867 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
W20260709 13:13:05.838199 26875 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
I20260709 13:13:05.778340 26842 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.851075 26636 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.840371 26593 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.854169 26676 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.758718 26868 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.768502 26711 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:05.793251 26527 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:05.863632 26593 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:05.867947 26846 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:05.778306 26789 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:05.796175 26788 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.001s
I20260709 13:13:05.796021 26573 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.796551 26640 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.798653 26394 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:05.801170 26879 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.484s	user 0.001s	sys 0.000s
I20260709 13:13:05.891705 26640 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:05.806670 26881 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:13:05.843925 26842 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.782336 26715 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.806021 26829 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.861096 26841 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.857144 26854 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.766161 26839 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.790637 26844 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.879956 26872 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.895079 26788 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.913038 26839 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.914759 26844 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.914920 26872 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.923434 26868 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:05.923758 26788 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.928038 26850 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.930485 26850 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.930688 26817 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:05.920411 26845 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.933410 26845 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.933662 26817 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:05.421727 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:05.974820 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 552934us
I20260709 13:13:05.456498 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.534189 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.985200 22765 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:05.986011 22761 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.012s	sys 0.044s
I20260709 13:13:05.985365 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:05.987088 22761 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.013s	sys 0.044s
I20260709 13:13:05.461153 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:05.990389 22766 raft_consensus.cc:2376] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8. Re-sending same reply.
I20260709 13:13:05.992528 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:05.377717 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 162684us
I20260709 13:13:05.548271 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:06.002703 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 458088us
W20260709 13:13:06.004144 23431 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.815s (SENT) [suppressed 11 similar messages]
I20260709 13:13:05.985445 22760 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:13:06.008311 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:05.619742 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:06.009270 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.009953 22908 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.010830 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:06.012478 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.012603 26715 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:05.606427 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:05.985351 22757 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:13:05.840912 26654 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.014439 22914 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.869470 26688 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:05.995462 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2517) took 1260 ms. Trace:
W20260709 13:13:06.005287 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 456973us
I20260709 13:13:06.014981 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:06.019117 26785 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.019s	user 0.008s	sys 0.004s
I20260709 13:13:06.020068 26785 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:05.989221 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.048640 22760 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:05.970909 22661 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 5 to 9, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 9 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.070451 22914 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:06.059216 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.081812 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:06.057893 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54084.3us
I20260709 13:13:06.063354 26789 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.054414 26527 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:13:06.055078 26593 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:13:06.053395 26785 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.069747 26844 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:13:06.111106 26844 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:06.111802 26844 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:06.082098 26868 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:13:06.055538 26715 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:13:06.116339 26715 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:06.016601 22909 rpcz_store.cc:276] 0709 13:13:04.735082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.816020 (+ 80938us) service_pool.cc:224] Handling call
0709 13:13:04.822279 (+  6259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:04.831495 (+  9216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:04.831524 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:04.835215 (+  3691us) log.cc:844] Serialized 26 byte log entry
0709 13:13:04.949276 (+114061us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b1800122860. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:13:05.084724 (+135448us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:05.084776 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.084863 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:05.994460 (+909597us) raft_consensus.cc:1685] finished
0709 13:13:05.994518 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.995090 (+   572us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8713,"raft.queue_time_us":11459,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":37542,"spinlock_wait_cycles":14133120,"thread_start_us":135241,"threads_started":1,"wal-append.queue_time_us":320300}
I20260709 13:13:06.103101 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2419) took 1054 ms. Trace:
I20260709 13:13:06.092622 26710 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.117117 26715 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:06.120852 22761 rpcz_store.cc:276] 0709 13:13:05.043775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.078806 (+1035031us) service_pool.cc:224] Handling call
0709 13:13:06.079569 (+   763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.081030 (+  1461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.081103 (+    73us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.081196 (+    93us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:06.098348 (+ 17152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.098444 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.098567 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:06.057075 26557 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.928s	user 0.003s	sys 0.010s
I20260709 13:13:06.124727 22908 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:06.116204 26868 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:06.088845 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:06.121071 26789 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.073012 22661 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.141909 26800 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.143378 22761 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:06.142702 22661 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.149806 26710 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.151067 26715 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.153137 26715 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:06.156174 22764 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:06.155869 26872 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:06.159153 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70943.7us
I20260709 13:13:06.161795 26593 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.172135 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.168658 22836 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:06.173295 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.174878 22764 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:13:06.176142 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.162979 26789 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:06.139257 26557 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.166301 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.180450 22837 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.167311 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.168612 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.163233 22762 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:13:06.145298 22661 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.175069 22834 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.188985 22833 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.190340 22838 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.132755 26868 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:06.153249 26710 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:06.199015 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.032s	sys 0.020s
W20260709 13:13:06.200167 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.033s	sys 0.021s
I20260709 13:13:06.201607 22841 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:06.213210 22764 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:13:06.212138 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75669.3us
I20260709 13:13:06.215572 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.217120 22841 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.228610 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.230158 22835 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.231078 26845 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.239823 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.246650 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:06.248874 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:06.250195 26844 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.252446 26844 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:06.257999 22762 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:13:06.259649 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.260339 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2418) took 1216 ms. Trace:
I20260709 13:13:06.261123 22757 rpcz_store.cc:276] 0709 13:13:05.043512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.059896 (+1016384us) service_pool.cc:224] Handling call
0709 13:13:06.060959 (+  1063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:06.062056 (+  1097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.062103 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:06.063686 (+  1583us) log.cc:844] Serialized 26 byte log entry
0709 13:13:06.197487 (+133801us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:06.222211 (+ 24724us) log.cc:844] Serialized 14 byte log entry
0709 13:13:06.249792 (+ 27581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.249905 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:06.260028 (+ 10123us) raft_consensus.cc:1685] finished
0709 13:13:06.260053 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.260224 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14475,"prepare.queue_time_us":17499,"raft.queue_time_us":29457,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"spinlock_wait_cycles":17160960,"thread_start_us":133046,"threads_started":1,"wal-append.queue_time_us":150858}
W20260709 13:13:06.264937 26039 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.013s
I20260709 13:13:06.265473 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.260385 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
W20260709 13:13:06.292943 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 82864us
I20260709 13:13:06.238430 26557 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.298723 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.239006 26868 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.235519 26845 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.314129 23459 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:13:06.314785 23459 pending_rounds.cc:85] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 8
I20260709 13:13:06.314858 26845 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.315440 23459 pending_rounds.cc:107] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:13:06.315860 23459 raft_consensus.cc:2889] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:06.325479 22835 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.335578 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.337366 26668 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:06.338610 26668 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.342821 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.343978 22835 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:06.340870 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:06.332578 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:06.346990 23466 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:06.352463 26756 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.527s	user 0.015s	sys 0.007s
W20260709 13:13:06.355628 26880 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.814s	user 0.003s	sys 0.000s
I20260709 13:13:06.360353 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2406) took 1388 ms. Trace:
I20260709 13:13:06.353076 26756 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.360879 22763 rpcz_store.cc:276] 0709 13:13:04.972053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.982544 (+ 10491us) service_pool.cc:224] Handling call
0709 13:13:04.983055 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:04.983510 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:04.983549 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:04.984148 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:13:05.084228 (+100080us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:05.084272 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.084348 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:06.354398 (+1270050us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:06.359697 (+  5299us) raft_consensus.cc:1685] finished
0709 13:13:06.359730 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.360110 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8408,"raft.queue_time_us":2906,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":8363,"spinlock_wait_cycles":15943936,"thread_start_us":99570,"threads_started":1,"wal-append.queue_time_us":366875}
I20260709 13:13:06.363649 22834 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:13:06.298062 26039 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.363976 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.367632 22836 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.374991 26557 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.375521 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2402) took 1406 ms. Trace:
I20260709 13:13:06.375901 26557 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.376760 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.377609 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.378147 22832 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:13:06.378894 22833 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.359916 26688 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:06.364845 26668 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.382905 26845 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:06.396188 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.396025 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:06.397450 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 148772us
I20260709 13:13:06.390053 26872 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.395640 26845 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.404448 26872 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.409193 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.410506 22834 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.411150 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.411890 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.376894 22758 rpcz_store.cc:276] 0709 13:13:04.968610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.972598 (+  3988us) service_pool.cc:224] Handling call
0709 13:13:04.972955 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:05.025999 (+ 53044us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.026050 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:05.034830 (+  8780us) log.cc:844] Serialized 26 byte log entry
0709 13:13:05.203903 (+169073us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b1800422ec0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:13:05.378588 (+174685us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:05.388052 (+  9464us) log.cc:844] Serialized 14 byte log entry
0709 13:13:05.388391 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.360738 (+972347us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b1800422ec0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419
0709 13:13:06.367511 (+  6773us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:06.367574 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:06.373667 (+  6093us) raft_consensus.cc:1685] finished
0709 13:13:06.373698 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.375315 (+  1617us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7290,"prepare.queue_time_us":228676,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":8461,"raft.queue_time_us":14878,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":3234,"spinlock_wait_cycles":159451520,"thread_start_us":183273,"threads_started":2,"wal-append.queue_time_us":815674}
I20260709 13:13:06.417716 26573 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:06.413110 26845 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:06.415797 26696 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.624s	user 0.017s	sys 0.005s
I20260709 13:13:06.425660 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:06.421001 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:06.429237 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.423589 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:06.419322 26573 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.432781 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.435369 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2420) took 1391 ms. Trace:
I20260709 13:13:06.448010 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.446189 22760 rpcz_store.cc:276] 0709 13:13:05.044007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.112428 (+1068421us) service_pool.cc:224] Handling call
0709 13:13:06.113397 (+   969us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:06.114771 (+  1374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.114833 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:06.116210 (+  1377us) log.cc:844] Serialized 52 byte log entry
0709 13:13:06.205640 (+ 89430us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:06.216504 (+ 10864us) log.cc:844] Serialized 14 byte log entry
0709 13:13:06.223774 (+  7270us) log.cc:844] Serialized 14 byte log entry
0709 13:13:06.434590 (+210816us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54002ba0a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6c733 00007f68a1af1dbf 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c
0709 13:13:06.434678 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.434805 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:06.435006 (+   201us) raft_consensus.cc:1685] finished
0709 13:13:06.435029 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.435253 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31204,"prepare.queue_time_us":10303,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":108,"raft.queue_time_us":31481,"raft.run_cpu_time_us":741,"raft.run_wall_time_us":747,"spinlock_wait_cycles":191324544,"thread_start_us":88809,"threads_started":1,"wal-append.queue_time_us":125447}
W20260709 13:13:06.437155 26695 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.619s	user 0.017s	sys 0.008s
I20260709 13:13:06.433815 26872 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.432155 26756 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.440943 26573 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:06.458654 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52899.4us
I20260709 13:13:06.459906 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.449915 26774 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:06.466921 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.454267 26696 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:06.469012 22758 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 7.
I20260709 13:13:06.471619 26854 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:06.465189 26774 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:06.480821 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80181.6us
I20260709 13:13:06.472512 26557 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.484163 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.474125 26854 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.459597 26695 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.475636 26905 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:06.490674 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:06.477154 26039 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:06.494428 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.495705 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:06.497331 22833 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:06.499267 22838 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.495580 26783 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.485852 26774 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:06.502094 26830 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:06.506197 26830 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:06.506961 26343 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.507228 26830 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.509958 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:06.510142 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.511446 26848 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.491060 26905 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.522472 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.523484 22833 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.491382 26854 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:06.536458 22840 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:13:06.530939 26710 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:06.540459 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.541658 22760 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:06.540647 26710 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:06.513085 26830 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.559810 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:06.558072 22652 maintenance_manager.cc:643] P 6e189717484b49e8bb20c98667e6d294: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 10.629s	user 0.226s	sys 0.161s Metrics: {"bytes_written":30014,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":658246,"dirs.run_cpu_time_us":683,"dirs.run_wall_time_us":56186,"drs_written":1,"lbm_read_time_us":426,"lbm_reads_lt_1ms":4,"lbm_write_time_us":368607,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":5,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":502372,"threads_started":2,"update_count":9,"wal-append.queue_time_us":192321}
I20260709 13:13:06.560920 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:06.564936 26896 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.658s	user 0.002s	sys 0.000s
I20260709 13:13:06.518610 26905 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:06.569660 26710 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:06.574260 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 616) took 1526 ms. Trace:
I20260709 13:13:06.579556 23460 rpcz_store.cc:276] 0709 13:13:05.048068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.313411 (+265343us) service_pool.cc:224] Handling call
0709 13:13:05.359609 (+ 46198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.771255 (+411646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.573603 (+802348us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400650254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:06.573963 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852320000}
W20260709 13:13:06.583513 26737 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.098s	user 0.020s	sys 0.006s
I20260709 13:13:06.607506 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:06.605279 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.613926 26737 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.717751 22833 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:06.716727 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.716840 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.719434 22763 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.720173 22836 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
W20260709 13:13:06.625401 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 150997us
I20260709 13:13:06.632908 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.598016 26695 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:06.644871 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.725232 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2409) took 1690 ms. Trace:
I20260709 13:13:06.725325 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.607506 26797 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.712352 22757 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:06.725317 22762 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:06.725926 22765 rpcz_store.cc:276] 0709 13:13:05.034991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.035121 (+   130us) service_pool.cc:224] Handling call
0709 13:13:05.046862 (+ 11741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.986537 (+939675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.608375 (+621838us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b540037fad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:06.725152 (+116777us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156136192}
I20260709 13:13:06.734254 26789 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.734907 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.739050 26789 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.740018 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.744200 23466 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:06.745343 22761 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:06.751377 26756 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:06.752116 26756 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.760144 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.767853 22765 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:06.775326 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.778883 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:06.780753 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 221081us
I20260709 13:13:06.785919 22758 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:06.789712 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.759676 26737 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.767261 26830 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.793442 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.794991 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.806437 22761 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.810791 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:06.816702 22764 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:06.823570 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.814013 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:06.827104 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:06.828147 22762 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.829280 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:06.832058 22765 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:06.838171 22836 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.848157 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:06.854120 22757 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.856673 26797 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:06.857336 26907 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:06.814342 26683 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.870009 26683 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:06.873647 26683 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:06.878829 26797 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:06.855821 22758 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:06.886683 22763 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:06.888502 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.898934 22763 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:06.902047 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.541s	user 0.030s	sys 0.097s
I20260709 13:13:06.913028 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.913003 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.919952 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.922566 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:06.931370 22758 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:06.930835 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.937819 22765 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.941694 22653 maintenance_manager.cc:419] P 6e189717484b49e8bb20c98667e6d294: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8147 bytes on disk
W20260709 13:13:06.947264 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 52854.3us
I20260709 13:13:06.949028 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.949898 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.950208 22838 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.951180 22833 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.952471 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:06.941744 22766 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:06.954413 26711 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.955300 26711 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.946933 22652 maintenance_manager.cc:643] P 6e189717484b49e8bb20c98667e6d294: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":637,"lbm_reads_lt_1ms":4}
I20260709 13:13:06.941902 26516 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:06.958665 26516 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:06.959219 26900 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:13:06.960832 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:06.964596 22763 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:06.901623 26756 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.961941 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:06.964918 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:06.975462 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.977115 22836 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:06.978825 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:06.979642 22764 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:06.994189 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 235703us
I20260709 13:13:06.854403 26830 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.999462 22914 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:07.001929 22840 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:06.930958 26789 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:06.899621 26774 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:06.998998 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.019574 26585 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:13:07.024022 26912 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.289s	user 0.000s	sys 0.002s
W20260709 13:13:07.034037 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 236754us
I20260709 13:13:07.037883 26854 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.039048 26711 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.041023 22761 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:07.042567 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:07.043480 22761 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.046025 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:07.049760 22762 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:07.060073 22765 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:07.061594 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:07.063988 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:07.064594 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:07.066586 22835 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:07.077324 22914 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:07.029664 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.080363 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:06.961437 26516 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:07.091210 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.875s	user 0.032s	sys 0.098s
I20260709 13:13:07.097275 22763 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:07.104506 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:07.114485 22766 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:13:07.120671 22765 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:07.121258 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.122916 22911 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:07.123134 22765 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:13:07.130843 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.142030 22913 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:07.112210 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.9353e+06us
I20260709 13:13:07.174407 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:07.174966 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 348856us
I20260709 13:13:07.175513 22766 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:07.108737 26710 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.503s	user 0.014s	sys 0.001s
I20260709 13:13:07.187522 22762 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:07.188045 22764 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:07.197062 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.208518 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.193586 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:07.218082 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 315054us
I20260709 13:13:07.220232 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.222491 22757 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:07.226919 22761 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:07.163537 26516 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:07.234829 22762 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:07.242488 22913 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.199133 26688 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:07.252494 22909 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.261134 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:07.264245 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 190108us
I20260709 13:13:07.223975 26573 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.270411 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.271433 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.281049 22907 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:07.278430 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.285923 22911 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:07.292014 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:07.292280 26845 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.843s	user 0.016s	sys 0.001s
I20260709 13:13:07.292213 22906 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:07.273010 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:07.199859 26710 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.281528 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.294910 22907 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:13:07.294790 26748 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:07.279733 22766 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:07.308770 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:07.310442 22910 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:13:07.311101 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 240887us
I20260709 13:13:07.312902 22909 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:07.284440 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.349169 22913 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:07.289295 26854 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.353634 22764 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:07.353848 22915 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:07.353853 22914 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:13:07.358232 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 85326us
W20260709 13:13:07.287187 26905 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.681s	user 0.018s	sys 0.000s
I20260709 13:13:07.360071 26905 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.362032 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.365082 22758 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:07.371877 22758 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:13:07.338310 26516 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.379829 26829 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.380741 26829 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.383411 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.385255 22840 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:07.385967 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.387056 22836 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:07.363493 26833 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:07.388988 26833 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:07.389489 26683 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:07.386045 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:07.415292 22906 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:07.363251 26905 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:07.430745 26830 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.451948 26830 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:07.390409 26833 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:07.431803 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.452667 26830 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.457278 22836 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:07.438874 26695 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:07.347904 26845 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.448288 26710 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:07.282191 26573 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:07.450830 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65205.5us
W20260709 13:13:07.474573 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 225866us
I20260709 13:13:07.463903 22836 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:07.433008 22832 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:13:07.492815 22915 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:07.500818 22763 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:07.504077 22759 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:07.503116 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:07.508610 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.505157 26695 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:07.527501 22836 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:07.533567 22836 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:07.505659 26710 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:07.542347 26585 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:07.542999 26585 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:07.527287 26695 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.547719 26774 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:07.548480 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 326389us
I20260709 13:13:07.550647 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:07.552179 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:07.553471 26830 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.555529 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.555925 26711 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.555864 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.556802 22915 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:13:07.556985 22906 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:07.557622 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.562583 22912 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:07.558094 26710 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:07.573068 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 212131us
I20260709 13:13:07.577486 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.578594 22915 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:07.563707 26756 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.556777 26711 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:07.582577 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 267026us
I20260709 13:13:07.570394 26907 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:07.594316 26854 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.594823 22906 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:07.594574 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.594940 26573 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.598387 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.599858 26756 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:07.600813 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 97837.3us
I20260709 13:13:07.601869 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.604442 26845 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.606763 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.616531 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:07.621798 22832 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:07.623051 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.628535 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.629370 22833 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:07.630050 26711 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:07.634152 22909 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:07.635565 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:07.639559 22911 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:07.643208 22835 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:07.570293 26710 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.648239 22914 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:07.651865 22915 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:13:07.623731 26573 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:07.654860 26573 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:07.659121 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:07.660701 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.661428 22911 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:13:07.669732 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 361222us
I20260709 13:13:07.671835 22912 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:13:07.684333 22879 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.540s (SENT)
I20260709 13:13:07.686231 26756 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.614264 26854 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:07.688755 26854 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:07.657289 26573 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:07.696173 22832 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
W20260709 13:13:07.698194 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62762.7us
I20260709 13:13:07.699841 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:07.701010 26905 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:07.701982 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:07.703460 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.705055 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.709587 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.711490 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.712471 22909 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:07.641840 26845 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.663028 26907 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:07.716305 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.720296 22914 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:07.725955 22833 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:07.759377 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.760360 22915 raft_consensus.cc:2376] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9. Re-sending same reply.
I20260709 13:13:07.763897 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:07.771956 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.784017 22911 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:13:07.784227 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.785471 22832 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:07.788224 22840 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.689401 26854 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:13:07.790647 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:07.793300 22832 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.766829 26710 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:07.806413 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104619us
I20260709 13:13:07.800064 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.807852 22915 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:07.800815 26748 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:07.801827 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:07.810668 23458 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:07.810966 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 283063us
I20260709 13:13:07.803894 26830 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:07.714308 26905 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:13:07.819546 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:07.816998 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.814983 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:07.816541 26830 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:07.828014 23461 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:07.800436 26932 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:13:07.829213 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:07.823374 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 247576us
I20260709 13:13:07.818723 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:07.798820 26845 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:07.826435 22832 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:07.845924 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:07.823701 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.856115 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 170153us
I20260709 13:13:07.846673 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:07.858629 22833 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:07.860124 22915 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:07.841996 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:07.862972 23466 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:07.832384 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:07.856055 23463 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:07.860884 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:07.876420 23460 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:07.879559 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:07.880295 26868 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:07.880522 23463 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:07.894198 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:07.908089 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:07.909142 22763 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.910413 22758 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:07.822769 26774 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:13:07.908082 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:07.817387 26585 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:07.923892 26924 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:13:07.925225 22757 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:07.890859 26933 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:07.934620 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:07.935104 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:07.926405 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:07.945178 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.951601 22764 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:07.953110 22760 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:07.955917 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:07.963313 22762 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:07.964939 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:07.973781 22907 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:07.983675 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87853.9us
I20260709 13:13:07.985288 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 627) took 1340 ms. Trace:
I20260709 13:13:07.986132 26829 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.962167 26830 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:07.989830 22758 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
W20260709 13:13:07.955856 22879 connection.cc:256] RPC call timeout handler was delayed by -1.348952977999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:13:08.006639 26830 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.007329 26516 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:08.008821 26829 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:07.985337 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.009464 26829 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:08.006875 22879 connection.cc:256] RPC call timeout handler was delayed by -1.1212529779999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 13:13:08.013171 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 113225us
I20260709 13:13:08.015089 22840 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:08.022540 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:08.023883 23463 raft_consensus.cc:2368] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
I20260709 13:13:08.024322 26585 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:07.951467 26573 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:08.027016 23432 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.553s (SENT)
W20260709 13:13:08.028244 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 85616.5us
I20260709 13:13:08.028918 22837 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:07.955507 26940 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:07.970088 26756 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:08.035948 22763 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:07.985788 23457 rpcz_store.cc:276] 0709 13:13:06.603182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.603647 (+   465us) service_pool.cc:224] Handling call
0709 13:13:06.712144 (+108497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:06.792974 (+ 80830us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:07.297192 (+504218us) log.cc:844] Serialized 14 byte log entry
0709 13:13:07.606316 (+309124us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:07.622838 (+ 16522us) log.cc:844] Serialized 26 byte log entry
0709 13:13:07.623036 (+   198us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:07.623073 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.623144 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:07.943513 (+320369us) raft_consensus.cc:1685] finished
0709 13:13:07.943536 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:07.943673 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28938,"prepare.queue_time_us":476222,"prepare.run_cpu_time_us":162,"prepare.run_wall_time_us":162,"raft.queue_time_us":46321,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":9512,"thread_start_us":657951,"threads_started":2,"wal-append.queue_time_us":493386}
I20260709 13:13:07.991544 26907 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:08.050762 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.025187 26585 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 13:13:08.062999 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48635.6us
I20260709 13:13:08.066527 23460 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:07.994053 26934 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:13:08.075765 23463 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:08.030980 26573 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.062301 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "072f69118119423dac7eba3f66879eee"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:08.095402 22910 raft_consensus.cc:606] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.096160 22910 raft_consensus.cc:614] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:08.098958 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.081794 26516 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:13:08.074930 26907 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:08.070124 26927 raft_consensus.cc:2764] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:08.116565 26845 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:08.117491 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.117748 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.117875 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:08.118481 23464 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:08.078485 26756 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.119011 26573 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:08.119091 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:08.106096 26829 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:08.122306 26756 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:08.123638 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57540.3us
I20260709 13:13:08.141247 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:08.086768 26937 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:13:08.131911 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.132352 26711 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:08.112738 26907 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.134588 26833 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.108970 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1730) took 1603 ms. Trace:
I20260709 13:13:08.135141 26923 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.135669 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:08.139081 26942 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.260s	user 0.004s	sys 0.000s
I20260709 13:13:08.161888 23463 raft_consensus.cc:2368] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12.
I20260709 13:13:08.156164 22759 rpcz_store.cc:276] 0709 13:13:06.510744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.511819 (+  1075us) service_pool.cc:224] Handling call
0709 13:13:06.512183 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.507294 (+995111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.108519 (+601225us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b540035eb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:08.108884 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280609536}
I20260709 13:13:08.146816 26711 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.155272 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.146183 26573 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:13:08.162882 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86309.7us
I20260709 13:13:08.164310 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.164680 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:08.163494 26845 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:13:08.165952 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.163455 26833 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.177888 26833 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:08.078775 26940 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.180611 26748 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:08.181128 26748 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:08.107206 26933 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.182425 26748 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.173260 26951 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:08.169806 26711 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:08.184429 26951 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.185884 26951 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:08.186379 26928 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.356s	user 0.003s	sys 0.000s
W20260709 13:13:08.183730 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80570.2us
I20260709 13:13:08.194334 26683 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:08.194916 26683 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:08.195902 26683 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.163610 26923 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.173889 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.204479 26940 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:08.174952 26516 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:08.182761 26933 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.205163 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.173672 26845 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:08.173293 26774 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:08.206259 26934 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:08.213644 26695 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:08.215497 26748 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.214027 26774 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.217147 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.215559 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2437) took 1212 ms. Trace:
I20260709 13:13:08.218634 26774 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:08.218021 22765 rpcz_store.cc:276] 0709 13:13:07.004174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.125154 (+120980us) service_pool.cc:224] Handling call
0709 13:13:07.159674 (+ 34520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.191975 (+ 32301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:07.192028 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:07.233817 (+ 41789us) log.cc:844] Serialized 26 byte log entry
0709 13:13:07.572884 (+339067us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:07.572944 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.573048 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:08.185449 (+612401us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:08.214891 (+ 29442us) raft_consensus.cc:1685] finished
0709 13:13:08.214919 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.215482 (+   563us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15950,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":10725,"spinlock_wait_cycles":5599104,"thread_start_us":302994,"threads_started":1,"wal-append.queue_time_us":517649}
I20260709 13:13:08.220621 26945 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.221405 26945 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.221859 26945 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.223419 26934 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:08.225713 26573 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:08.226688 26573 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:08.227026 26573 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:08.228163 22835 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:08.228034 26573 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.213124 26516 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.231503 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:08.232297 26516 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:08.233330 22730 leader_election.cc:400] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:13:08.233685 22730 leader_election.cc:403] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:08.234578 26950 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.235792 26927 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:08.209009 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.213085 26683 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:08.213163 26923 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:08.206571 26833 raft_consensus.cc:2764] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:08.213562 26845 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:08.241192 23458 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:08.249686 22879 leader_election.cc:400] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10.
I20260709 13:13:08.250919 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.167343 26756 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.229336 26573 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:08.252308 26905 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:08.292704 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.292963 26905 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:08.282464 26829 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:08.294059 26829 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.294301 26905 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.295320 26829 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.296252 26905 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.259574 26585 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.297710 26585 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.298154 26585 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.298242 26919 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:08.246856 26927 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:13:08.299469 26919 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.301095 26927 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:08.301602 26927 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:08.303615 26927 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.267522 26774 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:13:08.307493 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.316584 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 150731us
I20260709 13:13:08.324627 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.321573 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.333616 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.337898 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:08.339164 22765 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 12 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:13:08.342864 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:08.343802 22762 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:08.343935 26919 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:08.344583 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:08.345762 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:08.346503 22759 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:08.346654 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:08.348191 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:08.348419 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55899us
W20260709 13:13:08.352735 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 81329.9us
I20260709 13:13:08.356937 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.358376 26925 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:08.350346 26953 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.050s	user 0.003s	sys 0.000s
I20260709 13:13:08.370741 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.361218 26951 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:08.336009 26833 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:08.360788 26925 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:08.336620 26940 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:08.386842 22759 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:13:08.379252 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45784.6us
I20260709 13:13:08.379333 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:08.389578 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:08.364358 26927 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.390285 22766 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:08.367218 26695 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:08.401417 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.410233 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59427.1us
W20260709 13:13:08.411939 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 162623us
I20260709 13:13:08.414055 26941 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:08.419404 26945 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.376477 26938 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:13:08.423110 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.387476 26934 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:08.429396 26934 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.429853 26934 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:08.404066 26845 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.411425 26940 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.439640 26940 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:08.409691 26944 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:08.444389 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:08.388337 26925 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.442127 26951 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.463677 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:08.405798 26695 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.467388 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.471832 22879 leader_election.cc:400] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12.
I20260709 13:13:08.472105 22879 leader_election.cc:403] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:13:08.460199 26940 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.470077 26695 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:08.477869 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.478178 26955 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12.
W20260709 13:13:08.483808 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.472911 26944 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.465384 26756 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:08.496177 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73160.1us
I20260709 13:13:08.497248 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:08.498872 26711 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.502665 26711 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:08.487723 26944 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.506240 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.507078 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.507265 22911 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:13:08.507174 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.507961 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.508814 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.510562 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.511334 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.517879 22915 raft_consensus.cc:2368] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:08.471038 26774 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:08.513892 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.470620 26932 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:08.470928 26923 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.518329 22909 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:08.517916 22912 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:08.473367 26833 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:13:08.519766 26774 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:08.522624 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.528373 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.494416 26907 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:08.529109 22912 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:08.531141 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.535490 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.537730 22906 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:13:08.541111 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73774.2us
I20260709 13:13:08.521207 26945 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:08.543396 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77357us
I20260709 13:13:08.527316 26923 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.548312 26923 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.548545 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.549904 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.550624 22912 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:08.550942 22912 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:08.532533 26737 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:08.551702 22912 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.552735 22912 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:08.532712 26774 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:08.549966 26789 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:08.554253 26789 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.555128 26789 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.556689 26789 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:08.559010 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.561527 22909 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:08.496127 26940 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.565140 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:08.574643 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47447.4us
I20260709 13:13:08.564301 26737 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.544384 26951 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.577348 26833 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:08.567930 26933 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:08.631556 22909 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:13:08.575678 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:08.630399 22912 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:13:08.629550 26833 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:08.636664 26833 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.638540 26585 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.640690 26950 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:13:08.642247 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.643083 22912 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:13:08.643337 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.644004 22908 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:08.644125 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.645279 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.645619 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.645902 22914 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:08.645800 26774 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:08.646314 22906 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:08.646606 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:08.631587 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 88031us
I20260709 13:13:08.648435 26774 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:13:08.656816 22907 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:08.657109 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.656715 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.661324 22912 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:08.657998 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:08.668314 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 282761us
I20260709 13:13:08.669091 22914 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:13:08.669178 22915 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:08.670674 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.671754 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:08.676187 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100614us
I20260709 13:13:08.676913 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:08.674134 26748 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:08.681921 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:08.683441 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:08.683818 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 118754us
I20260709 13:13:08.687054 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.687803 23458 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:08.691037 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.691702 22915 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:08.658943 26829 raft_consensus.cc:2764] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:08.699425 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.703868 22907 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:08.641224 26695 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.701398 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.707894 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:08.708109 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.708906 22906 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:08.709245 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:08.709919 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59675.7us
I20260709 13:13:08.710969 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:08.679046 26905 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:13:08.735795 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58944us
W20260709 13:13:08.733724 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48692.4us
I20260709 13:13:08.741736 22915 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:08.702399 26710 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:08.700562 26748 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:08.752700 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.710848 26573 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:08.771363 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:08.772951 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.773774 22915 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:08.749491 26945 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:08.738831 26516 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.750085 26585 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:08.777871 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.749855 26907 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.705725 26950 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:13:08.783638 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81321.2us
I20260709 13:13:08.795353 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.779448 26711 raft_consensus.cc:2764] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:08.804296 23458 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:08.795281 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111918us
I20260709 13:13:08.800668 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.774430 26573 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:08.806634 26854 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:08.807317 26573 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:08.807891 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:08.801590 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.772964 26930 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.810024 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.812235 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49452.4us
I20260709 13:13:08.758366 26737 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.781157 26516 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.815884 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.776467 26940 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:08.790819 26829 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:08.822405 26695 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:08.828375 26961 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:13:08.828480 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.829840 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.848301 22906 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:13:08.830637 22732 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.637s (SENT)
I20260709 13:13:08.833995 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.850521 22913 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:08.834769 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.852281 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.853307 22912 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:08.791013 26907 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:08.837117 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:08.863684 22766 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:13:08.854166 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.854763 26955 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:08.855444 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.799508 26945 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.866228 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:08.867591 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.845634 26952 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:08.864777 22909 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:08.877136 22913 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:08.918174 23458 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:08.818194 26966 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.823004 26516 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.875787 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.868642 26907 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.810233 26930 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.876194 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:08.878314 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.930325 22907 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:08.883940 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.900840 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:08.897744 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.916119 22732 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.694s (SENT)
I20260709 13:13:08.897991 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:08.884920 26683 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.007s
I20260709 13:13:08.926474 22912 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:08.826642 26756 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:08.883471 26941 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:08.932588 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:08.933894 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 114681us
I20260709 13:13:08.934474 23464 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:08.937027 23459 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 9: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:08.846400 26933 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:13:08.857625 26829 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:08.865619 26711 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:13:08.898576 26952 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:08.932446 22732 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.699s (SENT)
I20260709 13:13:08.931236 22911 raft_consensus.cc:2376] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8. Re-sending same reply.
W20260709 13:13:08.949523 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.035136 26945 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.035770 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:09.036464 26925 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.000s
I20260709 13:13:09.037093 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.038689 26941 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.040043 26941 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.019821 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:09.042405 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.043777 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.045033 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.939555 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:09.049860 22909 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:13:09.050735 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:08.951016 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.051362 22909 raft_consensus.cc:2376] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8. Re-sending same reply.
I20260709 13:13:08.925961 26966 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:09.053835 26966 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.054342 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.055986 26833 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:08.956118 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.058167 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.058928 22912 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:09.059728 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.060464 22912 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:13:08.993026 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:09.061656 22909 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:09.062088 23458 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.064036 26930 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:08.924481 26845 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.069826 26933 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:09.070267 26933 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.071398 26933 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.036888 26925 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.075083 23463 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:09.082278 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:08.898808 26934 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.051847 22906 raft_consensus.cc:2368] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:09.034852 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.084053 22914 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:09.050783 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:08.936933 26756 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:08.996760 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.095849 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.090132 22912 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:13:09.102326 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:09.104393 22910 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:09.038689 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.108122 22911 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
W20260709 13:13:09.109110 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 176717us
I20260709 13:13:09.085626 26925 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:09.034549 26965 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:13:09.118064 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:09.114954 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.120631 22836 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:09.115056 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:09.093657 23458 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:13:09.113149 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.085745 26756 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.085454 26934 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:09.001612 26737 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.073367 26933 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.133778 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.135049 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.124094 26797 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:09.137590 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:09.138227 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:09.149104 23457 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:09.138191 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.138924 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:09.140997 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.152911 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.147897 23458 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:13:09.106192 26952 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:09.155541 23466 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:09.144240 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:09.144701 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:09.145663 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:09.138547 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.194324 23461 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.194538 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:09.196101 23460 raft_consensus.cc:2368] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12.
I20260709 13:13:09.197532 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.199393 22834 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:13:09.155753 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 293144us
I20260709 13:13:09.116860 26947 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:09.190146 23459 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.191020 26797 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.190523 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.193325 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.204283 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:09.205572 26979 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:09.206386 26972 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:09.206228 26979 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:09.202602 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50070.4us
W20260709 13:13:09.201633 26965 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:13:09.215781 26829 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.220072 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:09.200753 26947 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.222251 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:09.223608 26947 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:09.224030 26789 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.667s	user 0.016s	sys 0.000s
I20260709 13:13:09.142997 26925 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:13:09.149044 26737 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.227049 26955 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:09.227514 26737 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:09.228297 26934 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:09.229003 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.229703 26683 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:09.230090 26971 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:09.230698 26971 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.231110 26971 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:09.231809 26833 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:09.198367 26975 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.232724 26975 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.234088 26975 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.235603 26975 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.160840 26830 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:13:09.201840 26930 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:09.238813 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.239200 26573 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.241402 26930 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:09.244992 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.247828 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.254187 26977 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:09.255182 26977 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:09.256145 26977 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:09.256738 26977 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:09.257020 26977 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.257795 26955 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.259212 26955 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.264014 22908 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:09.264567 22840 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:09.265250 22841 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:09.264990 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.267409 26930 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:09.268404 26930 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:09.268931 26930 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:09.274257 26973 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:13:09.274600 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.275607 22841 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:09.276461 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.271454 26971 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.277635 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:09.278380 22834 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:09.279089 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.279579 26971 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:09.287976 26977 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.289682 26977 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:09.290987 23466 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:09.293812 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:09.294495 23457 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9. Re-sending same reply.
I20260709 13:13:09.298494 26516 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.299038 26966 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.300235 22915 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:09.300431 23462 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:09.302649 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.302933 26573 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:09.306258 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:09.140141 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:09.196878 22730 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.530s (SENT)
I20260709 13:13:09.197117 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:09.312023 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 115194us
I20260709 13:13:09.146867 26585 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:09.314476 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:09.313848 22730 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.582s (SENT)
I20260709 13:13:09.315161 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:09.316712 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 114253us
I20260709 13:13:09.317534 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.318280 22836 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
W20260709 13:13:09.329708 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.342633 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:09.341765 23459 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:09.349001 23458 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [9.10-11.11]
I20260709 13:13:09.343045 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.350657 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:09.351933 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:09.322504 26585 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:09.352870 26934 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:09.353067 26585 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:09.323104 26683 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:09.316437 26966 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:09.357826 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 217777us
I20260709 13:13:09.341066 26979 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:09.360668 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47045.2us
I20260709 13:13:09.363525 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.351493 26971 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:09.366539 22730 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.619s (SENT)
I20260709 13:13:09.367012 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:09.366055 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48639.5us
I20260709 13:13:09.366233 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:09.373533 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:09.369277 22732 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.530s (SENT)
I20260709 13:13:09.365427 26941 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:09.374725 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.375052 26941 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.375629 26941 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.373193 26872 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:13:09.376811 26963 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.370958 26573 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:09.341248 26833 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:13:09.379464 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.384531 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.387523 22836 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:09.364995 26683 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:09.406262 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.377631 26963 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:09.388502 26971 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:13:09.413362 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.419927 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:09.372561 26934 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.420657 22766 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.366012 26976 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:09.426321 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 181442us
I20260709 13:13:09.427345 22764 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:13:09.380091 26978 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:09.389628 26979 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:09.431121 22759 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:09.432596 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:09.433583 22763 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.432719 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:09.417400 26938 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:09.431486 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:09.372438 26789 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:13:09.414225 26948 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:09.444119 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.446125 26963 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.447556 26963 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.437513 26979 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.431979 26854 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:09.419085 26683 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.452348 26947 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:09.454182 26737 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:09.454655 26737 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.455084 26516 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:09.455303 26789 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:09.455791 26789 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.426972 26934 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.449997 26979 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:09.456415 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89917.5us
I20260709 13:13:09.457443 22761 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:13:09.463557 22730 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.637s (SENT)
I20260709 13:13:09.439606 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.464078 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.464807 22840 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:13:09.456284 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:09.466086 22764 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.469115 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.473475 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:09.474434 22766 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.477579 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:09.478462 22762 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.480551 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.455561 26737 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.482112 22835 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:09.482564 22839 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:09.390151 26797 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.462103 26830 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.461890 26976 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.498881 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:09.494653 26854 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:09.470111 26948 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.499090 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:09.506695 22730 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.694s (SENT)
I20260709 13:13:09.507257 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:09.493755 26923 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.945s	user 0.007s	sys 0.008s
I20260709 13:13:09.478462 26966 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.523408 26756 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.525868 26966 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.526902 22907 raft_consensus.cc:2368] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:09.510268 23458 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:09.528770 26756 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:09.491863 26833 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:13:09.511193 22760 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.486346 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:09.504127 22764 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:09.515312 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.522832 26955 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:09.523489 26845 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:09.523324 26984 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:09.524647 26948 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.536190 22763 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.543943 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:09.549149 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:09.549971 22764 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.551466 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:09.552381 22764 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:13:09.553165 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 124737us
I20260709 13:13:09.562172 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.566260 22835 raft_consensus.cc:2368] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:09.491947 26516 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:13:09.531301 26737 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.509913 26573 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:09.577683 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.529778 26966 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:09.601090 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.601668 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.567942 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:09.500976 26830 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.537335 26955 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:09.598253 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 129256us
I20260709 13:13:09.502636 26971 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:13:09.607149 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:09.616930 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 123838us
W20260709 13:13:09.618232 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50410.2us
I20260709 13:13:09.621265 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.536986 26923 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:09.652421 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.653151 22732 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.994s (SENT)
I20260709 13:13:09.662087 22763 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:09.509156 26789 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.672398 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.609256 26955 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:09.675896 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 56792.4us
I20260709 13:13:09.616150 26833 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.664796 26695 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:09.684499 22839 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:13:09.680356 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64222.6us
I20260709 13:13:09.618858 26978 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.605321 26984 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
W20260709 13:13:09.690327 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 69048us
I20260709 13:13:09.693665 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.707105 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:09.707479 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.721076 22915 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:13:09.723696 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.735518 22836 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:09.736835 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:09.675364 26951 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.738260 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.737654 22832 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:09.744921 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:09.683257 26516 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.750471 22913 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:09.741314 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.752627 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.757416 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:09.682781 26833 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.761029 22911 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:09.762770 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.764652 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.745148 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:09.745404 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:09.746800 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.769322 22834 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:09.763741 22841 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:09.776898 22907 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:09.773360 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.777644 22908 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:09.782313 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:09.786266 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.786649 22909 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:13:09.788378 26991 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:09.788086 26998 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:09.786525 26994 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:09.688022 26976 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:09.789909 26983 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:13:09.790279 26994 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.791308 26994 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:09.794600 26982 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:13:09.719903 26932 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.794740 22760 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:09.802986 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:13:09.722913 26695 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.726604 26993 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:09.803907 22837 raft_consensus.cc:606] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:09.808625 22837 raft_consensus.cc:614] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:09.804611 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 183433us
I20260709 13:13:09.748822 26789 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:09.743475 26938 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:09.698151 26944 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
W20260709 13:13:09.818370 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60932.6us
I20260709 13:13:09.761965 26516 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.807550 26573 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:09.762413 26930 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.834897 26923 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:09.835572 26934 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:09.836721 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.838663 26830 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:09.840246 26986 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:13:09.844826 26978 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.845711 26992 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:13:09.816102 26932 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.851902 22832 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:13:09.797571 26977 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.507s	user 0.013s	sys 0.000s
I20260709 13:13:09.785068 26845 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:09.797169 26854 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:09.817883 26952 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.002s
W20260709 13:13:09.870980 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 116816us
W20260709 13:13:09.882063 26975 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.007s
I20260709 13:13:09.865005 26868 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.893711 22765 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:09.776672 26951 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.910216 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.912635 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:09.892795 26830 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:09.891801 26845 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.916023 26938 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.897584 26933 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.917893 26933 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.920078 26933 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:09.923054 26907 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.983s	user 0.008s	sys 0.005s
I20260709 13:13:09.923593 26907 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.924197 26932 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:09.914222 27000 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.886087 26711 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:13:09.920518 26845 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.913975 26573 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.931720 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.943239 26907 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:09.943883 27000 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.897426 26952 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.951050 26955 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.951205 26979 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:09.897400 26868 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.958984 26994 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:09.974882 26998 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:09.975322 26998 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.956960 23458 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:09.941066 26941 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.565s	user 0.014s	sys 0.001s
I20260709 13:13:09.952029 26695 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.924225 26992 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:09.974004 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 288171us
W20260709 13:13:09.979013 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47379.9us
I20260709 13:13:09.980984 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:09.971797 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:09.987056 23458 raft_consensus.cc:2410] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:13:09.986874 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:09.982570 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 209317us
I20260709 13:13:09.988452 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.993950 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:09.998780 22912 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:09.920645 26933 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:13:10.003942 22914 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:09.991333 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:09.989229 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:09.973456 26930 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:09.973296 26919 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:09.986222 26789 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:09.941061 26932 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:13:10.012215 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.026672 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.027544 22832 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.017611 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.018877 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:09.984716 26994 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:09.997182 26975 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:10.030591 22730 leader_election.cc:400] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:10.038560 22730 leader_election.cc:403] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:10.031462 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:10.029237 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:10.039338 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 437726us
I20260709 13:13:09.997258 26907 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.003777 26737 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:10.033185 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.012290 26984 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:10.016443 26923 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:10.036722 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:09.985894 26941 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:09.953990 26516 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.022014 26919 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:10.042137 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.030869 26966 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:10.037741 26948 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:10.031014 26748 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:10.030265 26925 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.048085 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.030088 26789 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:10.048920 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:10.032876 27009 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:10.038471 26975 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:10.053802 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:10.037242 26977 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.068377 27007 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:10.069337 27007 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.051151 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.061324 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:10.062732 23464 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:10.043221 26737 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:10.064319 26797 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:10.096889 26830 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:10.070034 26925 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.054010 26919 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.062733 26789 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.033828 26998 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.049114 26923 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.081422 27000 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.083389 26966 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:10.083765 26683 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.633s	user 0.013s	sys 0.000s
I20260709 13:13:10.068735 26989 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:13:10.084878 26955 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.069478 26999 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:13:10.051362 26963 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:10.076545 26994 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.046201 26984 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.094275 23463 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.065673 26748 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.096019 26941 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.102180 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:10.160250 23464 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.108789 26998 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.111352 26925 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:10.101284 26952 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.112884 26975 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.033707 26868 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.176241 26868 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.097745 26737 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:10.178546 26979 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.097699 26977 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.165683 26955 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:10.182452 26955 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.184162 26977 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.045750 26907 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.094264 27007 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.118826 26797 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:10.190255 23464 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:10.051036 26978 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:10.122630 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 313114us
W20260709 13:13:10.095001 26585 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.004s
W20260709 13:13:10.129374 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78295.4us
I20260709 13:13:10.094801 26992 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:10.140988 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:10.099359 26948 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:10.131068 26923 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:10.170019 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 111368us
I20260709 13:13:10.212652 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.212951 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.173794 26951 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:10.120441 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.137948 26683 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.225764 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:10.227283 22757 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:13:10.228025 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.229336 22763 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:10.158973 26984 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:10.194051 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.191175 26971 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:13:10.172398 27004 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:13:10.211114 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:10.261996 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51003.5us
I20260709 13:13:10.263926 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.268316 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:10.269872 22757 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.271121 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:10.272212 22757 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.232893 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.211192 26992 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:10.253126 27014 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:13:10.305894 26992 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:10.254665 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67822.4us
I20260709 13:13:10.254739 22760 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:10.309005 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.255587 22761 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:13:10.311616 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:10.312304 22761 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:10.211082 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:10.150007 26966 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.209977 26975 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.260015 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.317018 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:10.317732 26748 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.318517 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:10.320248 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.321043 22759 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
W20260709 13:13:10.279527 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 66981.9us
I20260709 13:13:10.322397 23459 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:10.260072 26955 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.258173 26971 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:10.273939 26979 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:10.222748 26585 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.276012 26963 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:10.276918 26573 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:10.299019 22764 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:10.301004 26683 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:10.310235 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51471.4us
I20260709 13:13:10.279038 22758 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:10.310245 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:10.322598 23463 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:10.286811 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:10.221865 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.335994 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:10.353048 22836 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.338292 22757 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:10.343194 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:10.356026 23464 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:10.345466 22764 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:10.337231 22879 leader_election.cc:400] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12.
I20260709 13:13:10.361392 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.361843 22761 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
W20260709 13:13:10.368458 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.373409 22833 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:10.377946 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55007.7us
I20260709 13:13:10.380235 22762 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:10.383652 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.375902 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.388096 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.388381 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.388999 22915 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:10.371879 26830 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.391084 22836 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:10.340730 26999 consensus_queue.cc:1243] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:10.386127 22910 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:10.393620 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.394158 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.394550 22910 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:10.397158 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.398633 22761 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:13:10.399415 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:10.399874 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.400527 22764 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:10.389564 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.403239 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.404345 22909 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:10.406010 22836 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:10.405858 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:10.407914 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.288331 26948 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.372418 26947 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:10.356489 27000 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:10.414186 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.426355 22913 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:13:10.484138 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.486215 22757 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:10.488426 26947 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:10.489322 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.490370 22913 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:10.488438 22915 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.297801 26797 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.418915 22906 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.399086 22761 pending_rounds.cc:85] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 5
W20260709 13:13:10.475558 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.494711 22761 pending_rounds.cc:107] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:13:10.478099 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:10.421082 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.495132 22761 raft_consensus.cc:2889] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.495577 22761 pending_rounds.cc:107] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:13:10.495837 22761 raft_consensus.cc:2889] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.496620 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.497255 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.498014 22763 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:10.499137 22910 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.400557 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:10.483533 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101167us
I20260709 13:13:10.483875 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.500540 22760 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:10.484705 22841 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:10.484931 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.501618 22909 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.485236 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:10.390925 26977 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:10.365981 27009 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:10.493048 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.503036 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:10.504141 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 103741us
I20260709 13:13:10.417954 26830 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.588439 22913 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:10.593839 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.593839 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:10.489594 26947 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.586081 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:10.588796 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 85856.5us
I20260709 13:13:10.589546 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.608640 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.609462 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.504273 26955 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:10.524520 26585 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.503398 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.497939 27022 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:13:10.475704 27025 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.167s
I20260709 13:13:10.621447 22915 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:10.603919 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.613409 26748 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:10.611116 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:10.573043 22909 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:10.525010 27001 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.612098 22906 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:10.621696 26845 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.646s	user 0.004s	sys 0.009s
I20260709 13:13:10.474367 26971 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.602996 26930 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:10.593967 26979 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:13:10.585850 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 92865.3us
I20260709 13:13:10.610538 26977 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:10.502360 26999 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.587699 26907 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:10.635851 22910 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:10.640014 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.504344 26989 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 13:13:10.623175 22907 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:10.495064 27000 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.532807 26919 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:10.650198 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73167.4us
I20260709 13:13:10.648950 22880 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.652451 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.632853 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:10.648727 26585 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:10.666195 22913 raft_consensus.cc:606] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.647912 27015 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:10.642819 26748 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:10.655478 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 109002us
I20260709 13:13:10.621398 26992 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:10.657914 22730 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.701s (SENT)
I20260709 13:13:10.660957 22909 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:13:10.633889 26998 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:10.653373 26930 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.588824 27009 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.659318 26971 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.684443 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.653566 27025 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:13:10.654606 26845 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.670758 26748 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.669164 26585 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:10.669337 27015 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.690459 22908 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:10.673882 26930 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.676349 26977 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:10.705365 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.708637 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.709334 22909 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:10.714334 26951 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:10.716356 26951 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:10.717294 27031 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:10.718079 27029 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:13:10.718575 27031 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:10.718881 27029 consensus_queue.cc:1243] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:10.718979 27031 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:10.719514 27029 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:10.720132 27029 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.719955 27031 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.720706 23464 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:13:10.686702 27020 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:13:10.729211 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:10.780757 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51665.5us
I20260709 13:13:10.784557 22910 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:10.784986 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:10.785975 22913 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.782986 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.690187 26955 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:10.705267 26925 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:10.802125 26925 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.720607 27029 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.659090 26999 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.789908 26573 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:10.653748 27001 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.678264 26979 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:13:10.797204 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:10.799178 26737 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.001s
I20260709 13:13:10.721333 27031 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.819725 22909 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:10.863559 22909 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:10.695945 27015 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.840782 26585 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.821892 26573 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:10.689404 26998 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 13:13:10.813956 26683 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:10.805460 26925 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:10.852945 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:10.816017 27029 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:10.860062 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.808488 27000 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.781935 26907 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:10.828274 26923 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
W20260709 13:13:10.862592 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65485.5us
W20260709 13:13:10.919317 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.842218 26992 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:10.794302 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 109941us
I20260709 13:13:10.864634 22909 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.870085 26963 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:10.879302 22913 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:10.819655 26845 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:10.889925 26998 consensus_queue.cc:1243] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:10.890796 26919 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.891201 26830 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:10.891741 26966 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:10.893754 27001 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:10.878811 27020 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:13:10.940734 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:10.942323 23459 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:10.896672 22908 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:10.897783 26585 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:10.866552 26737 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.880491 26683 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.855103 27009 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:10.906260 23432 leader_election.cc:336] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.092s (SENT)
I20260709 13:13:10.907687 22910 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:10.951038 22912 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:10.874871 26868 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.004s
W20260709 13:13:10.914779 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.951136 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.926498 26951 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.923149 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:10.925617 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:10.918428 26984 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.000s
I20260709 13:13:10.934857 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:10.957500 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.958302 22913 raft_consensus.cc:606] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.958683 22913 raft_consensus.cc:614] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:10.937042 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:10.959621 22909 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.959957 22762 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:10.922808 27005 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.953224 22766 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.950304 26683 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:10.967988 22913 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:10.964843 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.938076 26963 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:10.927922 26923 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:10.971580 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:10.973097 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:10.974659 23464 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:10.976624 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:10.922540 26907 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:10.980686 26737 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.971611 26994 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:10.983989 26868 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:10.970350 27001 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.990151 23459 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:10.990185 22908 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:10.953150 26966 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.991374 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:10.995270 22909 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:10.962000 26919 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:10.972074 26998 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:11.015750 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.017334 22762 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 7.
I20260709 13:13:11.016479 27001 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.026827 26963 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.027397 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.028187 27001 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:10.989737 26992 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:11.034789 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 128530us
I20260709 13:13:11.032567 22766 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:11.036963 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:10.997578 26988 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.044644 22766 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:10.999917 26907 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:10.963073 26951 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.061918 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.062783 22834 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:11.062937 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.063488 22835 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.011199 26995 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:10.961882 26984 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.066721 26977 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:11.068928 26984 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.069844 22906 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:11.042461 26994 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:13:11.042928 26925 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:11.072710 26925 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.034188 26992 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.056380 26797 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.055941 26919 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:11.074230 26992 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:11.074790 26919 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:11.076385 26919 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.077529 26995 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:11.080276 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.081053 22762 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
W20260709 13:13:11.058215 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.036298 26789 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.886s	user 0.015s	sys 0.000s
I20260709 13:13:11.054572 23464 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:10.971282 27005 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.061437 26988 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.030681 26978 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:11.016326 26998 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:11.088274 23432 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.244s (SENT)
I20260709 13:13:11.106129 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:11.093353 23465 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:11.092674 26995 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.073125 26925 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:11.094223 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.094792 26789 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.095798 26923 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.100464 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:11.096953 23460 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:11.120163 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:11.121265 22909 raft_consensus.cc:606] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:11.121470 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.121851 22909 raft_consensus.cc:614] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:11.122458 22760 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.116219 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.124393 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:11.124537 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.117643 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.130163 22834 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:11.015116 26966 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.074518 26797 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.136416 26756 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.137327 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.046880 27029 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:13:11.135747 26978 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:11.140044 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.141049 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.153848 22832 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:11.110505 26998 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.138178 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.113404 26907 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:11.056212 26930 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.157318 22836 raft_consensus.cc:2376] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9. Re-sending same reply.
I20260709 13:13:11.158944 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1591) took 1462 ms. Trace:
I20260709 13:13:11.131585 26984 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:11.147096 27007 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.935s	user 0.016s	sys 0.000s
W20260709 13:13:11.163100 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.167178 26748 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.181092 22760 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:11.180642 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.184252 22834 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:11.186269 26930 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.186483 26748 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.151984 26756 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.187757 27018 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:11.188198 27018 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.188536 27018 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.155965 26978 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.195636 26978 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.199470 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.200434 22763 raft_consensus.cc:2368] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
I20260709 13:13:11.203325 23431 leader_election.cc:400] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
I20260709 13:13:11.203714 23431 leader_election.cc:403] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:13:11.206071 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:11.206784 23460 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.176015 27007 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.187364 26748 raft_consensus.cc:2764] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:11.210542 27018 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.189473 26756 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.213305 27007 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.216737 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.218081 27018 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:11.226909 27047 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:13:11.227325 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.228740 22834 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:11.230587 26966 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.238636 26756 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:11.239125 27005 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.239476 26756 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.240082 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.240749 22763 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:11.243801 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.244751 22757 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:11.245751 22716 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48951.6us
I20260709 13:13:11.246515 26930 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.247079 26930 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.249208 26756 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.158766 26907 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.182337 26984 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:11.114259 27015 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.196240 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.186520 26951 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.249471 26797 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.250181 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.279615 22834 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:11.282006 27044 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.431s	user 0.000s	sys 0.004s
I20260709 13:13:11.250499 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:11.284085 23464 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:11.286643 22835 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:11.214826 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2450) took 1765 ms. Trace:
I20260709 13:13:11.287968 26999 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.159579 22839 rpcz_store.cc:276] 0709 13:13:09.690356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.742121 (+ 51765us) service_pool.cc:224] Handling call
0709 13:13:09.798585 (+ 56464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.859257 (+ 60672us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:09.983974 (+124717us) log.cc:844] Serialized 14 byte log entry
0709 13:13:10.335323 (+351349us) log.cc:844] Serialized 14 byte log entry
0709 13:13:10.335829 (+   506us) log.cc:844] Serialized 14 byte log entry
0709 13:13:10.335993 (+   164us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:10.341915 (+  5922us) log.cc:844] Serialized 26 byte log entry
0709 13:13:10.342138 (+   223us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:10.342199 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.342305 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:10.878488 (+536183us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b18003dffc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04557 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:11.152275 (+273787us) raft_consensus.cc:1685] finished
0709 13:13:11.152304 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.152623 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134551,"prepare.queue_time_us":231051,"prepare.run_cpu_time_us":242,"prepare.run_wall_time_us":5330,"raft.queue_time_us":150455,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":12920,"spinlock_wait_cycles":26310784,"thread_start_us":322480,"threads_started":2,"wal-append.queue_time_us":352029}
I20260709 13:13:11.253423 26907 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.289266 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.289911 22839 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 8.
I20260709 13:13:11.290776 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.290858 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.291465 22841 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.291579 22839 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:11.281345 22906 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:11.293962 22906 raft_consensus.cc:606] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:11.294355 22906 raft_consensus.cc:614] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:11.255861 26789 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.287802 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:11.287503 22758 rpcz_store.cc:276] 0709 13:13:09.449789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.469275 (+ 19486us) service_pool.cc:224] Handling call
0709 13:13:09.469782 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.349818 (+880036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.214482 (+864664us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54002224d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:11.214734 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860647808}
I20260709 13:13:11.269235 22762 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:11.258239 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55120.8us
I20260709 13:13:11.311923 22841 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:11.301061 22730 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.276000 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:11.345930 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 44980.2us
I20260709 13:13:11.347565 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.348399 22766 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.368129 22834 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:11.383836 23460 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:11.383570 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.264243 27015 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.391553 22762 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:11.276788 27009 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:11.316298 26999 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:11.401297 26945 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:11.351153 26845 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:11.348140 26830 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.404479 26992 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.437393 22766 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:11.378542 26756 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.407163 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.438436 26992 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.408054 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:11.410699 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 499) took 1035 ms. Trace:
I20260709 13:13:11.413371 26944 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:11.377103 26683 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.352018 26966 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.259492 26984 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.297415 26833 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.440327 26945 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.448168 26977 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.455834 26977 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.418766 27009 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.460304 27009 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.462311 22764 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:11.461969 26966 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.463317 26683 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.446360 26756 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.420145 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.466099 22762 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.250656 26930 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.300585 27000 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 13:13:11.378765 27029 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.380784 27025 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.386524 26748 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
I20260709 13:13:11.420223 26978 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.422257 22759 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:11.252476 26998 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.349459 27005 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:11.440054 22837 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:11.443923 22914 rpcz_store.cc:276] 0709 13:13:10.374894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.375049 (+   155us) service_pool.cc:224] Handling call
0709 13:13:10.389716 (+ 14667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.605098 (+215382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:10.605134 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:10.605184 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:10.605223 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.410337 (+805114us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000cfb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:11.410411 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.410613 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494768768}
I20260709 13:13:11.500204 27005 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:11.434788 26925 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.501284 27005 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.503186 26925 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.439599 26845 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.504252 26845 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:11.446851 26944 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:11.508615 26987 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.508232 26944 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.509451 26944 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.509195 26987 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.510008 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:11.512924 27017 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:13:11.465207 26966 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.457181 26945 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.489732 27029 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.496866 27025 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.443567 26830 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.499491 26998 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.497699 26977 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.434870 26999 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.625123 22762 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:11.783393 23431 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.784822 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:11.785853 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:11.786293 27025 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.786545 22909 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.785099 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:11.785638 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:11.788029 22906 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:11.788349 26756 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.789723 26925 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:11.790498 26925 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:11.791273 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:11.793175 26830 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.793403 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:11.795053 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:11.533557 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2447) took 2108 ms. Trace:
I20260709 13:13:11.800938 22765 rpcz_store.cc:276] 0709 13:13:09.417443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.417587 (+   144us) service_pool.cc:224] Handling call
0709 13:13:09.432203 (+ 14616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.897913 (+465710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.525989 (+1628076us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400221fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:11.526287 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060964608}
I20260709 13:13:11.799443 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:11.555815 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.801030 22913 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:11.804931 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:11.786741 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7125133499999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
I20260709 13:13:11.563143 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:11.806190 22732 connection.cc:256] RPC call timeout handler was delayed by -1.7024022349999086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24500000000000002s for a response
I20260709 13:13:11.449102 26963 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:11.807363 26963 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:11.808591 26963 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.811753 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 500) took 1436 ms. Trace:
I20260709 13:13:11.812254 22912 rpcz_store.cc:276] 0709 13:13:10.375213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.375340 (+   127us) service_pool.cc:224] Handling call
0709 13:13:10.402216 (+ 26876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.955198 (+552982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.811449 (+856251us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b540020b7d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:11.811682 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259190656}
I20260709 13:13:11.813115 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.800776 22908 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:11.814088 22763 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:11.799059 22806 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.514s (SENT) [suppressed 57 similar messages]
I20260709 13:13:11.817744 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:11.806663 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6621133499999132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 13:13:11.845423 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.818542 22909 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
W20260709 13:13:11.821403 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 713395us
I20260709 13:13:11.849184 22762 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:11.851136 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:11.849913 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 286848us
I20260709 13:13:11.851135 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:11.853675 22914 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:11.855007 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.819063 22758 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:11.843016 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.851390 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.859711 22834 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:11.860690 22762 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:11.861958 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:11.862639 22915 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:11.862913 22906 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:11.863185 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.863335 22906 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:11.864018 22762 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:11.610383 26995 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.868350 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:11.816605 26907 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.005s
I20260709 13:13:11.867074 22911 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:11.627359 27000 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.791834 26925 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.911486 26907 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.867683 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.915411 22758 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:11.917471 22758 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:11.869645 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:11.865926 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:11.918812 22765 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:11.860532 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:11.919162 22759 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:13:11.920113 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 59685.6us
I20260709 13:13:11.922323 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:11.497723 26978 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.687842 26992 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.562505 27017 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.546108 27054 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:11.880731 22834 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:11.882505 26987 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.941812 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:11.884248 27029 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:11.885066 26919 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:11.863945 26683 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:11.886475 26845 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.890369 27036 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:11.886490 26999 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.892772 27000 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.876231 27025 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:13:11.563395 26797 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.310s
I20260709 13:13:11.963140 26797 consensus_queue.cc:1243] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:11.898375 27038 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:11.875607 26995 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.900386 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:11.969619 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69841.2us
I20260709 13:13:11.970785 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.971618 27000 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:11.901185 22763 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:11.867511 26998 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:11.972951 27005 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:11.973239 26998 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.973656 27005 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.973810 26998 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:11.974157 27005 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.862160 26966 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:11.870486 22906 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.976503 26966 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.978825 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:11.979738 26999 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:11.981477 26711 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:11.981925 26711 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:11.983492 26830 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:11.599751 27021 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:11.867503 26947 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:11.988109 27021 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.938875 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:11.988638 27021 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:11.971705 26978 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:11.990542 26978 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:11.940223 27054 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.991654 26978 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.991662 27054 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.994827 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 597) took 2669 ms. Trace:
I20260709 13:13:11.994560 26919 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.941339 27017 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:11.993606 26683 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.988481 26947 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:11.997228 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.998056 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:11.998301 26947 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:11.999457 26845 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:11.999989 26756 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:11.941715 26992 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:12.019419 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.020706 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:12.021204 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:12.023986 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 224953us
I20260709 13:13:12.025061 22762 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:13:12.020817 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.995222 22833 rpcz_store.cc:276] 0709 13:13:09.326026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.326185 (+   159us) service_pool.cc:224] Handling call
0709 13:13:09.353727 (+ 27542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.379075 (+1025348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.994478 (+1615403us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000398d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:11.994708 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344446336}
I20260709 13:13:12.028126 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:12.029036 22762 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:11.904757 22915 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:12.032500 22906 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:12.035737 26907 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.038823 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:11.958187 27015 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:12.046217 22840 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:12.050433 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:12.037180 26987 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:13:12.032814 26756 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:12.031662 27029 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 13:13:12.058398 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 192933us
I20260709 13:13:12.039669 26944 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:13:12.048287 26977 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:12.334461 26977 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.243335 27021 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.335155 26977 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:12.335604 26919 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.076854 22906 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:13:12.114094 26966 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.336608 26830 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:12.126394 22759 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
W20260709 13:13:12.341675 22732 connection.cc:256] RPC call timeout handler was delayed by -1.522526710999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 13:13:12.357311 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4876646010000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
W20260709 13:13:12.357751 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4400589989999162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:13:12.358114 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4312783560000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2703s for a response
I20260709 13:13:12.118541 26683 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:12.060496 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:12.075470 22833 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:12.078933 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:12.109395 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 238175us
I20260709 13:13:12.364423 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:12.342958 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 420812us
I20260709 13:13:12.365240 22840 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:12.343952 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:12.147651 22907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 502) took 1771 ms (client timeout 2244 ms). Trace:
W20260709 13:13:12.227159 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.336623 22906 pending_rounds.cc:85] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 10
I20260709 13:13:12.368268 22906 pending_rounds.cc:107] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:13:12.368536 22906 raft_consensus.cc:2889] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:12.350164 22839 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:12.043967 27003 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:13:12.358569 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4266208079999387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
I20260709 13:13:12.369783 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:12.370132 22732 connection.cc:256] RPC call timeout handler was delayed by -1.4006783559999576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2737s for a response
W20260709 13:13:12.370522 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.370630 22759 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:12.057137 26907 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:13:12.366484 22907 rpcz_store.cc:269] 0709 13:13:10.375816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.376850 (+  1034us) service_pool.cc:224] Handling call
0709 13:13:12.147299 (+1770449us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:12.147510 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505897728}
I20260709 13:13:12.124239 26845 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:12.373340 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3070861950000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:13:12.292582 27017 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.329239 27054 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:12.375849 26975 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:13:12.362083 27009 raft_consensus.cc:2764] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:13:12.362638 26829 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:12.381810 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2980861949999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 13:13:12.387012 27054 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.388564 27075 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:13:12.395116 27075 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:12.394986 26998 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.365466 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:12.397636 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1601) took 1258 ms. Trace:
I20260709 13:13:12.398052 22837 rpcz_store.cc:276] 0709 13:13:11.139560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.139706 (+   146us) service_pool.cc:224] Handling call
0709 13:13:12.397411 (+1257705us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001bc454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:12.397548 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440621312}
I20260709 13:13:12.349745 27071 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:12.399737 27071 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:12.401090 27069 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:12.401610 27022 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:12.401707 27069 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:12.402132 27022 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:12.402823 27069 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.403059 27022 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.393322 26994 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:13:12.405707 27000 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:12.406018 26966 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.406317 27000 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.406520 26966 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:12.408543 27000 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.408463 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:12.409286 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:12.409798 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:12.410038 23462 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:12.409339 23459 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:12.414148 27071 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:12.416646 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.417330 26966 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.417474 26711 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:12.419868 26711 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:12.420120 26966 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:12.383458 27021 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:12.391872 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1636531549999063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2929s for a response
I20260709 13:13:12.422160 26919 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:13:12.422588 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0772637939999186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 13:13:12.422821 26919 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:12.423110 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1784) took 1155 ms. Trace:
I20260709 13:13:12.423436 26830 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.350656 26756 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:12.429468 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64900.5us
I20260709 13:13:12.430323 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:12.431650 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:12.371232 22763 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:12.258934 26930 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.118168 26988 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.372727 27003 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:12.437757 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:12.374668 22758 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: [9.7-10.8]
I20260709 13:13:12.390431 26829 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.518846 26829 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:12.424929 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.056571 27015 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:12.520931 27015 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:12.423863 26919 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:12.524690 26919 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:12.529326 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:12.423866 26907 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:12.428958 22761 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:12.433670 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:12.911629 22763 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:12.912712 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:12.423540 22764 rpcz_store.cc:276] 0709 13:13:11.267418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.267642 (+   224us) service_pool.cc:224] Handling call
0709 13:13:11.268160 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:11.462951 (+194791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.422828 (+959877us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54002215d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:12.423046 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445415680}
W20260709 13:13:14.391743 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2479) took 1610 ms (client timeout 1546 ms). Trace:
W20260709 13:13:12.785310 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.792129 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:12.535435 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:14.521087 22758 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:12.829010 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:12.815232 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:14.577422 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2464) took 2151 ms (client timeout 1514 ms). Trace:
W20260709 13:13:14.577943 22760 rpcz_store.cc:269] 0709 13:13:12.425549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.426128 (+   579us) service_pool.cc:224] Handling call
0709 13:13:14.577301 (+2151173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.859282 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:12.427716 22762 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:13:12.913417 22763 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
W20260709 13:13:14.628404 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2495) took 1837 ms (client timeout 1514 ms). Trace:
W20260709 13:13:14.628811 22763 rpcz_store.cc:269] 0709 13:13:12.790760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.912405 (+121645us) service_pool.cc:224] Handling call
0709 13:13:14.628297 (+1715892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.983645 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1789) took 1185 ms. Trace:
I20260709 13:13:14.631494 22766 rpcz_store.cc:276] 0709 13:13:11.798051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.799087 (+  1036us) service_pool.cc:224] Handling call
0709 13:13:12.983350 (+1184263us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400393854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:12.983535 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405270784}
I20260709 13:13:14.632735 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:13.286501 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1798) took 1266 ms. Trace:
I20260709 13:13:14.635071 22765 rpcz_store.cc:276] 0709 13:13:12.019758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.032010 (+ 12252us) service_pool.cc:224] Handling call
0709 13:13:12.032424 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:12.033778 (+  1354us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:12.052721 (+ 18943us) log.cc:844] Serialized 14 byte log entry
0709 13:13:12.890068 (+837347us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b1800370600. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:13.051737 (+161669us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b1800370600. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:13:13.123679 (+ 71942us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:13.124711 (+  1032us) log.cc:844] Serialized 52 byte log entry
0709 13:13:13.124905 (+   194us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:13.284320 (+159415us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.285895 (+  1575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.286104 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.286168 (+    64us) raft_consensus.cc:1685] finished
0709 13:13:13.286186 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.286411 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":902,"prepare.queue_time_us":48776,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":13932,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":24191,"spinlock_wait_cycles":76585472,"thread_start_us":230396,"threads_started":2,"wal-append.queue_time_us":945336,"wal-append.run_cpu_time_us":3981,"wal-append.run_wall_time_us":1229786}
W20260709 13:13:13.009121 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:12.854142 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.675809 27091 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.006102 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:12.617926 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:13.001008 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.653330 22836 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:14.653855 22836 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1754) took 1657 ms (client timeout 1505 ms). Trace:
W20260709 13:13:14.654232 22836 rpcz_store.cc:269] 0709 13:13:12.996258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.000665 (+  4407us) service_pool.cc:224] Handling call
0709 13:13:14.653768 (+1653103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.655470 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:14.655694 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1759) took 1650 ms (client timeout 1983 ms). Trace:
W20260709 13:13:14.656230 22836 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1762) took 1648 ms (client timeout 1978 ms). Trace:
I20260709 13:13:12.997004 22832 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:14.661664 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1753) took 1666 ms. Trace:
I20260709 13:13:14.662060 22832 rpcz_store.cc:276] 0709 13:13:12.995824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.996073 (+   249us) service_pool.cc:224] Handling call
0709 13:13:12.996547 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.661333 (+1664786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.661601 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.663372 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:14.664170 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1764) took 1656 ms (client timeout 2105 ms). Trace:
W20260709 13:13:14.664551 22832 rpcz_store.cc:269] 0709 13:13:13.008120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.663061 (+1654941us) service_pool.cc:224] Handling call
0709 13:13:14.664111 (+  1050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.046793 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.666615 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1760) took 1659 ms. Trace:
I20260709 13:13:14.666989 22834 rpcz_store.cc:276] 0709 13:13:13.006862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.046408 (+ 39546us) service_pool.cc:224] Handling call
0709 13:13:14.666546 (+1620138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.668001 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:13.003773 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:14.670677 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1757) took 1667 ms (client timeout 1870 ms). Trace:
W20260709 13:13:14.671118 22839 rpcz_store.cc:269] 0709 13:13:13.003365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.003507 (+   142us) service_pool.cc:224] Handling call
0709 13:13:14.670591 (+1667084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.672052 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:14.672681 22839 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:12.998185 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:13.004908 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:14.677129 26938 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:14.282449 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.008s	sys 0.015s
W20260709 13:13:14.680290 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.145s	user 0.010s	sys 0.017s
W20260709 13:13:13.153321 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:13.665143 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.228s	user 0.022s	sys 0.022s
W20260709 13:13:14.690778 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.254s	user 0.022s	sys 0.023s
I20260709 13:13:14.691557 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:12.764277 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.291163 27093 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.694463 27093 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:12.850597 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:14.697145 22907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2594) took 1921 ms (client timeout 1978 ms). Trace:
W20260709 13:13:14.697594 22907 rpcz_store.cc:269] 0709 13:13:12.776537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.849911 (+ 73374us) service_pool.cc:224] Handling call
0709 13:13:14.697072 (+1847161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.698876 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:14.699623 22907 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:12.774125 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:14.709468 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2590) took 1936 ms (client timeout 2156 ms). Trace:
W20260709 13:13:14.709908 22909 rpcz_store.cc:269] 0709 13:13:12.773207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.773693 (+   486us) service_pool.cc:224] Handling call
0709 13:13:14.709395 (+1935702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.710938 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:14.712291 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:12.880877 22914 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:12.810859 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:14.714560 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2584) took 2273 ms. Trace:
W20260709 13:13:14.715204 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2589) took 1942 ms (client timeout 2122 ms). Trace:
W20260709 13:13:14.715678 22913 rpcz_store.cc:269] 0709 13:13:12.772916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.810355 (+ 37439us) service_pool.cc:224] Handling call
0709 13:13:14.715137 (+1904782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.716890 22913 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:13.763018 22908 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:12.905290 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:14.718961 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2587) took 1946 ms. Trace:
I20260709 13:13:14.719515 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2588) took 1946 ms. Trace:
I20260709 13:13:14.714955 22914 rpcz_store.cc:276] 0709 13:13:12.441134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.441284 (+   150us) service_pool.cc:224] Handling call
0709 13:13:12.863202 (+421918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.714244 (+1851042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.714473 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.719913 22912 rpcz_store.cc:276] 0709 13:13:12.772616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.904853 (+132237us) service_pool.cc:224] Handling call
0709 13:13:14.719445 (+1814592us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:12.827390 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:14.108533 23432 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.662s (SENT)
I20260709 13:13:14.723567 27105 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:13.962579 22936 debug-util.cc:398] Leaking SignalData structure 0x7b080018bb60 after lost signal to thread 21639
I20260709 13:13:12.461287 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:14.727455 22936 debug-util.cc:398] Leaking SignalData structure 0x7b0800042da0 after lost signal to thread 22698
W20260709 13:13:14.727599 23458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 768) took 2266 ms (client timeout 1546 ms). Trace:
W20260709 13:13:14.727957 23458 rpcz_store.cc:269] 0709 13:13:12.460799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.460941 (+   142us) service_pool.cc:224] Handling call
0709 13:13:14.727485 (+2266544us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:14.729034 22936 debug-util.cc:398] Leaking SignalData structure 0x7b080036df20 after lost signal to thread 22790
I20260709 13:13:12.465297 23464 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:14.719310 22908 rpcz_store.cc:276] 0709 13:13:12.772339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.762252 (+989913us) service_pool.cc:224] Handling call
0709 13:13:13.762775 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.718706 (+955931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.718907 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.730366 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 769) took 2268 ms. Trace:
I20260709 13:13:12.535631 23462 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
W20260709 13:13:14.732187 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 654) took 2325 ms (client timeout 1570 ms). Trace:
W20260709 13:13:14.732587 23462 rpcz_store.cc:269] 0709 13:13:12.406738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.408908 (+  2170us) service_pool.cc:224] Handling call
0709 13:13:14.732084 (+2323176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.733697 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:14.734987 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:14.735627 23462 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 6.
I20260709 13:13:12.464816 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:14.738065 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 765) took 2281 ms (client timeout 2087 ms). Trace:
W20260709 13:13:14.738550 23463 rpcz_store.cc:269] 0709 13:13:12.456538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.464429 (+  7891us) service_pool.cc:224] Handling call
0709 13:13:14.737978 (+2273549us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100992}
I20260709 13:13:14.739889 23463 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:13:12.530942 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:14.742285 23466 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 478) took 2219 ms (client timeout 1881 ms). Trace:
W20260709 13:13:14.742704 23466 rpcz_store.cc:269] 0709 13:13:12.522596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.530640 (+  8044us) service_pool.cc:224] Handling call
0709 13:13:14.742190 (+2211550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.436612 26748 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:12.474010 26756 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.424295 26830 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.752561 26830 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.429227 26845 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.755138 26845 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:12.531802 26907 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.763240 22907 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:12.688223 26919 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:13.715060 26963 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:14.765918 26919 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.766968 26963 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.768980 26919 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.769096 26963 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.444074 26930 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.770285 22839 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:14.765113 27109 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.763298 26975 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:14.777670 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 87442.6us
I20260709 13:13:14.778256 26975 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.780545 26975 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:12.751695 27022 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:12.529460 26977 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.279073 27025 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:14.790436 27025 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.793833 22915 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:14.796510 22915 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2582) took 2359 ms (client timeout 2371 ms). Trace:
I20260709 13:13:12.489990 26979 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:12.453040 27054 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:14.800082 27054 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:14.801313 22730 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:14.801107 27054 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.802284 26963 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:14.802443 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:14.802868 26963 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:14.803792 26963 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.793265 27025 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:14.796911 22915 rpcz_store.cc:269] 0709 13:13:12.437266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.437430 (+   164us) service_pool.cc:224] Handling call
0709 13:13:14.796223 (+2358793us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b540010fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:14.796395 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5416431360}
I20260709 13:13:12.397298 26998 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:12.533556 27005 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.002s
I20260709 13:13:14.809388 27005 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.810637 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:14.811295 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:14.811774 27005 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:13.400745 27009 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:12.474112 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:14.729384 22936 debug-util.cc:398] Leaking SignalData structure 0x7b0800363620 after lost signal to thread 22791
I20260709 13:13:14.813108 27009 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.814790 23459 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:14.815289 23459 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 767) took 2356 ms (client timeout 2042 ms). Trace:
W20260709 13:13:14.815559 22936 debug-util.cc:398] Leaking SignalData structure 0x7b0800342680 after lost signal to thread 22865
W20260709 13:13:14.815687 23459 rpcz_store.cc:269] 0709 13:13:12.459047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.460199 (+  1152us) service_pool.cc:224] Handling call
0709 13:13:14.815203 (+2355004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.587407 27015 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:14.817648 23461 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:14.817445 27015 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.819520 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 773) took 2337 ms. Trace:
I20260709 13:13:14.820118 27015 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.491153 27028 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:14.819913 23461 rpcz_store.cc:276] 0709 13:13:12.482152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.482325 (+   173us) service_pool.cc:224] Handling call
0709 13:13:12.531491 (+ 49166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.818137 (+2286646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.819268 (+  1131us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54002c9554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:14.819458 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5257224960}
I20260709 13:13:14.821323 27028 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:14.822383 27028 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.823665 27028 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:12.432394 27071 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.826453 27071 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.828902 27071 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:12.764814 27042 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:14.830308 27042 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.832613 27042 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:12.683743 27059 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.245s	user 0.004s	sys 0.001s
I20260709 13:13:12.305956 27061 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:13:14.838471 27061 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:14.838923 27061 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:14.839861 27061 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.453159 27075 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.842000 27075 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.844372 27075 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.438915 27068 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.845796 27068 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.847102 23457 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:14.848184 27068 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:13.008412 27069 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.852640 27069 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.855015 27069 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:14.333171 26966 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:14.855744 26966 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.858140 26966 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.342272 27094 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.858927 27094 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.861866 27094 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:12.827330 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:14.867063 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:14.398893 22764 rpcz_store.cc:269] 0709 13:13:12.781193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.391461 (+1610268us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:14.391643 (+   182us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:14.422216 27095 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.443853 27036 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:14.868254 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2478) took 2087 ms (client timeout 2042 ms). Trace:
I20260709 13:13:14.868099 27095 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.868175 27036 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:14.868767 22764 rpcz_store.cc:269] 0709 13:13:12.780917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.868055 (+2087138us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:14.868189 (+   134us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:13:14.869673 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2476) took 2089 ms (client timeout 2087 ms). Trace:
W20260709 13:13:14.869977 22764 rpcz_store.cc:269] 0709 13:13:12.780240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.869482 (+2089242us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:14.869617 (+   135us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:14.870427 27095 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:14.870594 27036 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:14.452917 22661 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 134) took 27507 ms (client timeout 15000 ms). Trace:
I20260709 13:13:14.870842 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:14.871687 22764 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:14.457544 27096 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.872732 27096 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.874230 22908 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:14.875514 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2596) took 2098 ms. Trace:
I20260709 13:13:14.875671 27096 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:14.473282 22806 connection.cc:256] RPC call timeout handler was delayed by -1.894159228000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:13:14.875939 22908 rpcz_store.cc:276] 0709 13:13:12.777216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.730236 (+1953020us) service_pool.cc:224] Handling call
0709 13:13:14.730621 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.874641 (+144020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.875277 (+   636us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002107d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:14.875451 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329325952}
W20260709 13:13:14.876780 22806 connection.cc:256] RPC call timeout handler was delayed by -1.8234972140000991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2878s for a response
W20260709 13:13:14.877213 22806 connection.cc:256] RPC call timeout handler was delayed by -1.60662994300003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 13:13:14.877590 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5987832100000787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
W20260709 13:13:14.877894 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5931979050000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
W20260709 13:13:14.878188 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4282009050000397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2371s for a response
I20260709 13:13:14.878276 22908 pending_rounds.cc:85] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 3
I20260709 13:13:14.878665 22908 pending_rounds.cc:107] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:13:14.518594 27100 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:14.878918 26941 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:14.879249 27100 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.879801 26941 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.881687 26872 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.882400 26941 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:14.547303 26585 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.883052 26585 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.883390 27100 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.885051 26585 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.885643 26585 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.886060 26585 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:14.871219 22661 rpcz_store.cc:269] 0709 13:12:46.945959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.946275 (+   316us) service_pool.cc:224] Handling call
0709 13:13:14.452737 (+27506462us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:06.466064 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:06.535116 (+ 69052us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:06.535200 (+    84us) write_op.cc:435] Acquired schema lock
0709 13:13:06.535251 (+    51us) tablet.cc:662] Decoding operations
0709 13:13:06.537184 (+  1933us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:06.537296 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:06.537331 (+    35us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:13:06.759076 (+221745us) tablet.cc:701] Row locks acquired
0709 13:13:06.780327 (+ 21251us) write_op.cc:260] PREPARE: finished
0709 13:13:06.814782 (+ 34455us) write_op.cc:270] Start()
0709 13:13:06.851632 (+ 36850us) write_op.cc:276] Timestamp: P: 1783602786800476 usec, L: 0
0709 13:13:06.861584 (+  9952us) op_driver.cc:348] REPLICATION: starting
0709 13:13:06.878502 (+ 16918us) log.cc:844] Serialized 5985 byte log entry
0709 13:13:07.756051 (+877549us) op_driver.cc:464] REPLICATION: finished
0709 13:13:08.055971 (+299920us) write_op.cc:301] APPLY: starting
0709 13:13:08.074053 (+ 18082us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:08.314940 (+240887us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:08.315096 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:08.835621 (+520525us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:08.836052 (+   431us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:13:08.836106 (+    54us) write_op.cc:312] APPLY: finished
0709 13:13:08.836660 (+   554us) log.cc:844] Serialized 139 byte log entry
0709 13:13:08.941793 (+105133us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:09.023265 (+ 81472us) write_op.cc:454] Released schema lock
0709 13:13:09.110836 (+ 87571us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:09.197072 (+ 86236us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:09.200846 (+  3774us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2468779,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":230556,"apply.run_cpu_time_us":43964,"apply.run_wall_time_us":1100974,"cfile_cache_hit":58,"cfile_cache_hit_bytes":60639,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":1893,"lbm_reads_lt_1ms":12,"num_ops":15,"prepare.queue_time_us":103083,"prepare.run_cpu_time_us":13583,"prepare.run_wall_time_us":424141,"replication_time_us":806554,"thread_start_us":229440,"threads_started":3,"wal-append.queue_time_us":292629,"wal-append.run_cpu_time_us":3239,"wal-append.run_wall_time_us":1135009}]]}
I20260709 13:13:14.565907 22758 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:14.571326 27103 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.575701 27038 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.888224 26585 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.889581 23462 raft_consensus.cc:1217] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 7.9->[9.10-11.11]   Dedup: 9.10->[11.11-11.11]
W20260709 13:13:14.579147 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 529) took 1573 ms (client timeout 1523 ms). Trace:
W20260709 13:13:14.888713 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2473) took 2451 ms (client timeout 1500 ms). Trace:
W20260709 13:13:14.890209 22760 rpcz_store.cc:269] 0709 13:13:13.005311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.578891 (+1573580us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:14.579066 (+   175us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:13:14.890316 22758 rpcz_store.cc:269] 0709 13:13:12.437294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.535151 (+ 97857us) service_pool.cc:224] Handling call
0709 13:13:14.888623 (+2353472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.891543 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:14.891625 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:14.892246 22760 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:14.892403 22758 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:14.890235 23462 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [9.10-11.11]
I20260709 13:13:14.579085 27104 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:14.586880 26516 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:14.895892 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 476) took 2373 ms. Trace:
I20260709 13:13:14.896343 23462 rpcz_store.cc:276] 0709 13:13:12.522021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.736413 (+2214392us) service_pool.cc:224] Handling call
0709 13:13:14.736751 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:14.894960 (+158209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.895590 (+   630us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400236254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:14.895829 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350896384}
I20260709 13:13:14.581929 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2465) took 2156 ms. Trace:
I20260709 13:13:14.897713 22759 rpcz_store.cc:276] 0709 13:13:12.425836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.426212 (+   376us) service_pool.cc:224] Handling call
0709 13:13:14.581845 (+2155633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.898840 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:14.589242 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2467) took 2162 ms. Trace:
I20260709 13:13:14.899546 22762 rpcz_store.cc:276] 0709 13:13:12.427023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.427170 (+   147us) service_pool.cc:224] Handling call
0709 13:13:12.427487 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.588901 (+2161414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.589185 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.900583 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:14.901122 22762 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:13:14.589558 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.79746e+06us
I20260709 13:13:14.878933 22908 raft_consensus.cc:2889] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:14.902243 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:14.903795 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:14.904944 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:14.590855 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 121811us
I20260709 13:13:14.607137 27102 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:14.905778 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.906038 27102 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:14.606142 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 139145us
I20260709 13:13:14.622722 26833 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:14.623488 26978 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.637883 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1749) took 2426 ms. Trace:
W20260709 13:13:14.640228 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.640779 27091 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:14.649577 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1804) took 2212 ms (client timeout 2153 ms). Trace:
W20260709 13:13:14.919333 22763 rpcz_store.cc:269] 0709 13:13:12.437251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.649349 (+2212098us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:14.649489 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:14.920641 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:14.650017 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:14.921629 22765 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:14.921797 22763 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:13:14.633368 22766 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
W20260709 13:13:14.922176 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2489) took 2133 ms (client timeout 1978 ms). Trace:
W20260709 13:13:14.922556 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 524) took 1918 ms (client timeout 1649 ms). Trace:
I20260709 13:13:14.913803 22841 rpcz_store.cc:276] 0709 13:13:12.211428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.211681 (+   253us) service_pool.cc:224] Handling call
0709 13:13:12.757343 (+545662us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:13:12.757651 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.477990 (+1720339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.478176 (+   186us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.479086 (+   910us) log.cc:844] Serialized 26 byte log entry
0709 13:13:14.598877 (+119791us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:14.598926 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.599007 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.637539 (+ 38532us) raft_consensus.cc:1685] finished
0709 13:13:14.637562 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.637819 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1719586,"raft.queue_time_us":1795546,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"spinlock_wait_cycles":2426240,"thread_start_us":119322,"threads_started":1,"wal-append.queue_time_us":149651}
I20260709 13:13:14.652666 22835 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:13.002494 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.929997 22837 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
W20260709 13:13:14.930670 22837 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1756) took 1933 ms (client timeout 1677 ms). Trace:
W20260709 13:13:14.931006 22837 rpcz_store.cc:269] 0709 13:13:12.997610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.002175 (+  4565us) service_pool.cc:224] Handling call
0709 13:13:14.930594 (+1928419us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:14.657080 22836 rpcz_store.cc:269] 0709 13:13:13.007505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.655145 (+1647640us) service_pool.cc:224] Handling call
0709 13:13:14.656171 (+  1026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.674466 26933 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.677762 26938 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:14.678506 22807 connection.cc:256] RPC call timeout handler was delayed by -1.3720634689999542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2371s for a response
W20260709 13:13:14.932888 22807 connection.cc:256] RPC call timeout handler was delayed by -1.0137481159999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 13:13:14.931603 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:14.934746 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:14.935421 23463 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:14.695667 26984 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.435964 22906 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:12.868041 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1671) took 1083 ms. Trace:
I20260709 13:13:14.944274 22911 rpcz_store.cc:276] 0709 13:13:11.784719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.784842 (+   123us) service_pool.cc:224] Handling call
0709 13:13:12.867765 (+1082923us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b5400041354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:12.867937 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183296384}
I20260709 13:13:14.911259 27094 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:14.889012 27103 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.702435 27108 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:14.674466 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1755) took 1677 ms (client timeout 2228 ms). Trace:
I20260709 13:13:14.946426 27108 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:14.946465 22838 rpcz_store.cc:269] 0709 13:13:12.997004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.997818 (+   814us) service_pool.cc:224] Handling call
0709 13:13:14.674381 (+1676563us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.947001 27102 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:14.706640 27093 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:14.922981 27104 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:14.721396 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2592) took 1942 ms. Trace:
I20260709 13:13:14.723762 27106 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:14.947850 27103 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:14.948092 22913 rpcz_store.cc:276] 0709 13:13:12.775295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.716391 (+1941096us) service_pool.cc:224] Handling call
0709 13:13:14.716689 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.717278 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.717462 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.929889 26978 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.948355 27106 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.950769 26978 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:14.951391 27106 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.724186 27105 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.940248 26933 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.952256 27091 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.950788 26984 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.953385 27105 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:14.724411 22909 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:14.953886 26933 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:14.723577 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:14.954496 27028 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:13:14.954859 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 846397us
I20260709 13:13:14.955364 26984 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:14.669415 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.956496 22834 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:14.957602 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.947870 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.958758 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:14.958768 27105 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:14.955364 26938 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.959977 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:14.960563 22834 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:14.725831 26868 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:14.959214 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.962333 26868 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:14.724718 22659 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 9, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:12.639818 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:14.728953 23458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 479) took 2206 ms (client timeout 2105 ms). Trace:
W20260709 13:13:14.963861 23460 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 477) took 2441 ms (client timeout 2077 ms). Trace:
W20260709 13:13:14.964082 23458 rpcz_store.cc:269] 0709 13:13:12.522866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.728769 (+2205903us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:14.728877 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:14.963749 22659 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:14.964334 23460 rpcz_store.cc:269] 0709 13:13:12.522326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.639330 (+117004us) service_pool.cc:224] Handling call
0709 13:13:14.963771 (+2324441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.730690 23464 rpcz_store.cc:276] 0709 13:13:12.462293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.462466 (+   173us) service_pool.cc:224] Handling call
0709 13:13:12.464823 (+  2357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.730057 (+2265234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.730288 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
I20260709 13:13:14.764489 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:14.965327 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:14.965965 22907 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:14.665419 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:14.965674 22659 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 7, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 7 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.938416 27095 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:13:14.967319 22907 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:13:14.967795 22659 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 4 to 7, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 7 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:14.968694 22907 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:12.917313 26999 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.773737 27109 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.969777 26829 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:14.970360 26999 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.971861 26829 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:14.972689 26829 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:14.775369 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.973300 26987 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:14.974318 22839 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:14.975236 26987 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:14.975411 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.975620 26987 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:14.975845 26947 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:14.976707 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.976462 26987 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:12.462072 26952 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.978394 26952 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:14.798511 26979 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:12.266750 26992 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:12.839818 26995 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:14.789287 26977 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:12.509086 27003 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.013468 27003 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:14.806056 22915 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:13:14.812487 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:15.016283 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 203885us
I20260709 13:13:14.988739 27091 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.017050 27091 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:15.020243 22730 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.721s (SENT)
W20260709 13:13:14.816195 22936 debug-util.cc:398] Leaking SignalData structure 0x7b08003da4a0 after lost signal to thread 22866
I20260709 13:13:12.485791 23465 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:15.021934 26979 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.022648 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 770) took 2560 ms. Trace:
I20260709 13:13:15.023155 27104 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:15.023046 23465 rpcz_store.cc:276] 0709 13:13:12.466522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.477293 (+ 10771us) service_pool.cc:224] Handling call
0709 13:13:12.477832 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.022360 (+2544528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.022588 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.023785 27104 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.024180 27104 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.024304 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:15.025000 22805 connection.cc:256] RPC call timeout handler was delayed by -2.4451877949999243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:13:14.283319 27066 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.026752 27066 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.028936 27066 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.849233 22757 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:14.849570 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 772) took 2379 ms. Trace:
I20260709 13:13:15.033222 26952 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.033622 23457 rpcz_store.cc:276] 0709 13:13:12.469976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.470140 (+   164us) service_pool.cc:224] Handling call
0709 13:13:12.488568 (+ 18428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.847438 (+2358870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.849279 (+  1841us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b54006707d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:14.849486 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5423440768}
I20260709 13:13:15.035014 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.035833 23457 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:15.036875 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:14.816210 27009 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.863420 27061 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:14.872205 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2485) took 2084 ms (client timeout 2122 ms). Trace:
I20260709 13:13:14.882198 26872 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:15.039458 22764 rpcz_store.cc:269] 0709 13:13:12.787702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.870563 (+2082861us) service_pool.cc:224] Handling call
0709 13:13:14.872155 (+  1592us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.040633 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:14.523784 27029 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.020282 27091 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.041915 27029 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.044981 27029 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:14.580196 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2497) took 1788 ms (client timeout 1500 ms). Trace:
W20260709 13:13:14.878923 22806 connection.cc:256] RPC call timeout handler was delayed by -1.280064996999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:13:15.047592 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2494649970000182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260709 13:13:15.047488 22761 rpcz_store.cc:269] 0709 13:13:12.791329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.858977 (+ 67648us) service_pool.cc:224] Handling call
0709 13:13:14.580126 (+1721149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:15.048045 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2345991199999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 13:13:15.048923 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.050141 22761 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:13:15.050552 22806 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.500s (SENT)
W20260709 13:13:15.050652 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2494) took 2260 ms (client timeout 2653 ms). Trace:
I20260709 13:13:14.615697 27101 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:15.051205 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.514s (SENT)
I20260709 13:13:15.051564 27101 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:14.910138 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.054448 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.055596 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.056188 23462 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:15.056970 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.057744 22759 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.068661 27101 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:15.071504 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.078888 23462 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:13:15.051194 22761 rpcz_store.cc:269] 0709 13:13:12.790492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.048578 (+2258086us) service_pool.cc:224] Handling call
0709 13:13:15.050578 (+  2000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.080576 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.081209 22761 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:14.910305 22758 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:13:14.918077 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2474) took 2139 ms (client timeout 2371 ms). Trace:
W20260709 13:13:14.923065 22766 rpcz_store.cc:269] 0709 13:13:13.004000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.632437 (+1628437us) service_pool.cc:224] Handling call
0709 13:13:14.922471 (+290034us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:15.085068 22762 rpcz_store.cc:269] 0709 13:13:12.779169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.900355 (+2121186us) service_pool.cc:224] Handling call
0709 13:13:14.917985 (+ 17630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.924156 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.086603 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2496) took 2295 ms (client timeout 2217 ms). Trace:
I20260709 13:13:14.926163 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:14.928390 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2491) took 2138 ms (client timeout 2363 ms). Trace:
W20260709 13:13:15.088083 22763 rpcz_store.cc:269] 0709 13:13:12.789616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.920271 (+2130655us) service_pool.cc:224] Handling call
0709 13:13:14.928325 (+  8054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.925101 22732 leader_election.cc:400] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 8. Current term is 9.
I20260709 13:13:14.937146 22835 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:14.937014 22760 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:13:15.089113 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.089448 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2486) took 2301 ms (client timeout 2156 ms). Trace:
W20260709 13:13:15.089867 22760 rpcz_store.cc:269] 0709 13:13:12.787975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.891201 (+2103226us) service_pool.cc:224] Handling call
0709 13:13:15.089378 (+198177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.937918 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:14.939411 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:14.938019 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:14.942938 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2581) took 2508 ms. Trace:
I20260709 13:13:15.091117 22906 rpcz_store.cc:276] 0709 13:13:12.434828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.434976 (+   148us) service_pool.cc:224] Handling call
0709 13:13:12.435360 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.942631 (+2507271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.942872 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:14.947144 27108 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:14.677341 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1758) took 1672 ms. Trace:
I20260709 13:13:14.932797 27096 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:13:14.771662 26930 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:14.977866 22914 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:13:14.977626 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:14.962597 22911 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:14.979563 22913 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:13:14.963734 23432 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.762s (SENT)
I20260709 13:13:14.962720 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.094681 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 120898us
I20260709 13:13:15.094946 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:14.977419 22839 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:14.984094 23463 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:14.984735 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:15.095845 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132146us
I20260709 13:13:14.996390 22834 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:14.998564 26993 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.973007 27109 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.000821 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.117650 23460 raft_consensus.cc:2376] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11. Re-sending same reply.
I20260709 13:13:15.117918 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.003360 26987 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:15.007522 26923 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.962754 26868 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:15.119234 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.022718 22936 debug-util.cc:398] Leaking SignalData structure 0x7b080026a000 after lost signal to thread 22939
I20260709 13:13:14.989028 22659 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 8, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 8 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:15.124930 22936 debug-util.cc:398] Leaking SignalData structure 0x7b080035bb20 after lost signal to thread 22940
I20260709 13:13:15.025632 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.126262 23465 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:13:14.803805 26830 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:15.030315 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2463) took 2605 ms. Trace:
I20260709 13:13:15.133785 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:15.134086 22757 rpcz_store.cc:276] 0709 13:13:12.425209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.425615 (+   406us) service_pool.cc:224] Handling call
0709 13:13:12.425955 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.029523 (+2603568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.030080 (+   557us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b5400310e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:15.030258 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5572320128}
I20260709 13:13:15.135476 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.135068 22659 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:15.136842 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.023279 26979 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:14.973054 26999 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:15.038034 22805 connection.cc:256] RPC call timeout handler was delayed by -2.0955108309999559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 13:13:15.138036 22805 connection.cc:256] RPC call timeout handler was delayed by -2.0550108310000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
I20260709 13:13:15.103392 26829 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:15.139125 26829 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:15.139627 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.015625 22915 pending_rounds.cc:85] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 8
I20260709 13:13:15.140249 23458 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:15.140595 22915 pending_rounds.cc:107] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:13:15.140890 22915 raft_consensus.cc:2889] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:15.045764 22832 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10. Re-sending same reply.
W20260709 13:13:15.140935 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 230296us
I20260709 13:13:15.046824 26872 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:14.656137 22833 rpcz_store.cc:269] 0709 13:13:13.005674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.005810 (+   136us) service_pool.cc:224] Handling call
0709 13:13:14.655628 (+1649818us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:14.922554 22765 rpcz_store.cc:269] 0709 13:13:12.788808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.637240 (+1848432us) service_pool.cc:224] Handling call
0709 13:13:14.922104 (+284864us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.144856 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.145076 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.091235 23461 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:14.706763 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:15.087852 22758 rpcz_store.cc:269] 0709 13:13:12.791051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.891289 (+2100238us) service_pool.cc:224] Handling call
0709 13:13:15.086525 (+195236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40192}
I20260709 13:13:15.146979 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.147624 22758 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:15.147953 22761 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:15.148497 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.098208 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.098575 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.150374 22763 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:15.150951 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.151568 22839 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:15.151738 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.152392 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.098930 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.153215 22762 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:15.099393 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.154915 22841 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:15.099746 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.155416 22760 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:15.103204 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.156859 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.102768 22759 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
W20260709 13:13:15.116050 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1750) took 2885 ms (client timeout 2768 ms). Trace:
I20260709 13:13:15.103668 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2593) took 2327 ms. Trace:
I20260709 13:13:15.159004 22914 rpcz_store.cc:276] 0709 13:13:12.775869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.720478 (+1944609us) service_pool.cc:224] Handling call
0709 13:13:14.720854 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.094299 (+373445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.103367 (+  9068us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54001b9ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:15.103573 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590284544}
I20260709 13:13:15.002260 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.122428 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.160730 22834 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
W20260709 13:13:15.159022 23432 leader_election.cc:336] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.026s (SENT)
I20260709 13:13:15.161952 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.157428 22836 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:15.123288 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:15.164789 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:15.128147 22936 debug-util.cc:398] Leaking SignalData structure 0x7b0800033a40 after lost signal to thread 23490
W20260709 13:13:15.165649 22936 debug-util.cc:398] Leaking SignalData structure 0x7b080037a0c0 after lost signal to thread 23491
I20260709 13:13:15.128824 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.169943 22766 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:15.170392 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2482) took 2384 ms. Trace:
I20260709 13:13:15.170691 22766 rpcz_store.cc:276] 0709 13:13:12.786692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.085657 (+2298965us) service_pool.cc:224] Handling call
0709 13:13:15.170327 (+ 84670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.130419 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.131978 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.131814 26923 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.132472 26945 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:15.175441 26948 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:15.175861 27109 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:15.121560 27031 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:13:15.133733 26830 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:15.177021 26830 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:15.134517 23463 raft_consensus.cc:2376] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10. Re-sending same reply.
I20260709 13:13:14.889277 27038 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:15.189229 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 523) took 2185 ms (client timeout 2128 ms). Trace:
W20260709 13:13:15.189851 22759 rpcz_store.cc:269] 0709 13:13:13.003710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.898567 (+1894857us) service_pool.cc:224] Handling call
0709 13:13:15.189042 (+290475us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540035e154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:15.189158 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364089216}
I20260709 13:13:15.190228 22762 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:15.141156 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.133839 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.191013 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2469) took 2763 ms. Trace:
I20260709 13:13:15.141906 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:15.092268 22840 rpcz_store.cc:276] 0709 13:13:13.004527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.004659 (+   132us) service_pool.cc:224] Handling call
0709 13:13:14.677271 (+1672612us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:15.149554 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 442841us
I20260709 13:13:15.172752 22838 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:15.173314 22837 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:15.192987 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.173835 22758 raft_consensus.cc:2376] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11. Re-sending same reply.
I20260709 13:13:15.176609 22912 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:15.133157 26868 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.093632 27061 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:15.158373 22835 rpcz_store.cc:269] 0709 13:13:12.211945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.212080 (+   135us) service_pool.cc:224] Handling call
0709 13:13:12.617415 (+405335us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:13:15.097717 (+2480302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2436864}
I20260709 13:13:15.093873 27100 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.183166 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.184944 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2591) took 2411 ms. Trace:
I20260709 13:13:15.208214 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.187197 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.189450 22763 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 13:13:15.208992 22915 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
W20260709 13:13:15.148552 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 526) took 2144 ms (client timeout 2461 ms). Trace:
W20260709 13:13:15.209676 22761 rpcz_store.cc:269] 0709 13:13:13.004549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.080277 (+2075728us) service_pool.cc:224] Handling call
0709 13:13:15.148492 (+ 68215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.211172 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2468) took 2783 ms. Trace:
I20260709 13:13:15.211179 22760 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:13:15.103320 27066 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:15.129967 26993 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:15.195441 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53631.2us
I20260709 13:13:15.197247 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2475) took 2417 ms. Trace:
I20260709 13:13:15.197799 22758 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:15.199753 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:15.200773 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2477) took 2420 ms. Trace:
I20260709 13:13:15.201458 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.206082 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.211788 22763 rpcz_store.cc:276] 0709 13:13:12.427343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.188797 (+2761454us) service_pool.cc:224] Handling call
0709 13:13:15.189180 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:15.209291 (+ 20111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.211110 (+  1819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.191357 22762 rpcz_store.cc:276] 0709 13:13:12.427607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.189783 (+2762176us) service_pool.cc:224] Handling call
0709 13:13:15.190063 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.190756 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.190958 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.212567 22760 pending_rounds.cc:85] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:13:15.208351 22909 rpcz_store.cc:276] 0709 13:13:12.773470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.713242 (+1939772us) service_pool.cc:224] Handling call
0709 13:13:14.724222 (+ 10980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.954084 (+229862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.120743 (+166659us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002f2ea0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af4a02 00007f68a1af0dd6 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:15.184581 (+ 63838us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b54002f2ea0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6e41d 00007f68a1af4a57 00007f68a1af0dd6 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:15.184856 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349869696}
I20260709 13:13:15.205371 26868 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:15.205273 27109 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:15.188885 27105 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:15.103083 26933 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:13:15.177973 27106 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.217095 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.280110 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.280321 22914 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:15.266368 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2483) took 2479 ms. Trace:
W20260709 13:13:15.266645 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 73722.9us
I20260709 13:13:15.277252 27105 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:15.281674 22763 rpcz_store.cc:276] 0709 13:13:12.787144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.265363 (+2478219us) service_pool.cc:224] Handling call
0709 13:13:15.265798 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.266003 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.266033 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.266073 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:15.266108 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.266167 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.266307 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408}
I20260709 13:13:15.277673 26756 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.282940 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.267297 22760 pending_rounds.cc:107] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:13:15.279212 27093 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:15.283209 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.283507 27109 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.283993 27093 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.284917 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.283746 22760 raft_consensus.cc:2889] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:15.287706 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2471) took 2859 ms. Trace:
I20260709 13:13:15.288026 22760 rpcz_store.cc:276] 0709 13:13:12.428166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.189662 (+2761496us) service_pool.cc:224] Handling call
0709 13:13:15.210724 (+ 21062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.287453 (+ 76729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.287650 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.269750 27031 consensus_queue.cc:1243] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:15.289019 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.289672 22760 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:13:15.292815 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.293984 23462 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:15.136749 22659 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 4 to 8, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 8 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.267938 26977 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.189812 26999 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:15.269825 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2480) took 2488 ms. Trace:
I20260709 13:13:15.178844 27103 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:15.270017 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.266424 22757 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:15.270452 22766 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:15.312837 22762 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:13:15.313021 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2484) took 2525 ms. Trace:
I20260709 13:13:15.272315 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.216322 22910 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:15.217425 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.314363 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.205394 26952 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:15.270412 22765 rpcz_store.cc:276] 0709 13:13:12.779713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.196431 (+2416718us) service_pool.cc:224] Handling call
0709 13:13:15.196732 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.196922 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.196947 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.197009 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:15.197041 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.197094 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.197187 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.315820 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.316570 22760 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
W20260709 13:13:15.270659 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 130364us
I20260709 13:13:15.247213 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2595) took 2470 ms. Trace:
W20260709 13:13:15.299028 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08001c6900 after lost signal to thread 21639
I20260709 13:13:15.317848 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.318059 22912 rpcz_store.cc:276] 0709 13:13:12.776891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.720491 (+1943600us) service_pool.cc:224] Handling call
0709 13:13:14.721217 (+   726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.194031 (+472814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.246773 (+ 52742us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b54000438d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:15.247112 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047393536}
I20260709 13:13:15.199246 26930 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.303399 23458 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:15.311339 22758 rpcz_store.cc:276] 0709 13:13:12.781451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.197104 (+2415653us) service_pool.cc:224] Handling call
0709 13:13:15.197418 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.269561 (+ 72143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.269766 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.277527 26516 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.269076 22759 rpcz_store.cc:276] 0709 13:13:12.780614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.199930 (+2419316us) service_pool.cc:224] Handling call
0709 13:13:15.200252 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.200431 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.200469 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.200515 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:15.200550 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.200608 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.200713 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.267503 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.313072 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2472) took 2884 ms. Trace:
I20260709 13:13:15.314419 22906 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:15.216572 22907 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:15.313598 22757 rpcz_store.cc:276] 0709 13:13:12.787423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.265424 (+2478001us) service_pool.cc:224] Handling call
0709 13:13:15.265940 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.312731 (+ 46791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.312958 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18304}
I20260709 13:13:15.315268 22832 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
W20260709 13:13:15.319096 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08000b0360 after lost signal to thread 22698
I20260709 13:13:15.140200 26829 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.290241 27095 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.321307 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.312608 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.279572 26984 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.374434 22763 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.386128 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.375610 22766 rpcz_store.cc:276] 0709 13:13:12.428422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.193705 (+2765283us) service_pool.cc:224] Handling call
0709 13:13:15.225739 (+ 32034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.312820 (+ 87081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.313015 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.386862 22762 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:15.385917 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.388365 22834 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:15.323602 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:15.370647 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.282650 27105 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.279337 26978 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:15.378930 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.377799 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.393702 22758 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
W20260709 13:13:15.382791 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0f00 after lost signal to thread 22718
I20260709 13:13:15.394608 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.395047 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800366640 after lost signal to thread 22762
W20260709 13:13:15.395747 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5620 after lost signal to thread 22790
W20260709 13:13:15.396024 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800371e00 after lost signal to thread 22791
I20260709 13:13:15.383037 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:15.323107 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.315713 26952 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.384372 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.390328 22912 raft_consensus.cc:2376] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10. Re-sending same reply.
I20260709 13:13:15.397805 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:15.397670 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2583) took 2958 ms. Trace:
I20260709 13:13:15.312139 26999 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.313601 26923 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.403963 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.207836 27061 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:15.401427 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 77901.9us
I20260709 13:13:15.423934 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.401352 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.424443 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 157025us
I20260709 13:13:15.375667 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.437717 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:15.432397 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.432634 22836 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.439208 22757 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:15.369899 27025 raft_consensus.cc:2764] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:13:15.403798 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:15.436945 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.385485 27022 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.381631 27075 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.431928 22910 rpcz_store.cc:276] 0709 13:13:12.438891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.439247 (+   356us) service_pool.cc:224] Handling call
0709 13:13:12.695364 (+256117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.314135 (+2618771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.385291 (+ 71156us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b540020a8d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:15.397576 (+ 12285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5796883840}
I20260709 13:13:15.443662 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.443898 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:15.444180 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.445430 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.446650 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.434886 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:15.447048 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:15.435269 22760 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:15.447892 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.442138 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2487) took 2653 ms. Trace:
W20260709 13:13:15.402697 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800351e80 after lost signal to thread 22806
I20260709 13:13:15.442054 22766 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:15.448796 22757 rpcz_store.cc:276] 0709 13:13:12.788258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.438546 (+2650288us) service_pool.cc:224] Handling call
0709 13:13:15.438882 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.441790 (+  2908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.442075 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.386206 26516 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.431363 26995 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:15.320168 26977 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.450512 22757 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:13:15.461745 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.465090 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2490) took 2676 ms. Trace:
I20260709 13:13:15.431823 26999 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.465492 22757 rpcz_store.cc:276] 0709 13:13:12.789278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.449899 (+2660621us) service_pool.cc:224] Handling call
0709 13:13:15.450190 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.464765 (+ 14575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.465025 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.442821 27103 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:15.504746 22914 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:15.507257 22766 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:15.508239 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2488) took 2719 ms. Trace:
I20260709 13:13:15.508857 22766 rpcz_store.cc:276] 0709 13:13:12.788528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.449621 (+2661093us) service_pool.cc:224] Handling call
0709 13:13:15.506770 (+ 57149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.507864 (+  1094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.508162 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.510561 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:15.513384 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51742.7us
I20260709 13:13:15.514182 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.514904 22834 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:15.514237 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.517105 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:15.454535 22763 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:15.460556 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.450271 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800344720 after lost signal to thread 22865
W20260709 13:13:15.519989 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800350940 after lost signal to thread 22866
I20260709 13:13:15.519883 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:15.520973 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003424c0 after lost signal to thread 22906
I20260709 13:13:15.517385 22760 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:15.521867 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6fa0 after lost signal to thread 22939
W20260709 13:13:15.522197 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800319760 after lost signal to thread 22940
I20260709 13:13:15.522545 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2492) took 2732 ms. Trace:
W20260709 13:13:15.522567 22862 debug-util.cc:398] Leaking SignalData structure 0x7b080037d620 after lost signal to thread 23429
I20260709 13:13:15.396278 26930 raft_consensus.cc:2764] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
W20260709 13:13:15.523941 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800031780 after lost signal to thread 23458
I20260709 13:13:15.462703 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.463930 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.438549 26952 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.448805 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:15.471567 27118 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.188s	user 0.004s	sys 0.000s
I20260709 13:13:15.451577 27066 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:15.473527 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.529265 22766 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:15.529767 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.451669 26992 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:15.432281 26978 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.518849 27009 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:15.479733 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.482091 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.484051 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.459405 27022 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.547647 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.548810 22834 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:15.440567 26938 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:15.436596 27003 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:13:15.487310 26907 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:15.489121 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.464399 27075 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:15.559319 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800394bc0 after lost signal to thread 23459
W20260709 13:13:15.559644 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003bdda0 after lost signal to thread 23460
W20260709 13:13:15.559870 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800395f80 after lost signal to thread 23462
W20260709 13:13:15.560072 22862 debug-util.cc:398] Leaking SignalData structure 0x7b080035b820 after lost signal to thread 23463
W20260709 13:13:15.560268 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08001938a0 after lost signal to thread 23464
W20260709 13:13:15.560484 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800143d40 after lost signal to thread 23465
W20260709 13:13:15.560688 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800364420 after lost signal to thread 23466
W20260709 13:13:15.561250 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003f7860 after lost signal to thread 23490
W20260709 13:13:15.491878 22807 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39296 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
W20260709 13:13:15.569633 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003f74a0 after lost signal to thread 23491
I20260709 13:13:15.493073 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:15.570194 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08002d10e0 after lost signal to thread 26748
W20260709 13:13:15.570614 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08001d37e0 after lost signal to thread 26756
I20260709 13:13:15.570673 22915 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
W20260709 13:13:15.571100 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003434a0 after lost signal to thread 26829
W20260709 13:13:15.571532 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003f7a40 after lost signal to thread 26830
I20260709 13:13:15.501566 26993 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.398149 27120 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:13:15.461827 26923 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.387042 27095 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.554289 26938 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:15.527247 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78574.7us
I20260709 13:13:15.464079 26995 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.391345 26933 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:13:15.529975 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2498) took 2738 ms. Trace:
I20260709 13:13:15.405124 26984 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.493036 27108 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.592303 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.593289 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1763) took 2585 ms. Trace:
I20260709 13:13:15.593878 22839 rpcz_store.cc:276] 0709 13:13:13.007811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.576000 (+2568189us) service_pool.cc:224] Handling call
0709 13:13:15.593198 (+ 17198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.596750 22906 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:15.523007 22760 rpcz_store.cc:276] 0709 13:13:12.789902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.452599 (+2662697us) service_pool.cc:224] Handling call
0709 13:13:15.517216 (+ 64617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.522190 (+  4974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.522469 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.477993 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.584386 26938 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.504094 27103 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.540275 23466 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:13:15.612689 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 95668.7us
I20260709 13:13:15.542855 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.545657 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63638.8us
I20260709 13:13:15.519151 27101 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:15.614240 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.582808 26930 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:15.555790 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.490595 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.628973 22837 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:15.630323 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.556780 22759 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:15.533643 27106 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:15.495194 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.634267 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:15.571959 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003dea40 after lost signal to thread 26845
I20260709 13:13:15.533121 26992 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.533358 27066 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.529831 22909 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:15.580523 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.641489 22833 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:15.581607 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.642520 22915 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:15.555280 26833 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.588173 22766 rpcz_store.cc:276] 0709 13:13:12.791582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.510086 (+2718504us) service_pool.cc:224] Handling call
0709 13:13:15.528922 (+ 18836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.529716 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.529919 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:15.558447 27104 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.534s	user 0.013s	sys 0.000s
I20260709 13:13:15.557011 26907 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.589274 26995 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.601259 22912 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:13:15.502449 22659 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 8 to 9, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.603562 27075 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.603704 22911 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:15.611965 26938 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.605403 26984 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.606537 26516 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.627715 26923 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.610790 27095 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.569967 22807 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39296 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1613)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1627)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 630)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1624)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1612)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1615)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1617)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1620)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1766)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 620)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 622)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 623)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 626)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1795)
I20260709 13:13:15.551926 23465 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:15.631162 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:15.636134 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003dc080 after lost signal to thread 26907
W20260709 13:13:15.682251 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003f75c0 after lost signal to thread 26919
W20260709 13:13:15.682638 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003f79c0 after lost signal to thread 26930
W20260709 13:13:15.682916 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003368c0 after lost signal to thread 26933
W20260709 13:13:15.683140 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003f7920 after lost signal to thread 26938
W20260709 13:13:15.683399 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800160a80 after lost signal to thread 26945
W20260709 13:13:15.683615 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003a59a0 after lost signal to thread 26947
W20260709 13:13:15.683831 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800357be0 after lost signal to thread 26948
W20260709 13:13:15.684038 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad440 after lost signal to thread 26952
W20260709 13:13:15.684296 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800394220 after lost signal to thread 26975
W20260709 13:13:15.684511 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800401520 after lost signal to thread 26976
W20260709 13:13:15.684716 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4ec0 after lost signal to thread 26977
W20260709 13:13:15.684954 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003dddc0 after lost signal to thread 26979
W20260709 13:13:15.685179 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800331f40 after lost signal to thread 26987
W20260709 13:13:15.685710 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800406d40 after lost signal to thread 26988
W20260709 13:13:15.686033 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003d70a0 after lost signal to thread 26992
I20260709 13:13:15.636729 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:15.644892 23432 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.569s (SENT)
I20260709 13:13:15.649483 22841 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:13:15.686271 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800382a60 after lost signal to thread 26995
W20260709 13:13:15.686930 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800271900 after lost signal to thread 26996
I20260709 13:13:15.687130 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1612) took 1042 ms. Trace:
I20260709 13:13:15.650772 22837 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:15.687143 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800373ca0 after lost signal to thread 26998
W20260709 13:13:15.688360 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08000bb2a0 after lost signal to thread 26999
W20260709 13:13:15.688591 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9600 after lost signal to thread 27003
I20260709 13:13:15.688588 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1615) took 1043 ms. Trace:
W20260709 13:13:15.689121 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800391720 after lost signal to thread 27005
W20260709 13:13:15.689471 22862 debug-util.cc:398] Leaking SignalData structure 0x7b080037b0c0 after lost signal to thread 27015
I20260709 13:13:15.653813 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 521) took 2650 ms. Trace:
I20260709 13:13:15.614840 27103 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.657666 22835 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:15.658496 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.603593 27108 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.589015 26830 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.607858 26978 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.628894 27042 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:15.666805 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:15.721607 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 90529.8us
I20260709 13:13:15.628057 26930 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:15.676390 22766 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:15.634954 26947 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:15.647811 26992 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.646492 26979 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:15.647725 27066 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:15.701829 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.711148 22833 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:13:15.690001 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800385c20 after lost signal to thread 27022
W20260709 13:13:15.740973 22862 debug-util.cc:398] Leaking SignalData structure 0x7b0800222720 after lost signal to thread 27028
I20260709 13:13:15.651873 26993 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:15.716212 22760 rpcz_store.cc:276] 0709 13:13:13.003099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.615293 (+2612194us) service_pool.cc:224] Handling call
0709 13:13:15.653277 (+ 37984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.653465 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.653488 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.653563 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:15.653597 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.653650 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.653754 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.731704 26977 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.750985 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.751953 22760 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:15.710254 27095 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:15.753094 27066 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.761193 27066 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.760588 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:15.698590 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 198085us
I20260709 13:13:15.766431 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 522) took 2763 ms. Trace:
I20260709 13:13:15.767009 22766 rpcz_store.cc:276] 0709 13:13:13.003410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.647087 (+2643677us) service_pool.cc:224] Handling call
0709 13:13:15.675869 (+ 28782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.741382 (+ 65513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.766350 (+ 24968us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.767385 22836 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:15.722936 22732 leader_election.cc:400] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:15.768342 22732 leader_election.cc:403] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:13:15.657059 26907 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:15.725008 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1761) took 2717 ms. Trace:
I20260709 13:13:15.780776 22835 rpcz_store.cc:276] 0709 13:13:13.007211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.647842 (+2640631us) service_pool.cc:224] Handling call
0709 13:13:15.648215 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.724664 (+ 76449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.724936 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.782543 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.784236 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.687656 22841 rpcz_store.cc:276] 0709 13:13:14.644271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.648738 (+1004467us) service_pool.cc:224] Handling call
0709 13:13:15.649079 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.686828 (+ 37749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.687068 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.786119 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.689193 22837 rpcz_store.cc:276] 0709 13:13:14.645225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.650181 (+1004956us) service_pool.cc:224] Handling call
0709 13:13:15.650536 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.688266 (+ 37730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.688525 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.728776 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.661763 22659 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 7 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:15.741291 22862 debug-util.cc:398] Leaking SignalData structure 0x7b080024abc0 after lost signal to thread 27031
W20260709 13:13:15.788360 22862 debug-util.cc:398] Leaking SignalData structure 0x7b080038de60 after lost signal to thread 27042
W20260709 13:13:15.788677 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003ea9e0 after lost signal to thread 27054
W20260709 13:13:15.788921 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4720 after lost signal to thread 27059
W20260709 13:13:15.789148 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08002bb2c0 after lost signal to thread 27061
W20260709 13:13:15.789425 22862 debug-util.cc:398] Leaking SignalData structure 0x7b080032bfa0 after lost signal to thread 27068
W20260709 13:13:15.693300 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56658.2us
I20260709 13:13:15.741209 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1617) took 1095 ms. Trace:
W20260709 13:13:15.789688 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60825.2us
I20260709 13:13:15.789965 22833 rpcz_store.cc:276] 0709 13:13:14.645792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.674224 (+1028432us) service_pool.cc:224] Handling call
0709 13:13:15.710752 (+ 36528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.740808 (+ 30056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.741146 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.743206 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.730749 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1613) took 1086 ms. Trace:
I20260709 13:13:15.790843 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:15.789731 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08003385e0 after lost signal to thread 27071
W20260709 13:13:15.792092 22862 debug-util.cc:398] Leaking SignalData structure 0x7b08002ea8a0 after lost signal to thread 27075
I20260709 13:13:15.764672 22760 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:15.770538 22764 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:15.717286 27127 raft_consensus.cc:2764] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:15.773135 26987 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:13:15.777034 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.806757 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.807755 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.657349 27104 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:15.792367 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49101.5us
I20260709 13:13:15.810582 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.791035 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.791112 22840 rpcz_store.cc:276] 0709 13:13:14.644527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.629446 (+984919us) service_pool.cc:224] Handling call
0709 13:13:15.730660 (+101214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.816735 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:13:15.693470 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 246475us
I20260709 13:13:15.796379 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.801115 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.785008 22835 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:15.802603 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.803865 22837 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:15.820117 26977 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:15.820283 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1620) took 1173 ms. Trace:
I20260709 13:13:15.820739 22837 rpcz_store.cc:276] 0709 13:13:14.646586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.803144 (+1156558us) service_pool.cc:224] Handling call
0709 13:13:15.803507 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.820031 (+ 16524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.820227 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:15.821117 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.821358 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.821889 22835 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:15.822198 26977 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:15.791574 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.739687 26930 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.824120 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.825820 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.822654 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:15.806207 27095 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.802232 26979 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:15.806211 26987 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:15.804735 27109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.014s	sys 0.033s
W20260709 13:13:15.845824 27109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.014s	sys 0.034s
I20260709 13:13:15.827266 22833 raft_consensus.cc:1217] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[8.6-12.8]   Dedup: 8.6->[10.7-12.8]
I20260709 13:13:15.827836 27101 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:15.828477 27123 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:15.873759 27123 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:15.874037 22761 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:15.874420 27123 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.876256 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.877058 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2481) took 3091 ms. Trace:
I20260709 13:13:15.829758 22839 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:15.894973 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.831372 22832 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:15.744524 26993 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:15.902451 26993 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.822181 22837 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:15.833258 22908 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:15.906888 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:15.834529 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 111787us
I20260709 13:13:15.908962 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:15.836879 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.838792 27066 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:15.840149 27108 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:15.840618 26978 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:15.846935 22833 pending_rounds.cc:85] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:15.952613 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.953150 22833 pending_rounds.cc:107] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:15.953414 22833 raft_consensus.cc:2889] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:15.953912 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.848073 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.954505 22908 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:15.847980 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.955224 23465 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:15.848800 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.849872 26988 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:15.853008 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:15.956329 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.853611 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:15.854964 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.956359 26988 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:15.858204 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.959065 27109 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:15.958973 26988 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.859759 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:15.960709 22763 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:15.963212 26830 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.860236 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.802407 26947 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:15.965917 22906 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:15.966470 26947 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:15.966991 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.967891 27038 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.969811 26947 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:15.972903 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.861198 22762 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:15.864216 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.974454 22914 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:15.974538 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:15.867885 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 525) took 2863 ms (client timeout 2223 ms). Trace:
W20260709 13:13:15.975476 22764 rpcz_store.cc:269] 0709 13:13:13.004280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.040339 (+2036059us) service_pool.cc:224] Handling call
0709 13:13:15.867629 (+827290us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400311d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:15.867806 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665299200}
I20260709 13:13:15.847491 26977 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:15.878340 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.976432 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:15.884546 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.886473 27093 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:15.887951 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.989893 22765 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:15.889458 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.991206 22759 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:15.890719 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.992301 22910 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:15.993884 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:15.862602 27129 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:15.893004 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:15.828617 27126 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:16.008001 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:16.008706 22764 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
W20260709 13:13:15.880800 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49522.1us
I20260709 13:13:15.895721 22760 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:16.010443 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:15.896729 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.012940 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.895361 22761 rpcz_store.cc:276] 0709 13:13:12.786061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.210517 (+2424456us) service_pool.cc:224] Handling call
0709 13:13:15.873745 (+663228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.874715 (+   970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.876722 (+  2007us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400311d50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:15.876951 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524421504}
W20260709 13:13:15.898818 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.910099 22766 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:15.905205 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.913383 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:16.025012 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:15.837205 22835 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:15.795662 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.025776 22835 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:16.026762 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.028342 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.928633 22909 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:16.029515 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:16.025422 23458 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:15.937907 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:16.031889 22764 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:15.856213 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:15.959348 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:15.976871 22834 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:16.033623 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:16.033911 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.035437 22834 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:15.879117 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.037333 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 527) took 3032 ms. Trace:
I20260709 13:13:15.851559 26938 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:15.799547 27105 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:15.891310 22659 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 8, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:15.996790 26829 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 13:13:15.996833 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 148918us
I20260709 13:13:16.055506 22758 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:13:16.056434 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 530) took 1347 ms. Trace:
I20260709 13:13:16.056732 22758 rpcz_store.cc:276] 0709 13:13:14.708654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.054652 (+1345998us) service_pool.cc:224] Handling call
0709 13:13:16.055011 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.056093 (+  1082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.056374 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:16.001564 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:16.001739 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.999687 27108 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:15.939960 27009 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:15.932986 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:15.958029 22915 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:13:15.981946 22838 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:16.062820 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.063750 22836 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:15.942193 27003 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:13:16.083973 22908 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:16.084888 22840 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:13:16.085682 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 626) took 1109 ms. Trace:
I20260709 13:13:16.086086 22840 rpcz_store.cc:276] 0709 13:13:14.975925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.036753 (+1060828us) service_pool.cc:224] Handling call
0709 13:13:16.084445 (+ 47692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.085357 (+   912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.085620 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:16.032918 22907 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:16.040776 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.041286 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:16.017004 27109 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:16.035950 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 623) took 1061 ms. Trace:
I20260709 13:13:16.106124 22834 rpcz_store.cc:276] 0709 13:13:14.974541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.034880 (+1060339us) service_pool.cc:224] Handling call
0709 13:13:16.035139 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.035733 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.035897 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:16.043299 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:15.995818 26948 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:13:15.998219 27031 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:16.118350 22760 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:16.050696 22763 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:16.120147 22907 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:13:16.120731 22907 pending_rounds.cc:85] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 7
I20260709 13:13:16.037869 22764 rpcz_store.cc:276] 0709 13:13:13.004804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.031274 (+3026470us) service_pool.cc:224] Handling call
0709 13:13:16.031576 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.037067 (+  5491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.037264 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:16.121029 22907 pending_rounds.cc:107] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:13:16.060158 27038 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:16.070861 27009 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:16.070992 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 138102us
I20260709 13:13:16.071853 27108 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:16.074198 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.076117 26987 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:16.077910 22762 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:16.079025 27066 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:15.941411 26585 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:13:16.047677 27042 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:16.089190 22759 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:13:16.042467 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:16.004736 27129 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:16.153905 26987 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:16.108603 22832 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:16.118697 27003 consensus_queue.cc:1243] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:13:16.108597 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 291924us
I20260709 13:13:16.120167 22761 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 7. Current term is 9. Ops: []
I20260709 13:13:16.121402 22907 raft_consensus.cc:2889] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:16.161757 22805 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:16.106822 26938 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:16.122705 22763 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:13:16.129055 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.140777 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.173902 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.135874 22764 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:16.144546 22766 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:16.089800 27022 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:16.152137 26995 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:16.123216 27038 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.183383 22763 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:16.153844 27042 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.154870 27129 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:16.155450 22832 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:15.997972 27120 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:13:16.044103 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.129305 27106 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:16.123296 27031 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:16.181789 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:16.259220 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77532.4us
I20260709 13:13:16.260213 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.195749 22832 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:16.150249 26948 consensus_queue.cc:1243] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:13:16.186520 27038 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:16.198645 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 154677us
I20260709 13:13:16.111300 26933 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:13:16.204044 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:16.251894 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61363.4us
I20260709 13:13:16.194010 27042 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.170786 26938 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.257701 22760 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:16.261878 22832 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.186385 27127 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:16.191924 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.256718 22766 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:13:16.304473 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.306219 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.306766 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.194188 27009 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:16.270617 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:16.296762 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:16.307601 23461 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:16.287355 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:16.283722 26999 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.002s
I20260709 13:13:16.262923 27031 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.262777 27093 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:16.276093 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2618) took 1079 ms. Trace:
I20260709 13:13:16.311370 22837 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:16.186239 27101 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.109s
W20260709 13:13:16.325842 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 488902us
W20260709 13:13:16.326979 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 56525.7us
I20260709 13:13:16.336621 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.295470 26948 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:16.269743 27126 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:13:16.299537 26952 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.004s
I20260709 13:13:16.337549 22909 rpcz_store.cc:276] 0709 13:13:15.196244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.029397 (+833153us) service_pool.cc:224] Handling call
0709 13:13:16.029753 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.029992 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:13:16.066054 (+ 36062us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.274163 (+208109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:16.274253 (+    90us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:16.275745 (+  1492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.275846 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.275989 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14140,"prepare.queue_time_us":37295,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3630,"raft.queue_time_us":14431,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":17507,"thread_start_us":193227,"threads_started":1}
W20260709 13:13:16.339350 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 354378us
I20260709 13:13:16.340878 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:16.303761 26938 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.327879 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.356809 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:16.346575 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:16.358820 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:16.352262 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:16.359635 23457 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.362294 22909 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:16.363197 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 527) took 1087 ms. Trace:
I20260709 13:13:16.363642 22909 rpcz_store.cc:276] 0709 13:13:15.276194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.361402 (+1085208us) service_pool.cc:224] Handling call
0709 13:13:16.362054 (+   652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.362838 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.363124 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:16.332461 26979 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:16.366521 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.376196 22909 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:16.367033 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:16.377399 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:16.334779 27093 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.412818 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:16.374686 26977 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:16.332916 27100 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:16.359975 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:16.309193 27022 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:13:16.332741 27133 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:13:16.333796 26999 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:16.374315 22832 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:13:16.457041 22832 pending_rounds.cc:85] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:13:16.457444 22832 pending_rounds.cc:107] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:13:16.457804 22832 raft_consensus.cc:2889] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:16.393040 27109 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:16.394413 26979 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.393714 27003 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:17.018128 27003 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.019035 27003 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:17.022828 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.800s (SENT) [suppressed 59 similar messages]
W20260709 13:13:16.433154 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.449594 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89659.4us
I20260709 13:13:16.455324 22841 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:13:17.045441 22908 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.004s	sys 0.000s
W20260709 13:13:17.046054 22908 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.005s	sys 0.000s
I20260709 13:13:17.045917 22841 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:16.357072 26952 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:16.501081 26979 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:16.504617 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 199124us
W20260709 13:13:16.958263 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.077235 27164 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:16.427816 26977 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.080248 26977 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:17.080368 27164 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:17.081588 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 528) took 4020 ms. Trace:
I20260709 13:13:17.082120 22761 rpcz_store.cc:276] 0709 13:13:13.005062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.038614 (+3033552us) service_pool.cc:224] Handling call
0709 13:13:16.119908 (+ 81294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.160656 (+ 40748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.024776 (+864120us) spinlock_profiling.cc:243] Waited 80.9 ms on lock 0x7b540002e4d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:17.025272 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186173696}
W20260709 13:13:16.958109 22833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.003s	sys 0.007s
W20260709 13:13:16.959038 22840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.007s	sys 0.015s
W20260709 13:13:17.086194 27134 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.754s	user 0.005s	sys 0.013s
W20260709 13:13:17.086481 22840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.007s	sys 0.026s
W20260709 13:13:16.936592 22764 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.002s	sys 0.015s
W20260709 13:13:17.087495 22764 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.002s	sys 0.027s
I20260709 13:13:17.091179 26952 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:17.086210 22833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.005s	sys 0.013s
I20260709 13:13:17.092823 26952 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.111953 27104 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:16.516330 27061 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:17.112608 27042 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.797s	user 0.012s	sys 0.012s
I20260709 13:13:16.506630 27091 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:13:16.428588 27093 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:16.449132 27100 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:17.166342 27100 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:16.400110 27105 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:16.387686 27106 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:16.500032 26948 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:13:16.464171 27109 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.227208 27109 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:17.228238 27170 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:17.228803 27170 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:16.963563 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 611117us
I20260709 13:13:16.394300 27126 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:16.448230 27135 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:13:17.234442 27126 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:16.534777 27137 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:13:17.236086 27126 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:17.011890 22910 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.002s	sys 0.011s
I20260709 13:13:16.930444 22659 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 7, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 7 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.239490 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 783) took 1423 ms. Trace:
W20260709 13:13:17.011971 22915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.006s
W20260709 13:13:17.240715 22915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.005s	sys 0.007s
I20260709 13:13:17.241946 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2619) took 2045 ms. Trace:
I20260709 13:13:17.022794 22911 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:13:17.038947 22757 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.014s	sys 0.027s
W20260709 13:13:17.243697 22757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.239s	user 0.020s	sys 0.032s
I20260709 13:13:17.242352 22915 rpcz_store.cc:276] 0709 13:13:15.196540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.073425 (+876885us) service_pool.cc:224] Handling call
0709 13:13:16.073876 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:16.074575 (+   699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.074609 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:16.075546 (+   937us) log.cc:844] Serialized 26 byte log entry
0709 13:13:16.424233 (+348687us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:16.456105 (+ 31872us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.102355 (+646250us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b48000b0c58. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:13:17.241233 (+138878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.241367 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.241418 (+    51us) raft_consensus.cc:1685] finished
0709 13:13:17.241582 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.241814 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39134,"prepare.run_cpu_time_us":26202,"prepare.run_wall_time_us":520769,"raft.queue_time_us":636164,"raft.run_cpu_time_us":2800,"raft.run_wall_time_us":1028486,"spinlock_wait_cycles":37259520,"thread_start_us":1132757,"threads_started":2,"wal-append.queue_time_us":969829}
W20260709 13:13:17.245604 22910 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.003s	sys 0.011s
I20260709 13:13:17.246834 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2616) took 2053 ms. Trace:
W20260709 13:13:17.044289 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.006s	sys 0.000s
W20260709 13:13:17.248009 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.007s	sys 0.000s
I20260709 13:13:17.247225 22910 rpcz_store.cc:276] 0709 13:13:15.193855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.001097 (+807242us) service_pool.cc:224] Handling call
0709 13:13:16.047284 (+ 46187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.047514 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:16.260105 (+212591us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.246260 (+986155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.246385 (+   125us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:17.246492 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.246604 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.246777 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18366,"prepare.queue_time_us":96309,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":638960,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":3400,"thread_start_us":1179396,"threads_started":3,"wal-append.queue_time_us":783692}
W20260709 13:13:17.249212 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 836475us
W20260709 13:13:17.044960 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.000s	sys 0.007s
I20260709 13:13:17.048316 22841 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.044987 27163 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:17.033491 26999 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:16.961236 22835 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.072757 22913 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:16.525295 26829 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:17.274577 26829 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.275207 26829 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.276403 26995 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.081884 27022 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:13:17.277171 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.278229 27022 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:17.278700 27022 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:17.280308 27022 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:17.119570 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 97019.3us
W20260709 13:13:17.285655 27031 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.918s	user 0.021s	sys 0.009s
W20260709 13:13:17.286219 23432 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.663s (SENT)
I20260709 13:13:17.286949 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.289788 27164 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.399560 27066 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:13:17.146196 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1795) took 2058 ms. Trace:
W20260709 13:13:17.140717 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45723.4us
W20260709 13:13:17.178220 22766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.005s	sys 0.008s
W20260709 13:13:17.178962 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.005s	sys 0.022s
I20260709 13:13:17.215220 27105 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:17.212077 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.014s	sys 0.010s
W20260709 13:13:17.269946 22765 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.006s	sys 0.014s
I20260709 13:13:17.139393 26952 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.134249 27042 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.233314 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:17.258572 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.000s	sys 0.008s
I20260709 13:13:17.271522 22835 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:17.273993 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:17.206723 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 128148us
I20260709 13:13:17.251523 27163 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.297041 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:17.302227 27136 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:13:17.240051 23466 rpcz_store.cc:276] 0709 13:13:15.815664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.815794 (+   130us) service_pool.cc:224] Handling call
0709 13:13:15.816130 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.816502 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:15.816541 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:15.817034 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:13:16.090643 (+273609us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:16.090699 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.090793 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:16.415085 (+324292us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b48000b0358. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:13:17.237674 (+822589us) raft_consensus.cc:1685] finished
0709 13:13:17.238826 (+  1152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.239191 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166705,"raft.run_cpu_time_us":4939,"raft.run_wall_time_us":425709,"spinlock_wait_cycles":9376512,"thread_start_us":250780,"threads_started":1,"wal-append.queue_time_us":492785}
I20260709 13:13:17.303932 27042 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.304440 27133 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:13:17.304762 26907 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.475s	user 0.004s	sys 0.029s
W20260709 13:13:17.301669 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.014s	sys 0.011s
I20260709 13:13:17.304658 22841 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:17.305838 22766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.007s	sys 0.011s
W20260709 13:13:17.305855 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.005s	sys 0.023s
W20260709 13:13:17.305150 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 938190us
I20260709 13:13:17.276435 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:17.321619 22765 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.007s	sys 0.014s
I20260709 13:13:17.230216 27170 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.269719 27165 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:17.322839 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2610) took 2123 ms (client timeout 1549 ms). Trace:
W20260709 13:13:17.331756 22913 rpcz_store.cc:269] 0709 13:13:15.183267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.186781 (+  3514us) service_pool.cc:224] Handling call
0709 13:13:17.293377 (+2106596us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540010f4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:17.306874 (+ 13497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4237001984}
W20260709 13:13:17.329874 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 96800.7us
W20260709 13:13:17.326043 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 979695us
W20260709 13:13:17.325897 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 984055us
I20260709 13:13:17.306572 27163 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:17.340992 26992 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.546s	user 0.011s	sys 0.028s
I20260709 13:13:17.341154 22909 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:17.342078 27156 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:13:17.341858 26992 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.294750 22840 rpcz_store.cc:276] 0709 13:13:15.087396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.087416 (+1000020us) service_pool.cc:224] Handling call
0709 13:13:16.134238 (+ 46822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.134519 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:16.360504 (+225985us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.139131 (+778627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.145559 (+  6428us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:17.145701 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.145837 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.146087 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42637,"prepare.queue_time_us":891035,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":111,"raft.queue_time_us":49003,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":22313,"thread_start_us":929401,"threads_started":2,"wal-append.queue_time_us":687787}
I20260709 13:13:17.343421 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.343838 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.344571 22840 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:17.345083 22906 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.346691 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 784) took 1530 ms. Trace:
W20260709 13:13:17.348150 27103 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.604s	user 0.031s	sys 0.021s
I20260709 13:13:17.347209 23462 rpcz_store.cc:276] 0709 13:13:15.815958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.816121 (+   163us) service_pool.cc:224] Handling call
0709 13:13:15.816424 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:16.280290 (+463866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.280341 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:16.297850 (+ 17509us) log.cc:844] Serialized 26 byte log entry
0709 13:13:16.727426 (+429576us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:16.727483 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.727582 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.346099 (+618517us) raft_consensus.cc:1685] finished
0709 13:13:17.346139 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.346426 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48520,"raft.queue_time_us":179172,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15349,"thread_start_us":869993,"threads_started":2,"wal-append.queue_time_us":873805}
I20260709 13:13:17.350852 26952 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.351305 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1791) took 2343 ms. Trace:
I20260709 13:13:17.338620 27105 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.351794 22836 rpcz_store.cc:276] 0709 13:13:15.007662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.064560 (+1056898us) service_pool.cc:224] Handling call
0709 13:13:16.111885 (+ 47325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:16.112321 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.112362 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:16.136566 (+ 24204us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.042022 (+905456us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b1800421960. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:13:17.322075 (+280053us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:17.322136 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.327312 (+  5176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.350688 (+ 23376us) raft_consensus.cc:1685] finished
0709 13:13:17.350718 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.351036 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":70327,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":13937,"spinlock_wait_cycles":2577152,"thread_start_us":170033,"threads_started":1,"wal-append.queue_time_us":1184885}
I20260709 13:13:17.354722 27185 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:13:17.357812 26993 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.426s	user 0.018s	sys 0.007s
I20260709 13:13:17.358593 26993 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.359169 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1806) took 2161 ms. Trace:
I20260709 13:13:17.360138 22907 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:17.361959 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:17.364989 27123 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.489s	user 0.012s	sys 0.021s
I20260709 13:13:17.365833 27170 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:17.365651 27123 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.367015 27160 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:17.368135 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.369441 27101 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:17.370559 27101 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:17.371148 27101 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.371672 27101 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:17.374373 27145 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.906s	user 0.006s	sys 0.000s
W20260709 13:13:17.375674 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:17.376413 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:17.376889 27129 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.118s	user 0.013s	sys 0.013s
I20260709 13:13:17.305433 26907 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.377354 22910 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:13:17.377460 27129 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.378417 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2623) took 1477 ms. Trace:
I20260709 13:13:17.378831 22910 rpcz_store.cc:276] 0709 13:13:15.901293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.249244 (+1347951us) service_pool.cc:224] Handling call
0709 13:13:17.298905 (+ 49661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.378034 (+ 79129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.378324 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:17.379527 26907 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.379920 27129 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.381016 22910 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:17.381168 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2614) took 2187 ms. Trace:
I20260709 13:13:17.381624 22914 rpcz_store.cc:276] 0709 13:13:15.193281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.997117 (+803836us) service_pool.cc:224] Handling call
0709 13:13:16.042618 (+ 45501us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:16.046229 (+  3611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.046263 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:16.046891 (+   628us) log.cc:844] Serialized 52 byte log entry
0709 13:13:16.267885 (+220994us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:16.302002 (+ 34117us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.302397 (+   395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.302591 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.211019 (+908428us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:17.380627 (+169608us) raft_consensus.cc:1685] finished
0709 13:13:17.380787 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.380983 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3061,"prepare.queue_time_us":33640,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":9392,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":46306,"spinlock_wait_cycles":72937216,"thread_start_us":179936,"threads_started":1,"wal-append.queue_time_us":311154}
W20260709 13:13:17.383947 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.384979 27143 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:17.387110 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.385672 27143 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.310058 27003 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.390661 27143 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.391582 27003 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.392800 22834 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:17.286232 27031 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.338357 27157 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:13:17.395511 27031 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.395303 27093 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.396389 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:17.397188 23457 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.036s	user 0.010s	sys 0.018s
I20260709 13:13:17.397681 23457 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:17.397835 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:17.398357 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 668) took 1041 ms. Trace:
I20260709 13:13:17.398720 23457 rpcz_store.cc:276] 0709 13:13:16.356824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.358517 (+  1693us) service_pool.cc:224] Handling call
0709 13:13:17.398216 (+1039699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:17.369465 23464 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:17.240321 22659 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 4 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.368358 27123 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:17.401754 23432 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.722s (SENT)
I20260709 13:13:17.359699 22765 rpcz_store.cc:276] 0709 13:13:15.197458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.064628 (+867170us) service_pool.cc:224] Handling call
0709 13:13:16.065309 (+   681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.065677 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:16.358161 (+292484us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.281315 (+923154us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:17.358288 (+ 76973us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.358605 (+   317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.358688 (+    83us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:17.358768 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.358884 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.359077 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4939,"prepare.queue_time_us":984826,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":54,"raft.queue_time_us":7157,"spinlock_wait_cycles":93070080,"thread_start_us":1195607,"threads_started":2,"wal-append.queue_time_us":865319}
I20260709 13:13:17.348757 27103 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.399053 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.410120 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2624) took 1505 ms. Trace:
I20260709 13:13:17.400367 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2625) took 1473 ms. Trace:
I20260709 13:13:17.410559 22910 rpcz_store.cc:276] 0709 13:13:15.901747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.379964 (+1478217us) service_pool.cc:224] Handling call
0709 13:13:17.380568 (+   604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.405639 (+ 25071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.406796 (+  1157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:17.411005 22914 rpcz_store.cc:276] 0709 13:13:15.911447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.383759 (+1472312us) service_pool.cc:224] Handling call
0709 13:13:17.384208 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.384523 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.384551 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.384621 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:17.384660 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.384726 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.384865 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:17.412992 26987 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.248s	user 0.014s	sys 0.007s
I20260709 13:13:17.413882 26999 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:17.414340 26999 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.414745 26999 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.414839 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1832) took 1669 ms. Trace:
I20260709 13:13:17.414353 26830 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:17.419451 27038 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.113s	user 0.011s	sys 0.011s
W20260709 13:13:17.420614 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.421995 27105 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.422413 27103 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.427044 27109 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:17.427677 27173 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.185s	user 0.002s	sys 0.000s
W20260709 13:13:17.428251 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.418080 23460 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:17.416599 27093 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.418006 27108 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:17.506438 27108 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.506990 27108 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:17.507241 27178 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.245s	user 0.000s	sys 0.006s
I20260709 13:13:17.507839 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 786) took 1304 ms. Trace:
I20260709 13:13:17.508497 23460 rpcz_store.cc:276] 0709 13:13:16.203275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.203697 (+   422us) service_pool.cc:224] Handling call
0709 13:13:17.507456 (+1303759us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400231ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:17.507667 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600636672}
W20260709 13:13:17.431509 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.509442 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:17.509783 27109 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.510576 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 666) took 1161 ms. Trace:
I20260709 13:13:17.511324 27009 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:17.512209 27009 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.434126 27127 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:17.513294 27127 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:17.433159 27022 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:17.515049 27127 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:17.438282 26947 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.468s	user 0.014s	sys 0.020s
I20260709 13:13:17.438939 22659 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:17.440075 27164 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.436043 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2621) took 1535 ms. Trace:
I20260709 13:13:17.442620 26992 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:17.443223 26938 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.064s	user 0.015s	sys 0.011s
I20260709 13:13:17.517331 22909 rpcz_store.cc:276] 0709 13:13:15.900641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.409411 (+508770us) service_pool.cc:224] Handling call
0709 13:13:16.409763 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.342358 (+932595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.435675 (+ 93317us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400211bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:17.435946 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060429056}
I20260709 13:13:17.518244 26938 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.518395 22659 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 7, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:17.519770 22909 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:13:17.440891 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2617) took 2246 ms. Trace:
I20260709 13:13:17.520893 22911 rpcz_store.cc:276] 0709 13:13:15.195881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.005842 (+809961us) service_pool.cc:224] Handling call
0709 13:13:16.017082 (+ 11240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.243033 (+1225951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.440489 (+197456us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54001257d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:17.440821 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150179584}
I20260709 13:13:17.521224 22909 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:17.522656 22911 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:13:17.444597 27181 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:13:17.447496 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 523) took 2171 ms. Trace:
I20260709 13:13:17.361308 26993 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:17.451612 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 86527.7us
I20260709 13:13:17.448030 27160 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:17.453197 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.546s (SENT)
I20260709 13:13:17.453647 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 785) took 1259 ms. Trace:
W20260709 13:13:17.454399 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55467.9us
I20260709 13:13:17.454277 22841 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:13:17.453647 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:17.457878 26930 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.634s	user 0.012s	sys 0.019s
W20260709 13:13:17.458334 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 69630.6us
I20260709 13:13:17.459903 27038 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.460448 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 525) took 2184 ms. Trace:
I20260709 13:13:17.432137 22764 rpcz_store.cc:276] 0709 13:13:15.745761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.184011 (+438250us) service_pool.cc:224] Handling call
0709 13:13:16.184303 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.184492 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:17.215347 (+1030855us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b18002edaa0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf
0709 13:13:17.215601 (+   254us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.329876 (+114275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.338016 (+  8140us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:17.411772 (+ 73756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.414421 (+  2649us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.414646 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":866486,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":35008,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":8712,"spinlock_wait_cycles":58719616,"thread_start_us":955838,"threads_started":2,"wal-append.queue_time_us":194763}
I20260709 13:13:17.431384 27100 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.457229 26999 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:17.524893 27171 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.281s	user 0.005s	sys 0.000s
I20260709 13:13:17.451529 27133 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:17.544677 27100 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.545156 27133 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.474936 26979 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.535650 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.502084 22840 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:17.511037 23465 rpcz_store.cc:276] 0709 13:13:16.348988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.349143 (+   155us) service_pool.cc:224] Handling call
0709 13:13:16.349589 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.014631 (+665042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.014722 (+    91us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.016121 (+  1399us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.137414 (+121293us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:17.137475 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.200589 (+ 63114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.426205 (+225616us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:17.509994 (+ 83789us) raft_consensus.cc:1685] finished
0709 13:13:17.510023 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.510333 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39959,"raft.queue_time_us":100412,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":10844,"spinlock_wait_cycles":38629248,"thread_start_us":456247,"threads_started":2,"wal-append.queue_time_us":208436}
W20260709 13:13:17.433136 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 95654.3us
I20260709 13:13:17.509720 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 530) took 2129 ms. Trace:
I20260709 13:13:17.431713 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1766) took 2513 ms. Trace:
I20260709 13:13:17.535705 23464 rpcz_store.cc:276] 0709 13:13:16.194658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.196564 (+  1906us) service_pool.cc:224] Handling call
0709 13:13:16.196952 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.412837 (+1215885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.453330 (+ 40493us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400093dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:17.453566 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2695397120}
I20260709 13:13:17.550397 22906 rpcz_store.cc:276] 0709 13:13:15.374904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.966700 (+591796us) service_pool.cc:224] Handling call
0709 13:13:17.460929 (+1494229us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540010dbd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:17.504215 (+ 43286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3166747392}
I20260709 13:13:17.551081 26930 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.531075 27127 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:17.553114 26999 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.520330 22659 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.554036 22758 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:17.545445 27100 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:13:17.556389 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 540) took 2511 ms. Trace:
I20260709 13:13:17.538509 26948 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.536182 22841 pending_rounds.cc:85] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:13:17.556736 22758 rpcz_store.cc:276] 0709 13:13:15.044711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.059702 (+1014991us) service_pool.cc:224] Handling call
0709 13:13:16.059974 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.554627 (+1494653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.556090 (+  1463us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54002dafd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:17.556326 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3435243264}
I20260709 13:13:17.557662 22841 pending_rounds.cc:107] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:13:17.557773 27159 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:17.558048 22841 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:17.557680 26948 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.559026 27159 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:17.559262 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1773) took 2565 ms. Trace:
I20260709 13:13:17.559569 27159 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.559736 26979 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.496582 27165 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:17.537655 27160 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.542321 26830 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:17.543160 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.543399 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:17.540508 22907 rpcz_store.cc:276] 0709 13:13:15.275860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.162789 (+886929us) service_pool.cc:224] Handling call
0709 13:13:16.194627 (+ 31838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.360715 (+1166088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.460136 (+ 99421us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000cfb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:17.460381 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2679629056}
I20260709 13:13:17.525163 22908 rpcz_store.cc:276] 0709 13:13:15.275557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.084785 (+809228us) service_pool.cc:224] Handling call
0709 13:13:16.085249 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:16.085680 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:17.046909 (+961229us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.222924 (+176015us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.329815 (+106891us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.330026 (+   211us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:17.365944 (+ 35918us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.399712 (+ 33768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.431858 (+ 32146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.447239 (+ 15381us) raft_consensus.cc:1685] finished
0709 13:13:17.447259 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.447416 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33450,"prepare.queue_time_us":80811,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":125,"raft.queue_time_us":73568,"raft.run_cpu_time_us":545,"raft.run_wall_time_us":45500,"thread_start_us":1070571,"threads_started":2,"wal-append.queue_time_us":221090}
I20260709 13:13:17.564872 22914 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:17.559763 22841 rpcz_store.cc:276] 0709 13:13:14.994682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.058598 (+1063916us) service_pool.cc:224] Handling call
0709 13:13:16.058883 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.455163 (+396280us) spinlock_profiling.cc:243] Waited 99.7 ms on lock 0x7b54001fcd20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af31c0 00007f68a1af4231 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c
0709 13:13:17.558718 (+1103555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.559195 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229368064}
W20260709 13:13:17.566675 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.571326 22835 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:13:17.532166 26952 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:17.584591 26585 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:13:17.550457 22833 rpcz_store.cc:276] 0709 13:13:14.918202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.806282 (+888080us) service_pool.cc:224] Handling call
0709 13:13:15.827050 (+ 20768us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:15.954097 (+127047us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:16.258569 (+304472us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.142562 (+883993us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:17.143896 (+  1334us) log.cc:844] Serialized 52 byte log entry
0709 13:13:17.144110 (+   214us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:17.144557 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.144956 (+   399us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.290620 (+145664us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b1800420d60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9
0709 13:13:17.333181 (+ 42561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.412220 (+ 79039us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.412306 (+    86us) raft_consensus.cc:1685] finished
0709 13:13:17.412333 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.431642 (+ 19309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41191,"prepare.queue_time_us":500639,"prepare.run_cpu_time_us":26083,"prepare.run_wall_time_us":539210,"raft.queue_time_us":51738,"raft.run_cpu_time_us":549,"raft.run_wall_time_us":28873,"spinlock_wait_cycles":96134400,"thread_start_us":1139186,"threads_started":3,"wal-append.queue_time_us":776461}
I20260709 13:13:17.586179 22912 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:13:17.586431 26977 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.505s	user 0.014s	sys 0.004s
I20260709 13:13:17.579525 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:17.587208 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43967.8us
I20260709 13:13:17.587005 26977 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.588294 22911 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:17.588677 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:17.588830 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.589200 27091 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:17.589800 27093 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:17.590184 26979 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:17.591049 27091 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:17.591360 27093 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.591519 27091 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:17.592007 27091 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:17.593722 26930 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.594137 27172 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:17.586021 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46524us
W20260709 13:13:17.594643 27028 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:13:17.594883 27172 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.595533 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.597622 27100 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:17.597743 27109 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.596441 22659 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 4 to 6, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:17.598208 27100 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.598690 27100 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:17.599251 22659 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 8, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:17.600488 27172 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.601398 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.602133 27009 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.602607 26830 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.604426 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.607182 26999 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:17.566849 27165 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.594594 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.613746 26830 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:17.614188 23432 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.922s (SENT)
I20260709 13:13:17.614672 26756 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:17.615357 26756 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:17.615594 23432 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.925s (SENT)
I20260709 13:13:17.616770 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.616469 26756 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.608628 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:17.619674 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.585179 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2626) took 1673 ms. Trace:
I20260709 13:13:17.619197 22880 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.620636 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1770) took 2690 ms. Trace:
I20260709 13:13:17.624768 22910 rpcz_store.cc:276] 0709 13:13:15.911809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.458218 (+1546409us) service_pool.cc:224] Handling call
0709 13:13:17.536523 (+ 78305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.584773 (+ 48250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.584809 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.584853 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:17.584889 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.584962 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.585092 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:17.562453 26938 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.625205 22835 rpcz_store.cc:276] 0709 13:13:14.929917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.822715 (+892798us) service_pool.cc:224] Handling call
0709 13:13:15.822979 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.463592 (+640613us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540041f3a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6f06f 00007f68a1ae7f60 00007f68a1ae97de 00007f68a1af3b49 00007f68a1af4416 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62
0709 13:13:17.581874 (+1118282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.620572 (+ 38698us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556830976}
I20260709 13:13:17.561520 26992 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:17.641134 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:17.641937 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.636449 26979 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:17.649943 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 196804us
I20260709 13:13:17.650784 27003 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.659514 26999 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:17.642935 27167 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:17.660817 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:17.659468 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.661589 26978 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.662115 27038 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.651394 27003 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.665233 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.666952 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.667186 27003 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.667061 27101 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.667761 22832 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:17.644224 26988 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:13:17.644531 26930 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:17.671701 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.672260 22837 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:17.672514 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.660881 27167 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:17.674103 27167 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.672585 22839 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:17.674693 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.678077 26952 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:13:17.681289 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.978s (SENT)
I20260709 13:13:17.681854 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:17.682196 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2615) took 2488 ms. Trace:
I20260709 13:13:17.682569 22912 rpcz_store.cc:276] 0709 13:13:15.193583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.000219 (+806636us) service_pool.cc:224] Handling call
0709 13:13:16.050372 (+ 50153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.586697 (+1536325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.681895 (+ 95198us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54001d2254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:17.682138 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3517171456}
I20260709 13:13:17.652670 26992 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.685945 27127 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.686604 27127 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.696421 27127 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.700750 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.701105 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:17.703403 26977 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.705902 27101 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.705991 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.706276 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:17.707103 23465 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.709872 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.710772 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:17.711210 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.711611 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:17.712249 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.688030 27184 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:17.711229 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:17.713129 22839 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:17.714103 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.715185 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1809) took 1866 ms. Trace:
I20260709 13:13:17.715263 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.716629 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.715669 22838 rpcz_store.cc:276] 0709 13:13:15.848313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.111063 (+262750us) service_pool.cc:224] Handling call
0709 13:13:16.137909 (+ 26846us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:16.423306 (+285397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.423344 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:16.966052 (+542708us) log.cc:844] Serialized 52 byte log entry
0709 13:13:17.105811 (+139759us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:17.105895 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.106044 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.460379 (+354335us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:17.714746 (+254367us) raft_consensus.cc:1685] finished
0709 13:13:17.714768 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.714997 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120480,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":22669,"spinlock_wait_cycles":16584064,"thread_start_us":371766,"threads_started":2,"wal-append.queue_time_us":230710}
I20260709 13:13:17.718840 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:17.720003 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.720659 26948 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.688745 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.722110 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:17.722266 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:17.690337 27179 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:13:17.655200 27108 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.725858 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:17.727144 22906 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:13:17.727273 27108 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.728397 26999 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:17.690410 27093 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.730300 26999 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:17.691448 27180 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:13:17.730896 26999 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:17.646376 22659 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 9, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.634286 26829 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.662194 26978 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.732183 26999 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.733811 26978 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.733952 26829 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.732209 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2493) took 4942 ms. Trace:
I20260709 13:13:17.734724 22837 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:17.734704 22757 rpcz_store.cc:276] 0709 13:13:12.790217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.466317 (+2676100us) service_pool.cc:224] Handling call
0709 13:13:15.466682 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.514712 (+ 48030us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:16.004867 (+490155us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18001bacc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b045e5 00007f68a1b04568
0709 13:13:17.244194 (+1239327us) log.cc:844] Serialized 14 byte log entry
0709 13:13:17.336940 (+ 92746us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.565960 (+229020us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.566151 (+   191us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:17.566175 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.581271 (+ 15096us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.731729 (+150458us) raft_consensus.cc:1685] finished
0709 13:13:17.731844 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.732129 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47562,"prepare.queue_time_us":1535267,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":80,"raft.queue_time_us":47948,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9607,"spinlock_wait_cycles":9185536,"thread_start_us":1324048,"threads_started":2,"wal-append.queue_time_us":207215}
I20260709 13:13:17.732926 22659 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 4 to 8, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 8 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:17.737076 22907 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:17.687556 26992 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:17.712519 23460 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:17.681875 22910 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.722885 22836 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:17.716172 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2622) took 1815 ms. Trace:
I20260709 13:13:17.688690 22832 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:17.704919 26977 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:13:17.738473 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89963us
I20260709 13:13:17.770975 22835 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:17.769452 22915 rpcz_store.cc:276] 0709 13:13:15.900995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.245038 (+1344043us) service_pool.cc:224] Handling call
0709 13:13:17.294768 (+ 49730us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:17.388995 (+ 94227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.389033 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:17.398928 (+  9895us) log.cc:844] Serialized 52 byte log entry
0709 13:13:17.626150 (+227222us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:17.626195 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.626282 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.715883 (+ 89601us) raft_consensus.cc:1685] finished
0709 13:13:17.715908 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.716083 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58926,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":12803,"thread_start_us":226730,"threads_started":1,"wal-append.queue_time_us":292818}
W20260709 13:13:17.776222 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54084us
W20260709 13:13:17.776859 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46680.7us
I20260709 13:13:17.778985 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.779147 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.784072 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:17.785145 22835 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:17.787731 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:17.788563 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:17.788905 22910 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.789355 23464 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:17.737317 22659 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 9 to 10, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 10 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.792863 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:17.793812 27187 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:17.790131 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:17.792661 27186 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:13:17.815135 22907 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:17.815734 23432 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.128s (SENT)
I20260709 13:13:17.820988 26988 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:13:17.821725 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 149501us
I20260709 13:13:17.797986 22915 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.835889 22910 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:17.739018 26947 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:13:17.785447 27164 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:17.846318 22906 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:17.848237 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:17.850284 22912 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.852743 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.858938 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.827564 26988 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:17.859638 22910 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:17.737854 27160 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.863144 22907 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.834208 27159 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.865840 27160 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.866008 27159 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:17.848419 27136 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:13:17.867481 27136 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:17.867331 27100 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.867892 27136 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:17.821748 27066 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:17.843823 27101 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.853055 27127 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.829577 27163 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.871672 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 543) took 1520 ms. Trace:
I20260709 13:13:17.881151 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.880061 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.838238 26992 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:13:17.877017 26756 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.877915 26979 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:17.880353 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:17.880613 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94695.8us
I20260709 13:13:17.880229 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:17.862663 27170 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:17.874075 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:17.885588 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 204344us
I20260709 13:13:17.886982 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.889164 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.896554 23465 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:17.850032 27091 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.896394 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.881367 22766 rpcz_store.cc:276] 0709 13:13:16.351340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.351475 (+   135us) service_pool.cc:224] Handling call
0709 13:13:16.351795 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.340966 (+989171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.341055 (+    89us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.341826 (+   771us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.504570 (+162744us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:17.504713 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.538186 (+ 33473us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.871432 (+333246us) raft_consensus.cc:1685] finished
0709 13:13:17.871457 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.871611 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":674067,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":404,"thread_start_us":1116406,"threads_started":2,"wal-append.queue_time_us":287035}
W20260709 13:13:17.903126 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64211.3us
I20260709 13:13:17.852883 27184 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 13:13:17.902551 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62109.2us
I20260709 13:13:17.875447 26977 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:13:17.907240 22913 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.908385 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:17.910995 27103 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:17.895159 27101 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:17.911556 23432 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.156s (SENT)
I20260709 13:13:17.912520 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.913663 27187 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:17.914675 27101 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.898869 27127 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.906673 27160 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.918309 27160 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:17.920742 27160 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.880046 27136 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.865358 27164 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.925689 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.905323 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.927584 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.934481 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.935873 22908 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:17.917289 27127 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:17.932564 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.920274 27143 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:13:17.909049 27157 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:17.950886 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:17.921747 26756 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:17.911588 27103 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:17.952983 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47791.7us
I20260709 13:13:17.922258 27136 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:17.992794 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 96508.1us
I20260709 13:13:17.954955 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:17.924088 26993 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.954504 22907 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:18.000351 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:17.948299 27159 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:17.901363 27066 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.004448 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:17.954070 26977 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:18.009374 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:18.010582 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99090.9us
I20260709 13:13:18.009755 26948 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:18.014570 26948 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.015537 26948 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.017127 26948 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:18.017910 27143 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:18.018335 27143 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.019316 27143 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.020767 27143 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:18.021740 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:18.022861 27160 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:18.022927 27164 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:18.023578 27160 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:18.023905 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:18.025305 27100 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.027519 26830 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:18.028092 26830 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.028599 26830 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.030221 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.030094 27103 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.031200 22766 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:18.034171 26978 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:18.034487 26977 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.034832 26978 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:18.035398 26977 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.036196 22757 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:18.035930 26978 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.036623 27038 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:18.024804 27160 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.037966 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.038667 22764 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:18.038839 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 115315us
I20260709 13:13:18.039186 27141 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:18.041865 26829 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:18.042068 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.042356 26829 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.042850 22765 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:13:18.042739 27184 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:18.043272 27003 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:18.043479 26829 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.043784 27184 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:18.044293 27003 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:18.044731 27003 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.044891 26938 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:18.009553 27172 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:18.045396 26938 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.046101 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.047266 22762 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:18.048840 27022 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:18.050894 27163 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.052729 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:18.052996 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.042s (SENT)
I20260709 13:13:18.053562 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:18.046661 26938 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.015973 27105 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:13:18.055053 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.087s (SENT)
I20260709 13:13:18.074234 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:18.055673 22909 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:18.006238 27170 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.851884 22659 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 9, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:18.057248 22838 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:13:18.049257 27022 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.056383 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:17.951711 26930 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.002754 27133 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.089793 22838 pending_rounds.cc:85] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 7
I20260709 13:13:18.053354 27028 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:18.069344 22912 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:18.080636 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.086232 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:18.087842 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.064337 27141 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:13:18.044905 26829 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:18.103094 22838 pending_rounds.cc:107] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:13:18.103608 23465 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:13:18.115110 22838 raft_consensus.cc:2889] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:18.044705 27184 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:18.052778 27091 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:18.071800 27031 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:18.112830 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.037248 27038 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.074076 27105 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:18.093166 27095 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:18.145457 23460 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:18.121606 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.104429 27167 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.151255 22759 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:18.121955 27165 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.000s
I20260709 13:13:18.156167 22762 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:18.094229 27170 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.121161 26938 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.004479 26988 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.164338 22764 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:18.121374 27184 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.121776 27031 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.168660 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.162982 23462 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:18.179222 22832 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:18.148090 27095 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.151705 26585 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:18.146396 27157 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:18.180480 22757 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.192968 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:18.196750 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.198261 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:18.199397 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.200023 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.160355 27187 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:13:18.202673 27066 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:18.206147 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.206418 27184 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.207988 27184 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:18.212121 27157 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:18.217092 27201 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:13:18.217927 27105 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.219437 27105 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:18.223433 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:18.112272 27003 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.184262 27031 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:18.238691 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.185459 27095 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:18.230816 22765 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:18.161909 27165 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.164831 26930 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.161806 27133 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.263399 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.198900 27160 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.199144 27022 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.248114 22762 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:18.247987 27109 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.254101 27003 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.266268 22915 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:18.257717 27208 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:18.269979 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.270749 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.258181 22912 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:18.285336 23465 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:18.258600 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.259406 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.260250 27038 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.260659 26988 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:18.293458 22764 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.256645 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:18.261972 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.262713 26930 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:18.256595 26979 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.266889 22833 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:18.298095 22833 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:18.297397 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 242383us
I20260709 13:13:18.283033 27022 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.261986 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.300674 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.272588 27143 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:18.278323 27066 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:13:18.274773 26756 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.292135 27170 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.291841 27109 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.173224 27167 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.265472 27133 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:18.296116 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.299527 22906 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:18.274228 27197 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.433s	user 0.006s	sys 0.000s
W20260709 13:13:18.271406 27202 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:13:18.303301 22914 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:18.309713 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.310829 27165 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:18.310648 22833 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.346072 22833 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:18.314697 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:18.351497 27127 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:18.352345 27127 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.280504 27160 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.355305 27127 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.355479 27160 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:18.356377 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99855.9us
I20260709 13:13:18.357679 22763 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:18.364657 26977 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:18.265734 22659 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 3 to 6, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 6 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:18.366546 22659 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:13:18.315373 27172 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:18.316337 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2627) took 1064 ms. Trace:
I20260709 13:13:18.316730 26979 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.375990 22762 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:18.324128 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:18.325642 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.326092 27066 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:18.326210 27208 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:18.328128 27109 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.295182 22766 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:18.329561 22907 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:18.332129 22765 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.338258 27167 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.284327 27164 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:13:18.339984 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:18.369380 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.338300 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.371893 22913 rpcz_store.cc:276] 0709 13:13:17.293894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.343028 (+ 49134us) service_pool.cc:224] Handling call
0709 13:13:18.316091 (+973063us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:18.316237 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1294612736}
I20260709 13:13:18.398670 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:18.407132 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 352139us
I20260709 13:13:18.408936 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:18.410890 23457 raft_consensus.cc:2368] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11.
I20260709 13:13:18.411146 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.412009 22757 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:13:18.413889 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.414484 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.415258 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.415702 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.418717 22759 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:18.418972 22760 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:18.424227 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.425611 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.424206 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.427472 22764 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:18.428650 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.429838 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.430557 22759 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:18.399729 27208 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:18.442309 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.445595 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:18.446013 22757 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:18.444283 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:18.448557 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.449585 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.451105 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.452360 22763 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:18.453179 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:18.461335 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.463083 22730 leader_election.cc:400] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11.
I20260709 13:13:18.463492 22730 leader_election.cc:403] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:18.464253 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.464816 22760 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:18.465304 22839 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:18.465750 23466 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:13:18.466220 22759 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:18.466919 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.469538 22765 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:18.470777 22765 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:13:18.471614 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.472860 22913 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:18.472446 27187 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:18.467665 27091 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.478489 27091 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.480332 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.480718 27091 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:18.481809 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.482427 26948 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:18.482811 26948 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.483412 22835 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:18.483664 26948 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.484934 26948 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.475785 22766 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.486696 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:18.493176 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 60423.9us
I20260709 13:13:18.493741 22833 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:13:18.494485 22833 pending_rounds.cc:85] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:18.495003 22833 pending_rounds.cc:107] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:13:18.495486 22833 raft_consensus.cc:2889] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:18.495827 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 165410us
I20260709 13:13:18.496791 23464 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:18.506753 22914 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
W20260709 13:13:18.509259 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.514848 26933 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:18.517104 26829 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:18.519181 26979 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:18.510720 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 51407.9us
I20260709 13:13:18.535593 26933 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.541270 26933 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.542457 26933 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:18.543370 26830 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.514s	user 0.003s	sys 0.009s
I20260709 13:13:18.543897 26830 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.554633 22908 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:18.562386 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.563769 22757 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:13:18.565240 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.569842 22906 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:18.510288 22759 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:18.531550 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.502382 26930 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11.
I20260709 13:13:18.572239 22911 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:18.497790 27031 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:18.538395 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1345497920000298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 13:13:18.585036 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:18.586232 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.627936 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.629540 22757 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:18.642040 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.643163 22838 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:18.646142 22765 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:18.651262 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.122s (SENT)
I20260709 13:13:18.651806 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:18.667116 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.678619 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.684710 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.688211 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.689107 22766 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:18.689977 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:18.690629 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63921.3us
I20260709 13:13:18.690837 22765 raft_consensus.cc:2368] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10.
I20260709 13:13:18.685709 22759 raft_consensus.cc:2376] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10. Re-sending same reply.
I20260709 13:13:18.654269 27219 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:18.682665 27222 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.157s	user 0.001s	sys 0.001s
I20260709 13:13:18.642928 27172 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:13:18.703107 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.700662 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.709789 22765 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:18.713198 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66748.4us
I20260709 13:13:18.695468 26830 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:18.729938 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.731078 22766 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:18.735251 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.709569 27172 consensus_queue.cc:1243] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:18.735437 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:18.740926 22806 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.156s (SENT)
I20260709 13:13:18.741333 22765 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:13:18.741533 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:18.741557 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.748322 22765 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:18.751819 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.752997 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.753695 27220 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:18.754104 22840 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:13:18.716054 27105 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:13:18.755244 22766 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:18.813442 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.815189 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.816488 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.754161 27003 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.779596 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.818313 22833 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:13:18.775714 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.819401 22765 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:18.766597 27229 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:13:18.786135 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.820206 22835 raft_consensus.cc:2368] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:18.817278 22766 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:13:18.794291 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.821364 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.793190 22841 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:13:18.801427 22837 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:18.801759 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.822105 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:18.802103 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66575.7us
I20260709 13:13:18.787874 27095 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.810258 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.810462 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.810911 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.835052 22912 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:18.784910 27226 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:13:18.761148 22757 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:18.817955 27003 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.841312 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.841255 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.845180 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:18.847514 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:18.850790 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.850914 22765 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:18.852598 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:18.854417 22766 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:18.855747 27095 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:18.856778 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.856907 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.857061 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.857662 22835 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:18.857873 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.858075 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.858525 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:18.858610 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.217s (SENT)
I20260709 13:13:18.859198 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.859258 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.855772 22766 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:18.860277 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.860647 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:18.836814 27031 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:18.859892 22806 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.230s (SENT)
I20260709 13:13:18.859295 22835 raft_consensus.cc:2368] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11.
I20260709 13:13:18.858158 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.860044 22837 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:18.863518 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.863245 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:18.872841 22762 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:18.873689 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.874465 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.876693 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:18.877451 23457 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:18.883054 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 191214us
I20260709 13:13:18.884215 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.885075 22833 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:18.885550 22912 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:18.886191 26948 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:18.887811 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.888258 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.888566 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.889056 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.890609 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.890689 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:18.894536 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:18.895483 22837 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:18.897588 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.898242 22910 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:18.899174 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.900017 22910 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:18.902361 22806 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.363s (SENT)
I20260709 13:13:18.902825 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:18.886838 26948 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:18.906512 26948 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:18.909740 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.910434 22909 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.897189 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.912568 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:18.914569 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.915253 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.918421 26952 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.918673 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.919095 26952 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.919987 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.920579 22913 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.926329 22840 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:18.931893 23457 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:18.935884 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:18.936959 22912 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:18.917135 27009 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:13:18.939831 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:18.940347 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.942690 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:18.945154 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.371s (SENT)
I20260709 13:13:18.945765 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:18.946112 27066 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.542s	user 0.013s	sys 0.000s
I20260709 13:13:18.946672 27066 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:18.946976 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.922825 26952 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:18.947925 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.948385 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.948524 26933 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:18.948761 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.949986 27208 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:18.951252 27208 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:18.956360 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:18.913569 27127 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:18.967670 22912 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:18.971158 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.972209 23458 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:18.973675 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:18.964447 26952 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:18.978806 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:18.981261 27127 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:18.976243 26938 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:18.985566 27245 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:18.994828 27245 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.995311 27245 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.986712 27127 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:18.997448 27127 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:18.999114 27241 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:13:18.993863 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 79516.4us
I20260709 13:13:19.000851 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.004984 23457 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:19.004781 22913 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:19.007838 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.008651 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 97587.1us
I20260709 13:13:19.011096 26938 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:19.013613 26988 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:19.027712 27105 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:19.029440 27247 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.030699 26933 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:13:19.031369 27246 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:19.055789 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.055365 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.057153 23457 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:19.057231 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.029980 27245 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.060838 22730 leader_election.cc:400] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:19.061436 22730 leader_election.cc:403] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:19.051781 27003 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:19.055120 27100 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:19.069111 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.070217 23461 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:19.073637 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.078647 27187 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:19.079321 27187 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:19.081681 27247 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.082643 27247 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 9. Current term is 10.
I20260709 13:13:19.083331 22915 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:19.083468 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52517.1us
I20260709 13:13:19.085693 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.087306 27104 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:19.087812 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.088937 27104 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:19.091583 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.091820 27038 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:19.092999 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.093243 23432 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.800s (SENT)
I20260709 13:13:19.093945 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.089341 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.096326 23460 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:19.099907 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.100729 22913 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:19.098019 27095 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.102942 27038 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:19.102872 27100 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.103713 27038 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.104383 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.060874 27246 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:19.105993 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.104868 27038 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.112293 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.109359 26977 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:19.113901 27208 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.114104 23466 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:19.114459 27141 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:19.115723 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.116180 26988 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:19.116869 26988 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:19.117008 26938 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.116271 27249 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.109354 27220 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:13:19.117990 26988 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.119656 26988 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:19.122928 27245 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:19.124264 27246 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.126683 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:19.127504 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.107779 27104 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.124939 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1844) took 1004 ms. Trace:
I20260709 13:13:19.128726 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.127463 22839 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:19.130414 22763 rpcz_store.cc:276] 0709 13:13:18.121048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.196356 (+ 75308us) service_pool.cc:224] Handling call
0709 13:13:19.124713 (+928357us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400221fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:19.124839 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346670464}
W20260709 13:13:19.132354 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72271.6us
I20260709 13:13:19.135174 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:19.137880 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.139653 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.129576 27095 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.145175 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.145936 22909 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:19.148448 22910 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:19.129303 26977 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.152696 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.138212 26830 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:13:19.155272 26977 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.176469 27061 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.177099 27217 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:19.177559 27217 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:19.151015 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:19.135923 27141 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:13:19.180348 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63103.7us
W20260709 13:13:19.156759 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67506us
I20260709 13:13:19.155774 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:19.182755 22839 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:13:19.182679 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.183822 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.185844 26948 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.186404 26948 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.155733 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.146921 27249 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.195487 27249 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:19.196091 27249 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.198732 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.147174 27250 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.168737 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.212257 22835 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:19.155197 26830 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:19.142949 27167 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:19.181020 27095 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.178966 27163 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.155431 27247 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.200892 27249 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:19.202889 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1834) took 1261 ms. Trace:
I20260709 13:13:19.204579 27100 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:19.225448 22838 rpcz_store.cc:276] 0709 13:13:17.949358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.160762 (+211404us) service_pool.cc:224] Handling call
0709 13:13:19.202585 (+1041823us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540017d9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:19.202789 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057467776}
I20260709 13:13:19.227195 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:19.228257 22838 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:19.205161 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 796) took 1031 ms. Trace:
I20260709 13:13:19.228869 23464 rpcz_store.cc:276] 0709 13:13:18.173584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.173848 (+   264us) service_pool.cc:224] Handling call
0709 13:13:19.204972 (+1031124us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:19.205097 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345064320}
I20260709 13:13:19.210292 27217 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.179287 22730 leader_election.cc:400] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11.
W20260709 13:13:19.231761 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52629.2us
I20260709 13:13:19.232071 27217 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:19.201103 26933 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:19.231793 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49250.5us
I20260709 13:13:19.234597 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.215297 27240 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.459s	user 0.001s	sys 0.001s
W20260709 13:13:19.221026 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52416.1us
I20260709 13:13:19.236874 23464 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:19.238284 23457 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:13:19.214639 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 121452us
I20260709 13:13:19.239114 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:19.240018 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:19.240837 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.240635 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:19.241771 27141 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.241797 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:19.242430 27141 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.242890 27141 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.244436 27003 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:19.245276 27003 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:19.246491 27003 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.136555 27104 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:13:19.220026 27127 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:19.212301 27250 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.213407 26830 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:19.259325 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72364.5us
I20260709 13:13:19.262470 26830 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.263984 26830 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:19.249888 27061 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.265327 27061 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.258518 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.224786 26979 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.248648 27167 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.259864 27250 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.257411 27255 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:19.309303 27167 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.313446 27095 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:19.315989 27231 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.571s	user 0.003s	sys 0.000s
I20260709 13:13:19.299422 27165 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:19.321909 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.300058 26979 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.319628 27247 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.321820 27160 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.310792 27163 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:13:19.344200 27217 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:19.344995 27217 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:19.348006 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 108998us
I20260709 13:13:19.310994 26978 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:13:19.330013 27261 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:13:19.349476 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 114987us
I20260709 13:13:19.351997 22834 pending_rounds.cc:85] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:19.352330 22834 pending_rounds.cc:107] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:19.352586 22834 raft_consensus.cc:2889] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:19.351320 27261 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:19.355371 27261 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.356997 27261 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.357760 26948 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:19.354225 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.365769 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.367682 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:19.368080 27216 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.891s	user 0.000s	sys 0.003s
I20260709 13:13:19.368614 22763 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 7: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:19.369001 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 544) took 1108 ms. Trace:
I20260709 13:13:19.369453 22908 rpcz_store.cc:276] 0709 13:13:18.260481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.261618 (+  1137us) service_pool.cc:224] Handling call
0709 13:13:19.368813 (+1107195us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b5400041854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:19.368941 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177548160}
I20260709 13:13:19.372819 22914 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:19.373025 22839 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:19.353371 27160 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:19.374459 27160 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.388859 22763 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:19.391460 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:19.392143 22759 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:19.389541 27255 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:13:19.395121 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:19.390628 27091 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.396744 27255 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:19.403366 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.404299 22839 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:19.360555 27271 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.407178 27223 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:13:19.425992 22759 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:19.421883 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.429620 22914 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:19.429093 26830 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.431254 22763 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:19.431649 22839 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:13:19.432047 22839 pending_rounds.cc:85] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 8
I20260709 13:13:19.432286 22839 pending_rounds.cc:107] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:13:19.432637 22839 raft_consensus.cc:2889] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:19.446756 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43498.4us
I20260709 13:13:19.449724 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1837) took 1008 ms. Trace:
W20260709 13:13:19.451678 22806 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.549s (SENT)
I20260709 13:13:19.413658 27091 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:13:19.453266 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.450572 22832 rpcz_store.cc:276] 0709 13:13:18.442312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.481570 (+ 39258us) service_pool.cc:224] Handling call
0709 13:13:18.482001 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.482288 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:18.636087 (+153799us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.432885 (+796798us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b1800420b80. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:13:19.448680 (+ 15795us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.449318 (+   638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:19.449377 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:19.449434 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.449531 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.449658 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45331,"prepare.queue_time_us":801738,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":1834,"raft.queue_time_us":9668,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":34052,"spinlock_wait_cycles":7820416,"thread_start_us":168758,"threads_started":2}
I20260709 13:13:19.463783 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:19.464581 22763 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:19.473429 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.475308 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.476758 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.477025 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.445122 27225 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.479532 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:19.481134 22839 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:19.450759 27031 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.485479 22763 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:19.486836 26830 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.418825 26979 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:19.496891 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45271.3us
I20260709 13:13:19.435257 27271 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.508111 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.508860 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.503799 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 547) took 1390 ms. Trace:
I20260709 13:13:19.504628 22760 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:19.499213 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2642) took 1084 ms. Trace:
I20260709 13:13:19.515656 22911 rpcz_store.cc:276] 0709 13:13:18.414451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.414732 (+   281us) service_pool.cc:224] Handling call
0709 13:13:18.570897 (+156165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.583650 (+ 12753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.498976 (+915326us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54002111d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:19.499147 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312435584}
I20260709 13:13:19.511058 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.504179 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.458397 27217 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:19.470672 27213 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.522209 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:19.479498 27160 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.478188 27247 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.507387 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:19.542485 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.510627 22761 rpcz_store.cc:276] 0709 13:13:18.113605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.113794 (+   189us) service_pool.cc:224] Handling call
0709 13:13:18.160973 (+ 47179us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:18.221862 (+ 60889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.221899 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:18.222753 (+   854us) log.cc:844] Serialized 52 byte log entry
0709 13:13:18.431744 (+208991us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:18.447840 (+ 16096us) log.cc:844] Serialized 14 byte log entry
0709 13:13:18.495449 (+ 47609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.495535 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.212829 (+717294us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b1800377da0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:19.503390 (+290561us) raft_consensus.cc:1685] finished
0709 13:13:19.503590 (+   200us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.503733 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53163,"prepare.queue_time_us":16655,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5496,"raft.queue_time_us":74192,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":20054,"spinlock_wait_cycles":19023104,"thread_start_us":208579,"threads_started":1,"wal-append.queue_time_us":295289}
I20260709 13:13:19.523351 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:19.525166 27251 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.465s	user 0.000s	sys 0.002s
I20260709 13:13:19.531008 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:19.518424 26830 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:19.521405 27104 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.521996 27257 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:19.497771 27270 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.529700 27091 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.504067 27271 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:19.540230 26948 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.551474 27066 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.549484 27264 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:19.541867 27003 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.577737 27003 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.581823 27254 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:19.582422 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.569010 27160 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.584666 27172 raft_consensus.cc:2764] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:19.588104 27163 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:19.589399 27172 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:19.589862 27172 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.590509 27264 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:19.590927 27172 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.591603 27264 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.592514 26933 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.592918 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.593078 26933 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.543659 27235 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:13:19.565306 26978 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:19.594970 26978 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.595965 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:19.597244 27095 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.598441 27095 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.597630 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.598876 27095 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.599503 27061 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.600294 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1835) took 1638 ms. Trace:
I20260709 13:13:19.600852 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.600687 22836 rpcz_store.cc:276] 0709 13:13:17.962601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.199376 (+236775us) service_pool.cc:224] Handling call
0709 13:13:18.370398 (+171022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.370887 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.370929 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.378220 (+  7291us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.581704 (+203484us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:18.644474 (+ 62770us) log.cc:844] Serialized 14 byte log entry
0709 13:13:18.645867 (+  1393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.645955 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:18.896069 (+250114us) raft_consensus.cc:1685] finished
0709 13:13:18.896096 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.600118 (+704022us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540028a0d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:19.600222 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":732092,"prepare.queue_time_us":59658,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":2201,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":28949,"spinlock_wait_cycles":294675840,"thread_start_us":124382,"threads_started":1,"wal-append.queue_time_us":226295}
I20260709 13:13:19.603474 26988 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.604302 26988 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.596654 26933 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:19.605997 26933 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.606647 27225 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:19.568234 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.619246 27066 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:19.620581 27172 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:19.542263 27219 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.624264 27219 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.540903 27246 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:13:19.563784 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 216976us
I20260709 13:13:19.626760 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.627214 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.627789 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.594379 26952 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.603691 27061 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.630755 27061 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:19.631286 27061 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.607511 26933 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.608155 26948 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.632229 27061 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.633458 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:19.634085 26948 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:19.559846 26830 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:19.638216 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:19.638466 26830 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.638940 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.629814 26952 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.640055 26952 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.640362 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.596489 26978 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.641798 27233 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:19.567108 27109 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:19.643620 27233 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:19.643734 27109 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.644351 27233 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.525619 27103 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:13:19.645630 27254 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:19.646569 27109 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:19.566392 27253 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:19.652736 27253 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:19.654691 22836 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:13:19.655378 22836 pending_rounds.cc:85] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:19.655830 22836 pending_rounds.cc:107] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:19.656139 22836 raft_consensus.cc:2889] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:19.655913 27253 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.620545 27066 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.657608 27253 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:19.658151 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:19.615196 22834 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:13:19.615664 27232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.009s	sys 0.005s
W20260709 13:13:19.663986 27232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.009s	sys 0.005s
I20260709 13:13:19.615456 22841 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:19.629204 27219 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:19.647380 27187 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 13:13:19.667171 27187 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.667568 27003 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:19.667943 22766 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:13:19.668413 23461 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:19.647491 22758 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:19.648310 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:19.651482 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:19.651685 26978 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.628996 27164 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:19.670969 27164 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:19.671820 26933 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:19.672124 27164 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:19.658217 27141 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:19.597420 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.652593 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:19.674121 22764 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:19.665957 22912 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:13:19.674731 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50877.7us
I20260709 13:13:19.612556 22763 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:13:19.677022 22763 pending_rounds.cc:85] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:13:19.671842 22761 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:19.677397 22763 pending_rounds.cc:107] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:13:19.679188 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.680819 23461 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:19.669926 27095 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:19.681471 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.682251 22763 raft_consensus.cc:2889] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:19.684849 27141 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:19.696519 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48899.7us
I20260709 13:13:19.689702 22760 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:19.691892 22914 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:13:19.692786 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:19.605178 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:19.699051 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.699362 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 94279.1us
I20260709 13:13:19.703876 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:19.750097 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.750664 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.751060 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:19.753427 22906 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
W20260709 13:13:19.755321 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.751930 22914 raft_consensus.cc:2376] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13. Re-sending same reply.
W20260709 13:13:19.755535 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55077.9us
W20260709 13:13:19.757011 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74006.8us
I20260709 13:13:19.763119 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:19.764097 22914 raft_consensus.cc:2376] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13. Re-sending same reply.
I20260709 13:13:19.766562 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:19.772730 22834 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:19.696602 27187 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:19.709088 27164 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.757478 27246 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.773058 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:19.785754 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.756825 26977 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:19.807870 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:19.822904 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 141568us
I20260709 13:13:19.828506 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.830933 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:19.844195 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.863032 22764 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:19.855084 22758 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:19.820180 27249 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 13:13:19.865932 22761 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:19.869956 22760 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:13:19.871012 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.871251 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.889998 22909 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:19.765228 27038 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:13:19.896731 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.787472 27141 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:13:19.876523 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.827168 27219 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.877043 27157 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.831820 26933 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.905550 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:19.871840 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:19.854733 27165 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:19.891237 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:19.925000 22915 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.825939 27103 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:13:19.885443 27235 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:19.930289 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53856.1us
I20260709 13:13:19.887663 27272 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.938088 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:19.853204 27187 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:13:19.942071 23461 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:19.904330 27255 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.910817 27219 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.883313 27225 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:13:19.958685 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 86904.2us
I20260709 13:13:19.917315 27157 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.961270 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:19.964316 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.900164 27249 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:19.971972 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96716.2us
I20260709 13:13:19.949257 27246 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:19.972591 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 147322us
I20260709 13:13:19.968634 26933 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:19.981841 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.956750 27217 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:19.965482 27219 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.960533 27235 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:19.970752 27157 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.965337 27247 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.997164 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:19.978237 26948 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.972090 27141 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.015544 22762 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:19.976890 26988 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:20.252647 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 232558us
I20260709 13:13:20.253990 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:19.983403 27217 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:20.045884 27220 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:20.062628 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.064157 27283 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 13:13:20.069662 27281 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
W20260709 13:13:20.070179 27285 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:13:20.048444 27095 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:20.078329 27290 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:20.015040 27246 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:19.993598 27187 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:13:20.169234 27093 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:20.170205 27225 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.081974 26933 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:20.170881 27250 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.852s	user 0.015s	sys 0.000s
W20260709 13:13:20.180387 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.993219 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 469949us
I20260709 13:13:20.061776 27235 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.214811 27287 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.999989 27255 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:13:20.055562 27165 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.224249 22915 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
W20260709 13:13:20.224728 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.324429 22838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.008s	sys 0.026s
W20260709 13:13:20.324821 22838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.008s	sys 0.026s
W20260709 13:13:20.224880 26978 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.004s
I20260709 13:13:19.989979 27232 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:13:20.224440 27136 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:20.228459 22763 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:13:20.231218 22833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.010s	sys 0.016s
W20260709 13:13:20.416734 22833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.010s	sys 0.016s
I20260709 13:13:20.417716 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 656) took 1527 ms. Trace:
I20260709 13:13:20.418254 22833 rpcz_store.cc:276] 0709 13:13:18.890676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.890882 (+   206us) service_pool.cc:224] Handling call
0709 13:13:18.910164 (+ 19282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.910428 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:19.688822 (+778394us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.417275 (+728453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:20.417361 (+    86us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:20.417414 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.417493 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.417648 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":559299,"prepare.queue_time_us":590457,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"thread_start_us":946791,"threads_started":2}
I20260709 13:13:20.015461 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:20.265792 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 270984us
W20260709 13:13:20.172099 27253 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.006s
I20260709 13:13:20.270349 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:20.274742 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 497071us
W20260709 13:13:20.259748 26952 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.613s	user 0.016s	sys 0.006s
I20260709 13:13:20.227984 27219 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:20.283270 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.260294 27257 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.017s
W20260709 13:13:20.285430 27289 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
W20260709 13:13:20.291678 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.264614 27220 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.257424 26948 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.263064 27061 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:13:20.330160 26977 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:20.258224 27141 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.492342 23461 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:13:20.303588 27252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.007s	sys 0.007s
W20260709 13:13:20.377746 27285 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:13:20.451122 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 435738us
W20260709 13:13:20.454366 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 184170us
I20260709 13:13:20.482272 22915 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:20.491194 22906 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:20.303890 27225 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.318470 27250 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.425387 27264 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:13:20.322934 27287 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.513935 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:20.513703 22909 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:20.585772 22832 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:13:20.513932 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:20.708276 22832 pending_rounds.cc:85] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:13:20.562966 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:20.708957 22832 pending_rounds.cc:107] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:13:20.709477 22832 raft_consensus.cc:2889] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:20.535778 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.709440 22835 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:20.716392 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 552) took 1027 ms. Trace:
W20260709 13:13:20.701959 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 224555us
W20260709 13:13:20.715207 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.483264 27093 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:20.417739 27136 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:20.469260 27220 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.783149 23461 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:20.478327 27257 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:20.478597 27298 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:20.802655 27298 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.470042 26952 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.803331 27298 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:20.533615 26979 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.262s
I20260709 13:13:20.807279 22911 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 11. Ops: []
I20260709 13:13:20.810487 27296 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.812034 27250 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.813455 22759 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:20.812621 27296 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.814682 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 553) took 1123 ms. Trace:
I20260709 13:13:20.815047 22911 rpcz_store.cc:276] 0709 13:13:19.690856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.690978 (+   122us) service_pool.cc:224] Handling call
0709 13:13:19.760695 (+ 69717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:20.812230 (+1051535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.814397 (+  2167us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000fc154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:20.814611 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406419712}
W20260709 13:13:20.788548 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.722s (SENT)
I20260709 13:13:20.746953 22910 rpcz_store.cc:276] 0709 13:13:19.689423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.690166 (+   743us) service_pool.cc:224] Handling call
0709 13:13:19.690516 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.768429 (+ 77913us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:19.793641 (+ 25212us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.264383 (+470742us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:20.273637 (+  9254us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.274408 (+   771us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:20.274776 (+   368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.274876 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.716125 (+441249us) raft_consensus.cc:1685] finished
0709 13:13:20.716155 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.716315 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65954,"prepare.queue_time_us":24915,"raft.queue_time_us":60881,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":38795,"spinlock_wait_cycles":27268352,"thread_start_us":384364,"threads_started":1,"wal-append.queue_time_us":804344}
I20260709 13:13:20.827312 26948 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.828433 27296 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.829679 26952 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:20.830307 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 682) took 1312 ms. Trace:
I20260709 13:13:20.831666 27141 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.830717 23458 rpcz_store.cc:276] 0709 13:13:19.518387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.518533 (+   146us) service_pool.cc:224] Handling call
0709 13:13:19.539694 (+ 21161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.564164 (+ 24470us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:19.564498 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.758228 (+193730us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.778230 (+ 20002us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.778393 (+   163us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:19.778445 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.778544 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.830062 (+1051518us) raft_consensus.cc:1685] finished
0709 13:13:20.830087 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.830239 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1446,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":5041,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":12756,"thread_start_us":193303,"threads_started":1,"wal-append.queue_time_us":337708}
W20260709 13:13:20.645242 27286 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
W20260709 13:13:20.574461 27261 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.215s	user 0.021s	sys 0.000s
W20260709 13:13:20.654482 27278 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:13:20.885025 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133544us
W20260709 13:13:20.712095 27252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.010s	sys 0.028s
I20260709 13:13:20.579058 26930 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:20.853842 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104390us
I20260709 13:13:20.904996 22911 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:20.534379 27003 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.650132 27031 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:20.887315 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:20.939011 22841 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:20.890012 22835 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:20.646991 27219 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.716732 27164 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:20.717121 27172 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.086s	user 0.006s	sys 0.016s
I20260709 13:13:20.894515 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:20.903311 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.763s (SENT)
I20260709 13:13:20.610164 27304 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.853176 27167 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:13:20.493057 27272 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.376s
I20260709 13:13:20.877566 27298 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.641567 27284 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.557s
I20260709 13:13:20.868237 27160 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.867781 27213 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:20.958621 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.013s	sys 0.041s
I20260709 13:13:20.832403 27294 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 13:13:20.414409 27217 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:20.967469 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.943243 27003 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.560352 27246 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.901669 27038 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:20.832549 27093 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:20.904783 26930 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:20.979887 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:21.068367 22911 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:20.990056 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:20.990696 22908 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:20.653122 26977 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.114714 27317 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.115327 27317 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.118181 27317 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.805040 27136 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.120945 27136 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:20.941900 27249 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:20.595227 27187 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.295s
I20260709 13:13:20.976280 27235 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.130191 27235 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.131178 27308 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:21.131978 27308 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.132611 27255 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:21.133113 27255 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:20.928996 27293 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.490s
I20260709 13:13:20.989745 27225 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:21.045800 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.013s	sys 0.042s
W20260709 13:13:21.168697 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 463369us
W20260709 13:13:21.169298 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.899695 27250 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.049481 22834 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:13:21.057181 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.006307 27219 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:20.977478 27031 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:20.983060 27164 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.024881 27163 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:21.061693 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 158373us
I20260709 13:13:21.066633 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.076218 26988 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:21.057467 27213 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.079223 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.090490 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 563) took 1219 ms. Trace:
I20260709 13:13:21.094424 27093 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.092146 22911 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:21.097137 26930 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:21.098186 27307 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:20.979663 27304 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:21.099444 27310 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:13:21.099866 22841 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:21.110389 27003 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.112779 27298 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.042323 27160 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:21.020915 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.030s	sys 0.017s
W20260709 13:13:21.243214 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.031s	sys 0.017s
I20260709 13:13:21.133265 27235 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:21.187655 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.133651 27313 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.217773 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:21.137755 27318 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 13:13:21.139390 27315 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:21.140575 26830 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:21.142956 27301 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:13:21.143419 27217 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:21.143476 22759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1866) took 1496 ms (client timeout 1515 ms). Trace:
I20260709 13:13:21.149708 27038 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.150990 27294 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.151991 27287 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.094473 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 668) took 1392 ms. Trace:
I20260709 13:13:21.155220 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.361171 22762 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:21.362879 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.156263 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.164923 22760 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:13:21.065070 27157 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.010s	user 0.017s	sys 0.000s
I20260709 13:13:21.187644 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:21.190193 22832 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:13:21.381546 22832 pending_rounds.cc:85] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:13:21.381976 22832 pending_rounds.cc:107] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:13:21.382320 22832 raft_consensus.cc:2889] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.193162 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:21.194197 23461 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:21.200861 22837 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:21.201223 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.169380 27141 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:21.223815 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.059s	sys 0.108s
I20260709 13:13:21.188336 22764 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:21.195446 27255 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.192068 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.194599 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:21.396256 23466 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:21.205646 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 555) took 1513 ms. Trace:
W20260709 13:13:21.396749 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 691385us
I20260709 13:13:21.397115 22908 rpcz_store.cc:276] 0709 13:13:19.692443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.692563 (+   120us) service_pool.cc:224] Handling call
0709 13:13:21.205319 (+1512756us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:21.205562 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2971245440}
W20260709 13:13:21.225637 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.187961 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.188048 27311 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.191363 27247 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.415125 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 665) took 1789 ms. Trace:
I20260709 13:13:21.414619 27247 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.416448 27272 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:21.233040 27306 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:21.415735 22838 rpcz_store.cc:276] 0709 13:13:19.625454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.627876 (+  2422us) service_pool.cc:224] Handling call
0709 13:13:19.628225 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:19.685517 (+ 57292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.685556 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:19.686261 (+   705us) log.cc:844] Serialized 52 byte log entry
0709 13:13:20.325193 (+638932us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:20.408616 (+ 83423us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.413438 (+  4822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.413559 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.649724 (+236165us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b18003d1440. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:21.414568 (+764844us) raft_consensus.cc:1685] finished
0709 13:13:21.414679 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.415017 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103191,"prepare.queue_time_us":90179,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":59767,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":218,"spinlock_wait_cycles":80333056,"thread_start_us":551425,"threads_started":1,"wal-append.queue_time_us":1165672}
I20260709 13:13:21.417800 27306 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:21.188315 27250 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.419080 27324 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.188954 27319 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.419665 27324 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.420734 27250 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.324856 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.421847 27213 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.421895 23457 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:21.208449 22765 rpcz_store.cc:276] 0709 13:13:19.871308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.871593 (+   285us) service_pool.cc:224] Handling call
0709 13:13:19.871933 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.875171 (+  3238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.875222 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:20.424815 (+549593us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b18002251c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a
0709 13:13:20.425415 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.889612 (+464197us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:20.915001 (+ 25389us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.915304 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.915393 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.090073 (+174680us) raft_consensus.cc:1685] finished
0709 13:13:21.090211 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.090404 (+   193us) inbound_call.cc:177] Queueing success response
0709 13:13:21.201920 (+111516us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b18002251c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
Metrics: {"mutex_wait_us":7313,"prepare.queue_time_us":39285,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":15464,"raft.queue_time_us":3879,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":9289,"spinlock_wait_cycles":28230528,"thread_start_us":421016,"threads_started":1,"wal-append.queue_time_us":619411}
W20260709 13:13:21.322463 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 105088us
I20260709 13:13:21.268766 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:21.329337 22759 rpcz_store.cc:269] 0709 13:13:19.646598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.647976 (+  1378us) service_pool.cc:224] Handling call
0709 13:13:21.143190 (+1495214us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003403d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:21.143370 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2627762944}
I20260709 13:13:21.417557 27272 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:21.429534 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.360903 22836 rpcz_store.cc:276] 0709 13:13:19.702167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.702815 (+   648us) service_pool.cc:224] Handling call
0709 13:13:19.749716 (+ 46901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.750097 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.750131 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.750700 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.271971 (+521271us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:20.272025 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.272119 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.094025 (+821906us) raft_consensus.cc:1685] finished
0709 13:13:21.094218 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.094404 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9225,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":12129,"thread_start_us":465699,"threads_started":1,"wal-append.queue_time_us":1318593}
I20260709 13:13:21.341445 26830 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.431733 27225 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:13:21.432464 27322 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.433257 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.433949 22907 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:21.434624 26830 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.370638 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 562) took 1499 ms. Trace:
I20260709 13:13:21.435393 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.436226 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 558) took 1654 ms. Trace:
I20260709 13:13:21.436844 22761 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:21.371794 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:21.435485 22766 rpcz_store.cc:276] 0709 13:13:19.870888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.871513 (+   625us) service_pool.cc:224] Handling call
0709 13:13:19.891065 (+ 19552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.902378 (+ 11313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.902433 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.909880 (+  7447us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.286717 (+376837us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:20.286778 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.286872 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.370244 (+1083372us) raft_consensus.cc:1685] finished
0709 13:13:21.370274 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.370533 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9894,"raft.queue_time_us":18654,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":17537,"thread_start_us":325840,"threads_started":1,"wal-append.queue_time_us":876289}
I20260709 13:13:21.379979 27294 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.371765 27038 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.438866 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.373703 27160 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.373715 27308 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.440184 27294 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.406060 27157 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:21.442600 27315 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:13:21.444128 27157 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.436609 22914 rpcz_store.cc:276] 0709 13:13:19.782554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.782724 (+   170us) service_pool.cc:224] Handling call
0709 13:13:19.783249 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:20.220495 (+437246us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b48000b0c58. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:13:20.227118 (+  6623us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b1800201d00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:13:20.228898 (+  1780us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:20.524091 (+295193us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.022283 (+498192us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.022953 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:13:21.023147 (+   194us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:21.023191 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.023379 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.435994 (+412615us) raft_consensus.cc:1685] finished
0709 13:13:21.436017 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.436159 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17484,"prepare.queue_time_us":393512,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":241753,"raft.run_cpu_time_us":2985,"raft.run_wall_time_us":315597,"spinlock_wait_cycles":25247872,"thread_start_us":728496,"threads_started":3,"wal-append.queue_time_us":873655}
I20260709 13:13:21.445271 23462 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:21.202320 27261 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.204281 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.446151 27261 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.354732 27187 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:21.191324 27164 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.230309 26930 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.320689 27284 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.193447 27317 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.193315 27249 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.413663 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.414844 27311 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.419802 27319 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:21.253072 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 282393us
I20260709 13:13:21.423195 27247 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.251690 27304 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.365732 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:21.426913 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:21.176862 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.426779 23458 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:21.428738 22757 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:21.336236 27003 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:21.430142 27272 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:21.518970 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.498402 27320 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.520731 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.521481 23465 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:21.521246 27320 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.523084 22841 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:21.523291 27261 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.523751 27320 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.515190 27311 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.484485 27322 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.498764 26830 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.532342 26830 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.501741 27249 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.536412 26830 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.536671 27249 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.540920 27324 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.385444 22839 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:21.378710 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.547209 22758 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:13:21.548949 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.189463 27031 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.187588 27219 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.486799 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:21.553241 22832 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:21.489459 22764 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:21.555195 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.556066 22764 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:21.490311 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:21.496042 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.358606 27313 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.424536 27141 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.515285 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.517207 27319 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.436754 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.589006 22759 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:21.575240 22760 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:21.513988 27327 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.559582 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:21.562640 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 72438us
I20260709 13:13:21.591032 23464 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:21.568032 22832 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:13:21.584702 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.592432 23465 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:13:21.437984 22762 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:21.594312 22832 pending_rounds.cc:85] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 7
I20260709 13:13:21.594826 22832 pending_rounds.cc:107] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:13:21.595268 22832 raft_consensus.cc:2889] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.489318 26930 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.615157 22837 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:13:21.615275 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.615867 22837 pending_rounds.cc:85] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
I20260709 13:13:21.616179 22837 pending_rounds.cc:107] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:13:21.616430 22837 raft_consensus.cc:2889] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.616842 22759 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:13:21.617224 22759 pending_rounds.cc:85] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 5
I20260709 13:13:21.617441 22759 pending_rounds.cc:107] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:13:21.617702 22759 raft_consensus.cc:2889] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.617873 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:21.618748 22907 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:21.620991 22914 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:21.622539 27321 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:21.623296 27321 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.623951 27306 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.627337 27272 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.628504 27272 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:21.630841 27141 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.599151 27105 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:21.634456 27105 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.635809 27219 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.636758 27327 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.638419 27235 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.603982 22834 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:21.605111 22760 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:21.650418 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:21.632977 27141 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.551489 27322 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:21.639859 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:21.574328 27304 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.645416 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.556526 27255 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:21.549273 27247 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.468214 27187 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.651646 22760 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:21.494750 27284 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.611850 22908 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:21.680298 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.640612 27320 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:21.559900 27061 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:21.572237 27323 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.659957 23464 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:21.656018 27105 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.598862 27324 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:21.583053 22906 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.003s	sys 0.007s
W20260709 13:13:21.733175 22906 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.004s	sys 0.008s
I20260709 13:13:21.722465 22834 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:21.666121 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.668210 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:21.669003 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55210us
I20260709 13:13:21.573688 27160 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.501895 27317 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.652266 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:21.714677 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:21.738579 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 75223.5us
I20260709 13:13:21.739291 22832 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:21.739209 22908 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:21.696653 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.721297 22760 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:21.663386 22758 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:21.725267 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.730985 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:21.712142 26977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.019s	sys 0.047s
W20260709 13:13:21.757081 26977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.019s	sys 0.047s
I20260709 13:13:21.667075 27219 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.745836 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:21.746358 27317 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.746835 22836 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:21.746966 27306 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:21.761204 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.675493 27247 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.765137 27247 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.749822 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.766141 27213 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:21.766274 22911 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:21.766923 27213 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.768167 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:21.769745 27213 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.774752 22763 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 13:13:21.776475 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.786664 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:21.750036 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.750590 27031 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:21.750706 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.749914 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:21.753890 27293 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:21.755419 26978 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:21.756466 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:21.805418 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49054us
I20260709 13:13:21.761056 27219 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:21.762205 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 811) took 1823 ms (client timeout 1895 ms). Trace:
W20260709 13:13:21.811072 23462 rpcz_store.cc:269] 0709 13:13:19.938856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.939189 (+   333us) service_pool.cc:224] Handling call
0709 13:13:21.761690 (+1822501us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400603454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:21.762067 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3400361856}
I20260709 13:13:21.747556 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.812332 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:21.813740 22762 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:21.655421 27235 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.807680 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.810072 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.810153 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:21.683522 27249 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:21.683283 27261 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:13:21.732846 27324 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.819756 27327 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.814644 27247 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.817137 27297 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:21.825080 22806 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.009s	sys 0.013s
I20260709 13:13:21.681635 27141 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.684273 27304 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:21.914180 22806 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.012s	sys 0.014s
W20260709 13:13:21.916563 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 725855us
I20260709 13:13:21.821141 27172 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:13:21.929219 23465 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:21.733302 27323 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.851966 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:21.805945 27031 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:21.806582 27213 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:21.729799 27320 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:21.806133 27318 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.733479 27061 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:21.828590 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:21.866547 22834 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:13:21.745087 27160 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:21.823787 26933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.005s	sys 0.017s
I20260709 13:13:21.814406 27294 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:21.866268 22833 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
W20260709 13:13:21.800134 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:21.957042 27217 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.001s
I20260709 13:13:21.891548 27324 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.000061 27327 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:22.001153 27327 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.003834 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2668) took 1108 ms. Trace:
I20260709 13:13:22.006017 27109 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.006608 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:22.007503 22763 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:22.006933 27109 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.008596 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:22.000151 27187 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:13:22.012310 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:22.013675 27298 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:22.014169 27298 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.014571 27298 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.015306 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.016405 22841 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:22.004760 22906 rpcz_store.cc:276] 0709 13:13:20.895148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.906756 (+ 11608us) service_pool.cc:224] Handling call
0709 13:13:20.907399 (+   643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:20.953234 (+ 45835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:20.953284 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.003266 (+ 49982us) log.cc:844] Serialized 26 byte log entry
0709 13:13:21.733786 (+730520us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:21.733867 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.733951 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.003511 (+269560us) raft_consensus.cc:1685] finished
0709 13:13:22.003547 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.003742 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29404,"raft.queue_time_us":71479,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9428,"thread_start_us":672979,"threads_started":1,"wal-append.queue_time_us":895404}
I20260709 13:13:22.019754 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:22.023995 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.058s (SENT) [suppressed 15 similar messages]
W20260709 13:13:22.029414 23466 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.000s
I20260709 13:13:21.825281 26952 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:13:21.821550 26977 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:13:21.817203 27328 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:13:22.088259 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:21.946180 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 117741us
W20260709 13:13:21.937392 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 85521.5us
I20260709 13:13:22.090287 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:22.089284 22841 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:22.090708 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:21.865473 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:21.941288 22837 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 7.
I20260709 13:13:21.941140 27031 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:21.953686 27304 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.954348 27329 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.959757 27141 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.960868 27321 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.963440 27160 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:21.964812 27313 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.965714 26930 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.966499 22908 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:22.096774 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.093717 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:22.097855 22841 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:21.967978 27323 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.970006 22910 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:21.842197 22835 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.010s	sys 0.013s
W20260709 13:13:22.100998 22835 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.010s	sys 0.013s
I20260709 13:13:21.981937 22833 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:13:21.942759 27136 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.003s
I20260709 13:13:21.985428 27294 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:21.989631 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:22.109673 23463 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:13:22.111800 22760 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:13:22.113344 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.115190 22757 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:21.843447 27308 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:13:21.915740 27247 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.118278 22841 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:13:22.118762 27247 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.119059 22841 pending_rounds.cc:85] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
W20260709 13:13:21.973297 26933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.006s	sys 0.018s
I20260709 13:13:22.119642 22841 pending_rounds.cc:107] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:13:22.120114 22841 raft_consensus.cc:2889] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.946381 27255 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:22.051120 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.121112 22832 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:22.121495 27163 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:21.972865 27233 raft_consensus.cc:2764] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:13:22.122232 27061 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:22.121943 27163 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:22.123481 27235 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:22.060170 22912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.034s	sys 0.087s
I20260709 13:13:22.123536 27163 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.124312 22912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.035s	sys 0.088s
I20260709 13:13:22.124106 27329 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.125125 27163 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.126962 27329 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.128072 26930 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:22.128602 26930 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.129027 26930 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:22.131431 27187 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:22.135356 27320 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:21.971536 27318 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.030573 23466 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:13:22.084764 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 140823us
I20260709 13:13:22.087672 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.140764 22841 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:22.102775 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:21.941363 27297 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.103283 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.106535 27327 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.108705 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:21.947918 27213 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.154239 22836 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:13:21.894405 27249 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.110008 27294 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.121876 26948 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:22.059897 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.138964 27318 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.102653 22761 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:22.156265 22840 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:22.143476 22760 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:22.144882 22762 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
W20260709 13:13:22.157016 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54339.6us
I20260709 13:13:22.161603 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:22.147032 27309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.008s	sys 0.014s
I20260709 13:13:22.159395 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:22.207890 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:22.204604 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:22.147295 27321 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:22.221786 22910 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:13:22.223603 22910 pending_rounds.cc:85] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 7
I20260709 13:13:22.223899 22910 pending_rounds.cc:107] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:13:22.224275 22910 raft_consensus.cc:2889] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:22.224628 22910 pending_rounds.cc:107] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:13:22.224877 22910 raft_consensus.cc:2889] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:22.222266 22907 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:13:22.222942 22762 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:22.228303 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:22.208510 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49257.4us
I20260709 13:13:22.230593 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:22.229476 22762 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:22.236312 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.153911 27255 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:22.237457 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.240516 22909 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:22.241979 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.172838 26948 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:22.207847 27309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.009s	sys 0.014s
I20260709 13:13:22.259135 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:22.177457 26830 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:22.206281 27327 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:22.243451 27165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.012s	sys 0.023s
I20260709 13:13:22.228713 27321 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.236635 27284 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.209934 27318 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:22.227753 27109 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:22.229020 27329 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:22.266614 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:22.271421 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:22.290462 22764 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.003s	sys 0.014s
W20260709 13:13:22.291049 22764 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.003s	sys 0.014s
W20260709 13:13:22.291674 22759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.011s	sys 0.008s
W20260709 13:13:22.292043 22759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.011s	sys 0.008s
W20260709 13:13:22.292954 23457 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.009s	sys 0.009s
W20260709 13:13:22.293267 23457 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.009s	sys 0.009s
I20260709 13:13:22.294838 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:22.301578 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.010s	sys 0.008s
W20260709 13:13:22.302219 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.010s	sys 0.008s
W20260709 13:13:22.302634 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 620255us
I20260709 13:13:22.241250 27225 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:22.228561 27320 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:22.270342 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:22.236238 27246 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.000s	sys 0.024s
I20260709 13:13:22.275555 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.282550 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:22.277971 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:22.292790 27003 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.004s	sys 0.018s
I20260709 13:13:22.346351 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:22.370872 27003 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.004s	sys 0.018s
I20260709 13:13:22.371328 27003 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:22.371497 22909 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:22.291498 27066 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.003s	sys 0.014s
W20260709 13:13:22.348505 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44682.7us
W20260709 13:13:22.370911 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83749.3us
W20260709 13:13:22.350891 22839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.000s	sys 0.025s
I20260709 13:13:22.324971 22908 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
W20260709 13:13:22.300894 27095 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.010s	sys 0.007s
I20260709 13:13:22.282739 27321 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:22.273597 27219 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.358541 22761 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:13:22.341944 27259 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.370020 22910 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:13:22.375057 22839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.000s	sys 0.025s
I20260709 13:13:22.345681 27320 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:22.278383 27220 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.435s	user 0.016s	sys 0.003s
I20260709 13:13:22.300732 27304 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.268703 26830 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:22.382318 27095 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.011s	sys 0.007s
W20260709 13:13:22.282783 27165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.012s	sys 0.023s
W20260709 13:13:22.364027 27246 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.001s	sys 0.026s
I20260709 13:13:22.372650 27003 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.270990 27187 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:22.381264 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5850485639998624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 13:13:22.450755 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5445485639999106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 13:13:22.451243 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3243559699999423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
W20260709 13:13:22.415405 22765 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.009s	sys 0.017s
W20260709 13:13:22.373517 27066 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.003s	sys 0.015s
W20260709 13:13:22.452184 22765 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.009s	sys 0.017s
W20260709 13:13:22.354368 27317 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.014s	sys 0.010s
I20260709 13:13:22.273382 27327 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:22.305785 27141 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 13:13:22.293939 27318 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.420022 27320 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:22.414788 27093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.017s	sys 0.009s
W20260709 13:13:22.476318 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.007s	sys 0.013s
I20260709 13:13:22.424628 26830 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:22.470441 27317 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.014s	sys 0.011s
I20260709 13:13:22.362641 27329 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.487314 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.008s	sys 0.013s
I20260709 13:13:22.449167 27324 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.498560 22907 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
W20260709 13:13:22.499778 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.400786 27255 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.469054 27284 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:22.496513 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 677565us
W20260709 13:13:22.503728 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 479921us
I20260709 13:13:22.505591 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:22.448727 27304 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:22.509677 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2663) took 2637 ms (client timeout 2209 ms). Trace:
I20260709 13:13:22.422793 27259 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:22.492460 27093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.018s	sys 0.009s
W20260709 13:13:22.502483 22758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.021s	sys 0.044s
I20260709 13:13:22.426041 27220 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.477478 27327 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:22.471921 27187 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.510360 22913 rpcz_store.cc:269] 0709 13:13:19.872148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.872455 (+   307us) service_pool.cc:224] Handling call
0709 13:13:22.509321 (+2636866us) spinlock_profiling.cc:243] Waited 2.54 s on lock 0x7b54000438d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:22.509591 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5846172416}
W20260709 13:13:22.516716 22758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.022s	sys 0.045s
W20260709 13:13:22.495359 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.004s	sys 0.015s
W20260709 13:13:22.503461 27322 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.017s
I20260709 13:13:22.522452 22915 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:22.523603 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.529047 22908 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
W20260709 13:13:22.527571 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.004s	sys 0.016s
I20260709 13:13:22.532003 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:22.532651 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.533051 23466 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:22.550957 27235 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.550958 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.551905 27235 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.553129 27259 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:22.530133 27322 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.004s	sys 0.017s
I20260709 13:13:22.547034 27213 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.557710 27235 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.559396 27296 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:22.559952 27296 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.560854 27272 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.932s	user 0.012s	sys 0.002s
I20260709 13:13:22.561892 22911 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:22.515064 27141 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:22.515586 27163 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.575902 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.520394 27297 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.582588 22841 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 13:13:22.528550 27261 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:22.583139 22841 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 4
I20260709 13:13:22.571468 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:22.591616 22841 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:13:22.592001 22841 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:22.586129 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 291359us
I20260709 13:13:22.589689 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.595916 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.595974 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.596916 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.597379 22837 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:22.599284 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:22.512178 27284 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:22.601786 23465 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:22.606236 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.606778 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:22.601455 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 380359us
I20260709 13:13:22.603704 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.562430 27296 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.618770 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.619562 23466 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:22.603864 23463 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:13:22.607960 22837 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:22.620755 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.640103 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:22.640547 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.609768 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:22.564723 27061 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:13:22.618636 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.648685 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.658183 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.582835 27141 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:22.602849 27233 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:22.670508 22833 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.024s	sys 0.009s
I20260709 13:13:22.607895 27136 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:13:22.696774 22833 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.024s	sys 0.009s
W20260709 13:13:22.710858 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 101168us
I20260709 13:13:22.716698 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.717872 22880 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:22.712128 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:22.710089 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.618686 27297 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:22.700140 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.714092 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:22.610805 27347 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:22.582695 27163 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.710474 27334 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.328s	user 0.000s	sys 0.002s
W20260709 13:13:22.711383 27330 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:13:22.748126 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.666702 26948 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:22.749943 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 375749us
I20260709 13:13:22.718057 27233 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.668915 27249 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.620477 27157 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 13:13:22.740587 26952 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:22.722668 27297 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.724458 27235 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:22.765322 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.749190 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.753090 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:22.746804 27347 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:22.794850 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1718) took 2319 ms. Trace:
W20260709 13:13:22.799568 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.801473 23463 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 7.
I20260709 13:13:22.813973 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.819705 22837 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:22.765805 27233 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.821559 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.758849 27272 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:22.800686 27249 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:13:22.797695 27109 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:22.828748 27324 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.800344 27297 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:22.798789 27259 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:22.832599 27187 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:22.834534 27346 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:13:22.842244 27304 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:22.843822 27061 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:22.844121 27163 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.845830 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2680) took 1202 ms. Trace:
W20260709 13:13:22.854115 27338 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:13:22.860422 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 568) took 1591 ms. Trace:
I20260709 13:13:22.861538 27329 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.873713 27141 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:22.875681 27220 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.875774 27287 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:22.873134 27298 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.858s	user 0.010s	sys 0.002s
W20260709 13:13:22.806134 27336 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:13:22.889369 22837 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:22.805896 22912 rpcz_store.cc:276] 0709 13:13:20.475636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.475794 (+   158us) service_pool.cc:224] Handling call
0709 13:13:20.514690 (+ 38896us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:20.933373 (+418683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:20.933399 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:20.967812 (+ 34413us) log.cc:844] Serialized 52 byte log entry
0709 13:13:22.124811 (+1156999us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:22.577345 (+452534us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.577779 (+   434us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.793467 (+215688us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b1800386f80. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f68a16a7d70 00007f68a169b293 00007f68a1690cea 00007f68a169123a 00007f68a1afba2a 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:13:22.794129 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.794261 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.794476 (+   215us) raft_consensus.cc:1685] finished
0709 13:13:22.794497 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.794701 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44538,"prepare.queue_time_us":640830,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":10689,"raft.queue_time_us":199896,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8151,"spinlock_wait_cycles":35337856,"thread_start_us":1954077,"threads_started":3,"wal-append.queue_time_us":1344274}
I20260709 13:13:22.887914 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:22.900754 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.908118 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:22.948165 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 60412.5us
I20260709 13:13:22.808377 27318 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:22.909327 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:22.951489 23463 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:22.883214 27293 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.054s	user 0.009s	sys 0.002s
I20260709 13:13:22.813853 27347 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:22.924417 22832 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:22.923859 22914 rpcz_store.cc:276] 0709 13:13:21.643840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.644055 (+   215us) service_pool.cc:224] Handling call
0709 13:13:21.644426 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.644793 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.644830 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.019178 (+374348us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b1800302c20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a
0709 13:13:22.019660 (+   482us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b2c00051d28. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:22.020211 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.493347 (+473136us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:22.493429 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.493565 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.845568 (+352003us) raft_consensus.cc:1685] finished
0709 13:13:22.845593 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.845759 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2823,"raft.queue_time_us":2255,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":12274,"spinlock_wait_cycles":14564352,"thread_start_us":472537,"threads_started":1,"wal-append.queue_time_us":823308}
I20260709 13:13:22.930269 22758 rpcz_store.cc:276] 0709 13:13:21.269475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.790558 (+521083us) service_pool.cc:224] Handling call
0709 13:13:21.810014 (+ 19456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.810421 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.810454 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.814477 (+  4023us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.527111 (+712634us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:22.527164 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.527270 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.860063 (+332793us) raft_consensus.cc:1685] finished
0709 13:13:22.860099 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.860330 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63172,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":7648,"thread_start_us":630404,"threads_started":1,"wal-append.queue_time_us":893476}
I20260709 13:13:22.979871 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 672) took 1784 ms. Trace:
I20260709 13:13:22.817813 27235 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.940743 22757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.014s	sys 0.041s
I20260709 13:13:22.942202 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 569) took 1615 ms. Trace:
I20260709 13:13:22.984139 22766 rpcz_store.cc:276] 0709 13:13:21.327123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.010254 (+683131us) service_pool.cc:224] Handling call
0709 13:13:22.010653 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.011075 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.011107 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.011753 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.423362 (+411609us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:22.941453 (+518091us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.941757 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.941860 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.941919 (+    59us) raft_consensus.cc:1685] finished
0709 13:13:22.941937 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.942122 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":183183,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":1197,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":867272,"threads_started":2,"wal-append.queue_time_us":638112}
W20260709 13:13:22.945088 22764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.011s	sys 0.031s
I20260709 13:13:22.972199 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.980334 22835 rpcz_store.cc:276] 0709 13:13:21.195730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.195958 (+   228us) service_pool.cc:224] Handling call
0709 13:13:21.196418 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.197102 (+   684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.197157 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.197928 (+   771us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.101568 (+903640us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:22.101630 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.101731 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.979611 (+877880us) raft_consensus.cc:1685] finished
0709 13:13:22.979637 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.979806 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2136,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"thread_start_us":902835,"threads_started":1,"wal-append.queue_time_us":1075236}
W20260709 13:13:22.987010 22764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.011s	sys 0.031s
W20260709 13:13:22.983758 22757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.014s	sys 0.041s
I20260709 13:13:22.995972 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 567) took 1751 ms. Trace:
I20260709 13:13:22.909834 27109 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:22.970808 27321 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.002s
I20260709 13:13:23.003541 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2540) took 1841 ms. Trace:
W20260709 13:13:22.979045 26930 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.850s	user 0.013s	sys 0.003s
I20260709 13:13:23.005571 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:22.944175 27061 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:23.006276 22764 rpcz_store.cc:276] 0709 13:13:21.161784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.556882 (+395098us) service_pool.cc:224] Handling call
0709 13:13:21.557368 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.558354 (+   986us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:22.301239 (+742885us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.002988 (+701749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.003086 (+    98us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.003161 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.003267 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.003453 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34359,"prepare.queue_time_us":953505,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"thread_start_us":1404884,"threads_started":2}
I20260709 13:13:23.009217 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.982406 27003 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:23.010281 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:22.972905 27250 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:22.986729 27293 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:23.062067 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.080228 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:22.996649 22759 rpcz_store.cc:276] 0709 13:13:21.244709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.618719 (+374010us) service_pool.cc:224] Handling call
0709 13:13:21.619054 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.619269 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:22.292503 (+673234us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.995535 (+703032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.995610 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:22.995658 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.995728 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.995897 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23411,"prepare.queue_time_us":908843,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":300473,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":59,"thread_start_us":1375397,"threads_started":3,"wal-append.queue_time_us":678701}
I20260709 13:13:22.996201 27264 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:23.076332 27357 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:13:22.974642 27298 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.169930 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:23.174665 22906 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:23.041020 27003 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.974354 27287 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.013051 27321 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:23.196105 23464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.021s	sys 0.040s
I20260709 13:13:23.180893 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:23.211876 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.212273 23459 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.969066 27320 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.210700 22835 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:23.021379 26930 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:23.214043 23464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.021s	sys 0.041s
I20260709 13:13:23.177327 27284 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:23.209097 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 454898us
W20260709 13:13:23.197544 27031 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.057s	user 0.016s	sys 0.001s
W20260709 13:13:23.217730 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.198784 27366 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:13:23.228026 27031 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.229032 27321 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:23.230688 27031 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:23.016450 27364 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.192s
I20260709 13:13:23.196201 27264 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.231508 27287 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:23.232394 26930 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:23.219225 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.034s	sys 0.030s
W20260709 13:13:23.249083 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.034s	sys 0.030s
I20260709 13:13:23.251920 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:23.253377 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:23.235396 22834 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.022s	sys 0.042s
W20260709 13:13:23.244104 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.023s	sys 0.049s
W20260709 13:13:23.257951 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.023s	sys 0.049s
W20260709 13:13:23.266040 22806 connection.cc:256] RPC call timeout handler was delayed by -1.2299062809998986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 13:13:23.166380 27250 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:23.234616 26830 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.734s	user 0.014s	sys 0.005s
I20260709 13:13:23.222000 27003 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:23.266886 22806 connection.cc:256] RPC call timeout handler was delayed by -1.1323938959999396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 13:13:23.258104 22834 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.023s	sys 0.042s
W20260709 13:13:23.270063 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 517071us
I20260709 13:13:23.078007 27061 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.256038 27329 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:13:23.257264 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1893) took 1045 ms. Trace:
W20260709 13:13:23.284440 23460 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.019s	sys 0.003s
W20260709 13:13:23.323674 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 51578.6us
W20260709 13:13:23.335110 27348 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.590s	user 0.004s	sys 0.000s
W20260709 13:13:23.369200 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.000s	sys 0.025s
W20260709 13:13:23.379492 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.000s	sys 0.026s
I20260709 13:13:23.380002 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:23.268886 27287 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.342350 22837 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:13:23.325676 27361 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:13:23.372455 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.003s	sys 0.010s
W20260709 13:13:23.373083 23460 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.020s	sys 0.004s
W20260709 13:13:23.325029 27368 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:13:23.380507 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 711810us
W20260709 13:13:23.331895 27337 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.017s	sys 0.009s
I20260709 13:13:23.400609 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:23.256431 27259 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.124s
W20260709 13:13:23.291853 27362 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:13:23.323930 27095 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:13:23.270881 26952 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:23.383030 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:23.383708 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1867) took 1818 ms. Trace:
I20260709 13:13:23.385185 27160 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:23.266935 27247 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:23.327869 27335 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.964s	user 0.001s	sys 0.005s
W20260709 13:13:23.403213 27337 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.022s	sys 0.011s
I20260709 13:13:23.388975 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:23.405961 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:23.391633 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:23.429889 23464 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:23.296118 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1724) took 1642 ms. Trace:
I20260709 13:13:23.370441 22760 rpcz_store.cc:276] 0709 13:13:22.211621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.233985 (+ 22364us) service_pool.cc:224] Handling call
0709 13:13:22.234379 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.234626 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:22.789077 (+554451us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b1800473080. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf
0709 13:13:22.789330 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.256839 (+467509us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.256900 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:23.256949 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.257024 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.257192 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":355136,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":69452,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":124,"spinlock_wait_cycles":74514816,"thread_start_us":827236,"threads_started":2}
W20260709 13:13:23.395793 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.004s	sys 0.011s
I20260709 13:13:23.271201 26930 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:23.324317 27372 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:13:23.398437 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.326603 27381 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:23.289088 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0999485639998738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
I20260709 13:13:23.340690 23459 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:23.392776 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:23.467610 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0882485639998549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260709 13:13:23.403406 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:23.468046 22806 connection.cc:256] RPC call timeout handler was delayed by -1.0081485639998391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23010000000000003s for a response
I20260709 13:13:23.405301 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:23.331678 27264 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:23.382349 27219 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:23.306627 27378 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:23.408113 22839 rpcz_store.cc:276] 0709 13:13:21.564941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.565096 (+   155us) service_pool.cc:224] Handling call
0709 13:13:21.602866 (+ 37770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.639413 (+ 36547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.639450 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.639982 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.406020 (+766038us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:22.406071 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.423289 (+ 17218us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b54000ec754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:22.423365 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.334615 (+911250us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b240029f348. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea4309d 00007f68a167510e 00007f68a168c88c 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221
0709 13:13:23.383441 (+ 48826us) raft_consensus.cc:1685] finished
0709 13:13:23.383473 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.383633 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7457,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3218,"spinlock_wait_cycles":240771584,"thread_start_us":695559,"threads_started":1,"wal-append.queue_time_us":943776}
I20260709 13:13:23.263890 27318 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:13:23.292372 26830 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.422207 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 571) took 2034 ms. Trace:
W20260709 13:13:23.486434 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93764.8us
I20260709 13:13:23.288014 27294 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:13:23.396907 27351 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
W20260709 13:13:23.431639 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 673728us
I20260709 13:13:23.431143 22911 rpcz_store.cc:276] 0709 13:13:21.653869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.767593 (+113724us) service_pool.cc:224] Handling call
0709 13:13:23.295793 (+1528200us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b540010fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:23.296030 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1823093504}
I20260709 13:13:23.440809 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:23.487632 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:23.487818 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:23.455088 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:23.418778 27298 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:23.489461 22759 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.491425 26830 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:23.482324 22763 rpcz_store.cc:276] 0709 13:13:21.388198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.093667 (+705469us) service_pool.cc:224] Handling call
0709 13:13:22.108041 (+ 14374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.142125 (+ 34084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.142157 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.154025 (+ 11868us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.594759 (+440734us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:22.594808 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.594883 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.421935 (+827052us) raft_consensus.cc:1685] finished
0709 13:13:23.421962 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.422132 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33321,"raft.queue_time_us":38398,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":4896,"thread_start_us":358613,"threads_started":1,"wal-append.queue_time_us":729640}
I20260709 13:13:23.492092 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:23.493906 26952 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.428838 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:23.466805 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60993.6us
I20260709 13:13:23.398088 27287 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.504140 27287 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:23.505306 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:23.506039 27377 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:13:23.471195 27219 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:23.506462 27298 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:23.507067 27298 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.507534 27298 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.479271 27378 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.510246 27378 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:23.484586 27320 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:23.478513 27382 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:23.514989 27382 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:23.515432 27382 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.448900 26930 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.520092 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1896) took 1305 ms. Trace:
W20260709 13:13:23.522662 27390 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:23.520629 22761 rpcz_store.cc:276] 0709 13:13:22.215112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.413937 (+198825us) service_pool.cc:224] Handling call
0709 13:13:22.414362 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.883917 (+469555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.883955 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.884644 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.363516 (+478872us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:23.363600 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.363858 (+   258us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.519771 (+155913us) raft_consensus.cc:1685] finished
0709 13:13:23.519808 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.520008 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":459716,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":102,"thread_start_us":892231,"threads_started":2,"wal-append.queue_time_us":595800}
I20260709 13:13:23.523034 22759 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
W20260709 13:13:23.527707 27327 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.978s	user 0.010s	sys 0.005s
I20260709 13:13:23.528281 27327 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.529596 22840 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:23.530186 27373 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:13:23.529991 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:23.533627 27389 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:23.534286 27389 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:23.534823 27389 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.543114 27109 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:23.543949 27386 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.220s	user 0.001s	sys 0.002s
I20260709 13:13:23.544651 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 820) took 1944 ms. Trace:
I20260709 13:13:23.547600 23430 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:23.549958 27066 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:23.485579 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:23.554370 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68892.7us
I20260709 13:13:23.545063 23457 rpcz_store.cc:276] 0709 13:13:21.600165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.600382 (+   217us) service_pool.cc:224] Handling call
0709 13:13:21.600796 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.295317 (+694521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.295349 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.295982 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.755702 (+459720us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:22.755767 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.755875 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.544381 (+788506us) raft_consensus.cc:1685] finished
0709 13:13:23.544406 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.544568 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36943,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":1047362,"threads_started":2,"wal-append.queue_time_us":722637}
I20260709 13:13:23.547470 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1874) took 1945 ms. Trace:
I20260709 13:13:23.559031 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:23.561029 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73475.9us
I20260709 13:13:23.565272 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:23.569739 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 156763us
I20260709 13:13:23.572229 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:23.573736 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:23.559286 22833 rpcz_store.cc:276] 0709 13:13:21.606312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.117431 (+511119us) service_pool.cc:224] Handling call
0709 13:13:22.117771 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.118178 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:22.716793 (+598615us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.249251 (+532458us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.250299 (+  1048us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.250479 (+   180us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:23.250527 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.250623 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.542750 (+292127us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:23.546979 (+  4229us) raft_consensus.cc:1685] finished
0709 13:13:23.547153 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.547383 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1417,"prepare.queue_time_us":891144,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":2461,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":7465,"spinlock_wait_cycles":64448896,"thread_start_us":1062666,"threads_started":2,"wal-append.queue_time_us":566301}
I20260709 13:13:23.578886 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:23.611552 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:23.572276 27109 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:13:23.586174 27287 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:23.617189 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:23.656447 22912 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.577208 27329 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:23.644717 27233 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.003s
I20260709 13:13:23.578898 27160 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:23.569741 27272 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:23.604470 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:23.667337 22913 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:23.559384 27284 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:23.591522 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:23.572360 27347 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.004s
W20260709 13:13:23.602381 27379 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.262s	user 0.003s	sys 0.000s
I20260709 13:13:23.604903 27317 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:13:23.581274 27383 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:23.659037 27351 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:23.588894 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.604246 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:23.715739 22911 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:23.644515 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1891) took 2017 ms. Trace:
I20260709 13:13:23.655974 27287 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.618045 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:23.646309 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67523.9us
I20260709 13:13:23.676499 27381 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.610533 27397 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:23.521214 27298 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.521122 26930 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:23.626734 27264 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:13:23.695178 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:23.593447 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:23.712577 22912 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:23.684094 27297 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.764s	user 0.016s	sys 0.000s
I20260709 13:13:23.645908 22836 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:23.805080 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:23.715206 27351 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.680362 27284 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.717212 22757 rpcz_store.cc:276] 0709 13:13:21.627241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.116145 (+488904us) service_pool.cc:224] Handling call
0709 13:13:22.116508 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:22.141907 (+ 25399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.141938 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:22.172721 (+ 30783us) log.cc:844] Serialized 52 byte log entry
0709 13:13:22.988459 (+815738us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:23.107350 (+118891us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.166284 (+ 58934us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.181979 (+ 15695us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b48002ac0d8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:13:23.384578 (+202599us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b1800473260. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:23.386242 (+  1664us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b1800473260. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:23.387462 (+  1220us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800473260. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:23.643679 (+256217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.643985 (+   306us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.644154 (+   169us) raft_consensus.cc:1685] finished
0709 13:13:23.644182 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.644420 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83132,"prepare.queue_time_us":208470,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":162432,"raft.run_cpu_time_us":15324,"raft.run_wall_time_us":1035789,"spinlock_wait_cycles":165479424,"thread_start_us":1184917,"threads_started":2,"wal-append.queue_time_us":996114}
I20260709 13:13:23.759027 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:23.604768 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.676115 27272 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.688032 27327 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:23.761394 27287 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:23.792477 22911 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:13:23.797178 27371 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.375s	user 0.001s	sys 0.001s
I20260709 13:13:23.801519 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:23.702147 27396 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:23.819865 22833 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.822654 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1887) took 1212 ms. Trace:
I20260709 13:13:23.741654 22913 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:13:23.814764 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55827.3us
W20260709 13:13:23.778100 27384 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:13:23.831678 27396 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.683104 27233 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.833619 23458 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:23.817786 27284 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.835094 27233 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:23.816992 26830 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:23.836592 26830 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.837944 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2541) took 2675 ms. Trace:
I20260709 13:13:23.838433 27259 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:23.821343 27367 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:23.838367 22765 rpcz_store.cc:276] 0709 13:13:21.162029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.557247 (+395218us) service_pool.cc:224] Handling call
0709 13:13:21.557709 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.656523 (+ 98814us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:22.452703 (+796180us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.923690 (+470987us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.952986 (+ 29296us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.953202 (+   216us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:22.953522 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.953721 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.953823 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.837671 (+883848us) raft_consensus.cc:1685] finished
0709 13:13:23.837704 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.837870 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79012,"prepare.queue_time_us":913468,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":4916,"raft.queue_time_us":563901,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":29499,"thread_start_us":1178690,"threads_started":2,"wal-append.queue_time_us":840578}
I20260709 13:13:23.839720 26830 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:23.821003 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 125963us
I20260709 13:13:23.842223 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1886) took 1232 ms. Trace:
I20260709 13:13:23.817519 27272 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.821669 27160 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:23.825587 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:23.842634 22840 rpcz_store.cc:276] 0709 13:13:22.609773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.609947 (+   174us) service_pool.cc:224] Handling call
0709 13:13:22.610343 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.530112 (+919769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.841957 (+311845us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400038ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:23.842149 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113493248}
I20260709 13:13:23.843997 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:23.844688 22840 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.844825 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:23.832366 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 692) took 1230 ms. Trace:
I20260709 13:13:23.684834 27347 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.839808 27367 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.846642 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:13:23.856653 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 386152us
I20260709 13:13:23.846848 23460 rpcz_store.cc:276] 0709 13:13:22.602241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.602464 (+   223us) service_pool.cc:224] Handling call
0709 13:13:22.647929 (+ 45465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.710712 (+ 62783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.710745 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.711426 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.404275 (+692849us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:23.404341 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.404447 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.832066 (+427619us) raft_consensus.cc:1685] finished
0709 13:13:23.832093 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.832284 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5249,"raft.queue_time_us":62392,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":16767,"thread_start_us":632758,"threads_started":1,"wal-append.queue_time_us":646685}
I20260709 13:13:23.823022 22834 rpcz_store.cc:276] 0709 13:13:22.610586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.610761 (+   175us) service_pool.cc:224] Handling call
0709 13:13:22.660649 (+ 49888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.371763 (+711114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.371800 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.372345 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.633276 (+260931us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.636241 (+  2965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.636619 (+   378us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.822406 (+185787us) raft_consensus.cc:1685] finished
0709 13:13:23.822427 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.822580 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131559,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":905632,"threads_started":2,"wal-append.queue_time_us":426569}
I20260709 13:13:23.813968 27351 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:23.858232 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 369573us
I20260709 13:13:23.854372 22833 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:23.824472 27399 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:23.801333 27329 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:13:23.870141 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.871238 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.871791 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.874868 22762 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:23.872251 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.869410 22840 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:23.846608 27061 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:23.768841 27003 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.860524 27308 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:23.938020 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:23.923810 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50803us
I20260709 13:13:23.932142 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.950349 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.856791 27294 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:23.858214 27259 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.812654 27297 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:23.925314 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:23.939450 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1897) took 1204 ms. Trace:
W20260709 13:13:23.924741 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 333363us
I20260709 13:13:23.872514 27293 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.770186 27381 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:23.955755 22806 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.842s (SENT)
I20260709 13:13:23.955753 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.919354 27399 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:23.958763 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:23.961287 22840 rpcz_store.cc:276] 0709 13:13:22.732212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.935099 (+1202887us) service_pool.cc:224] Handling call
0709 13:13:23.935588 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.935800 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.935846 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.936084 (+   238us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.936141 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.936215 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.936369 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:23.961350 22834 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:23.966023 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:23.940778 27308 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.969728 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:23.862192 27382 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:23.989677 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:23.978992 27141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.008s	sys 0.009s
W20260709 13:13:24.001026 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45287.5us
W20260709 13:13:24.006897 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.199s (SENT)
I20260709 13:13:23.986518 27308 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.995939 27380 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:23.975416 27293 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.028172 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.041401 27141 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.050565 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.051614 22908 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:13:24.056475 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.057269 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.059983 27163 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:24.060622 27163 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.063495 27381 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.063818 27163 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.069579 22912 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:24.069483 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2670) took 3166 ms. Trace:
I20260709 13:13:24.072404 22915 rpcz_store.cc:276] 0709 13:13:20.903560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.903730 (+   170us) service_pool.cc:224] Handling call
0709 13:13:20.936755 (+ 33025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.528529 (+1591774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.069192 (+1540663us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54003c87d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:24.069409 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3646259456}
W20260709 13:13:24.044243 22806 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.212s (SENT)
I20260709 13:13:24.079974 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.080760 22911 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:13:24.088248 27406 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:13:24.092504 22866 heartbeater.cc:646] Failed to heartbeat to 127.21.33.62:44143 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.33.62:44143 timed out after 15.000s (SENT)
I20260709 13:13:24.093252 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1892) took 1472 ms. Trace:
I20260709 13:13:24.087994 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.094007 22839 rpcz_store.cc:276] 0709 13:13:22.621232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.493363 (+872131us) service_pool.cc:224] Handling call
0709 13:13:23.493911 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.494405 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.494435 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.494897 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.862849 (+367952us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.862893 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.862964 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.092995 (+230031us) raft_consensus.cc:1685] finished
0709 13:13:24.093018 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.093183 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":149,"thread_start_us":367442,"threads_started":1,"wal-append.queue_time_us":541313}
I20260709 13:13:24.039680 27411 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:24.078922 27389 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.543s	user 0.015s	sys 0.002s
I20260709 13:13:24.100471 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:24.098284 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:24.097545 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53482.9us
I20260709 13:13:24.072988 27294 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.104993 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.113960 22915 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.111179 22912 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:24.106957 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.301s (SENT)
I20260709 13:13:24.114503 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.120275 22839 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:24.117157 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.127852 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.130307 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:24.094673 27364 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.137944 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.139120 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.116256 27320 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:24.141084 27247 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:24.141358 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.150152 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.140671 27255 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:24.153950 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.154891 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.155092 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 686) took 1178 ms. Trace:
W20260709 13:13:24.155339 23432 connection.cc:256] RPC call timeout handler was delayed by -1.1265449879998641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:13:24.155756 23432 connection.cc:256] RPC call timeout handler was delayed by -1.0988476549998722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
W20260709 13:13:24.157624 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50710.6us
I20260709 13:13:24.158557 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:24.159387 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 681) took 1210 ms. Trace:
I20260709 13:13:24.155514 22835 rpcz_store.cc:276] 0709 13:13:22.976883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.005234 (+ 28351us) service_pool.cc:224] Handling call
0709 13:13:24.154814 (+1149580us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54003228d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:24.155025 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457323264}
I20260709 13:13:24.160370 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:24.161034 22841 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:24.161190 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.162007 22835 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:24.148240 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.152100 22761 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11.
I20260709 13:13:24.128111 27369 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.153321 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.159735 22840 rpcz_store.cc:276] 0709 13:13:22.949023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.982412 (+1033389us) service_pool.cc:224] Handling call
0709 13:13:23.982886 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.983190 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:23.989581 (+  6391us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.159010 (+169429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.159085 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:24.159147 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.159227 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.159337 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":183,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":3856,"thread_start_us":168962,"threads_started":1}
I20260709 13:13:24.141041 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:24.164353 27417 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.172802 22764 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:24.174010 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.174718 22764 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:24.175642 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.176395 22764 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:24.177183 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.177748 22764 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:24.032795 27399 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.184979 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:24.219215 22915 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:24.173728 27247 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.126636 27389 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.132144 27398 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.126237 27411 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:24.141283 27320 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.186293 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.219096 27396 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.218832 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.222714 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.316018 22837 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:13:24.128908 27347 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.227926 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 682) took 1278 ms. Trace:
I20260709 13:13:24.317493 22839 rpcz_store.cc:276] 0709 13:13:22.949353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.226160 (+1276807us) service_pool.cc:224] Handling call
0709 13:13:24.226661 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.226932 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.226971 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.227038 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:24.227084 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.227157 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.227651 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:24.229247 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57673.6us
I20260709 13:13:24.230947 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.320912 22757 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:24.322155 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.322805 22757 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:24.323685 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.231212 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:24.324319 27380 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:24.232017 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.326668 22835 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:24.327406 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 683) took 1377 ms. Trace:
I20260709 13:13:24.327832 22835 rpcz_store.cc:276] 0709 13:13:22.949626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.326107 (+1376481us) service_pool.cc:224] Handling call
0709 13:13:24.326470 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.327131 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.327349 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.329156 22835 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:13:24.233050 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.325666 27427 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:24.143414 27250 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.171833 27287 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.246335 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.142262 27382 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.145828 27293 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:24.247016 27255 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:24.249953 27417 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.328923 27233 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:24.247148 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:24.258903 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 102493us
I20260709 13:13:24.246430 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:24.254277 27378 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.743s	user 0.015s	sys 0.010s
I20260709 13:13:24.285135 27272 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:24.262760 23458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 693) took 1657 ms (client timeout 2079 ms). Trace:
I20260709 13:13:24.142270 27367 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:24.271194 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 112625us
I20260709 13:13:24.246829 27317 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:24.217769 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.277657 27411 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:24.278604 27409 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:13:24.279845 23463 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:24.281947 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:24.283629 27415 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:13:24.284730 27404 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:24.286773 27426 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:24.128708 27294 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.150537 27364 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:24.290104 27416 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:13:24.291836 27389 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.294246 27339 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:24.232583 27385 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.295075 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1895) took 1638 ms. Trace:
I20260709 13:13:24.412390 22833 rpcz_store.cc:276] 0709 13:13:22.657143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.932013 (+1274870us) service_pool.cc:224] Handling call
0709 13:13:23.932432 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.932854 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.932887 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.933415 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.069328 (+135913us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:24.069390 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.069482 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.294495 (+225013us) raft_consensus.cc:1685] finished
0709 13:13:24.294523 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.294695 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3743,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10796,"thread_start_us":103942,"threads_started":1,"wal-append.queue_time_us":341782}
I20260709 13:13:24.387398 27322 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.414304 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:24.295428 27351 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.296442 27398 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.281191 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:24.300120 27061 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:24.301918 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.425132 22760 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:24.217942 27031 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.304852 22758 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:24.427074 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.221318 22761 raft_consensus.cc:2368] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:24.365990 27378 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.311712 22834 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:24.234412 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.304708 27399 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:24.334779 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 687) took 1349 ms. Trace:
I20260709 13:13:24.338913 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:24.303967 27297 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.361814 27233 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.447757 22835 rpcz_store.cc:276] 0709 13:13:22.981972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.328604 (+1346632us) service_pool.cc:224] Handling call
0709 13:13:24.328936 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.330687 (+  1751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.330900 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.452061 22913 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:24.452406 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1904) took 1009 ms. Trace:
W20260709 13:13:24.343497 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 111570us
I20260709 13:13:24.454945 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.218763 27369 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.332119 27427 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:24.250959 22841 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:24.459003 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1905) took 1002 ms. Trace:
I20260709 13:13:24.459774 22841 rpcz_store.cc:276] 0709 13:13:23.456799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.457632 (+1000833us) service_pool.cc:224] Handling call
0709 13:13:24.458095 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.458335 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.458373 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.458435 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:24.458484 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.458565 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.458713 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.360198 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:24.363018 23458 rpcz_store.cc:269] 0709 13:13:22.605699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.605934 (+   235us) service_pool.cc:224] Handling call
0709 13:13:24.262488 (+1656554us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400650254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:24.262678 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2820026240}
I20260709 13:13:24.380365 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:24.462719 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 102657us
I20260709 13:13:24.382113 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:24.466771 23464 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:24.384147 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.468421 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:24.389164 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:24.370486 27317 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:24.347920 27347 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.337678 27429 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:13:24.394719 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.379714 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:24.476006 22759 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:24.476336 22765 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:24.402928 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.277485 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1664) took 2643 ms. Trace:
W20260709 13:13:24.409601 27284 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 13:13:24.431236 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.432443 22833 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:24.481316 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.428018 22758 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:24.432430 22764 raft_consensus.cc:2368] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:24.427460 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 684) took 1477 ms. Trace:
I20260709 13:13:24.483929 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.483884 22839 rpcz_store.cc:276] 0709 13:13:22.949964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.329955 (+1379991us) service_pool.cc:224] Handling call
0709 13:13:24.330252 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.330473 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:24.338215 (+  7742us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.426962 (+ 88747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.427085 (+   123us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:24.427161 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.427249 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.427373 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10728,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":78632,"thread_start_us":71256,"threads_started":2}
I20260709 13:13:24.448410 22757 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11.
I20260709 13:13:24.476811 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.486649 22760 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
W20260709 13:13:24.476019 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 96468us
I20260709 13:13:24.491103 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.477880 22832 rpcz_store.cc:276] 0709 13:13:21.623117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.121924 (+498807us) service_pool.cc:224] Handling call
0709 13:13:22.122241 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.983244 (+861003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.266110 (+1282866us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54001bce54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:24.266333 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844285184}
I20260709 13:13:24.498976 22761 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
W20260709 13:13:24.499372 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.391965 27421 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:13:24.417263 27426 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:24.495857 22757 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:24.499447 22761 pending_rounds.cc:85] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 5
I20260709 13:13:24.509712 22761 pending_rounds.cc:107] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:13:24.509809 22834 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:24.510031 22761 raft_consensus.cc:2889] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.509403 23462 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:24.502295 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.510962 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.511809 22760 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:24.479041 27404 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:13:24.508144 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:24.452823 22835 rpcz_store.cc:276] 0709 13:13:23.442831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.451530 (+1008699us) service_pool.cc:224] Handling call
0709 13:13:24.451865 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.452040 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.452065 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.452109 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:24.452154 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.452211 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.452331 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:24.516189 22759 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:24.492954 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.505716 22763 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:24.508530 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.513614 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.464937 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.517619 22915 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:24.517858 22758 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:13:24.518388 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53535.2us
I20260709 13:13:24.520882 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.521814 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.520947 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 568) took 2310 ms. Trace:
I20260709 13:13:24.476722 27061 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:24.516646 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:24.526104 22764 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:24.523478 22909 rpcz_store.cc:276] 0709 13:13:22.210342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.372526 (+162184us) service_pool.cc:224] Handling call
0709 13:13:22.372972 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.510001 (+137029us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:22.987955 (+477954us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.370942 (+382987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.371061 (+   119us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:23.371142 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.520640 (+1149498us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540020b7d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:24.520691 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.520883 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18883,"prepare.queue_time_us":187111,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":72,"raft.queue_time_us":8190,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":3407,"spinlock_wait_cycles":314532608,"thread_start_us":851970,"threads_started":2,"wal-append.queue_time_us":532451}
I20260709 13:13:24.496330 27261 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:24.482980 27321 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.500183 27160 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.425453 27322 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.544539 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.495709 27367 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.533807 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.541464 22761 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:24.532896 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.446344 27031 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.546622 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.547978 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.551751 22911 raft_consensus.cc:2376] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14. Re-sending same reply.
I20260709 13:13:24.418710 27255 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.551458 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:24.546555 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 207988us
I20260709 13:13:24.551458 22910 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:24.557991 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:24.559234 22841 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:13:24.559960 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.447105 27435 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 13:13:24.521893 27343 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.533134 27297 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:24.579558 27031 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.566998 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:24.580977 27233 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.426473 27385 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.572292 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:24.585386 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.587620 22909 raft_consensus.cc:606] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.529881 27347 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:24.533116 27261 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:24.505586 27308 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.588145 22909 raft_consensus.cc:614] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:24.532262 27157 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:13:24.425231 27339 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.617827 22757 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:24.499229 27284 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.622290 27284 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:24.606062 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46595.5us
I20260709 13:13:24.624193 27436 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.582528 27317 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.622093 27438 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.531234 27061 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:24.632164 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.609035 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.636543 22908 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:24.636730 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.619962 27364 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.624925 27436 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.645972 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1898) took 2429 ms. Trace:
I20260709 13:13:24.646922 27436 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.614328 27321 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:24.615953 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:24.614631 27261 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.587761 26830 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.619089 27329 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.587463 27320 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:24.642004 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.969s (SENT)
I20260709 13:13:24.618399 27264 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.592624 22841 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:24.641250 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.675132 22912 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:24.646462 22766 rpcz_store.cc:276] 0709 13:13:22.216411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.001207 (+784796us) service_pool.cc:224] Handling call
0709 13:13:23.015108 (+ 13901us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:23.406904 (+391796us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:23.426677 (+ 19773us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.875419 (+448742us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:23.925283 (+ 49864us) log.cc:844] Serialized 52 byte log entry
0709 13:13:23.925471 (+   188us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.932729 (+  7258us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.932950 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.933023 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.051064 (+118041us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b1800473020. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:24.061321 (+ 10257us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b1800473020. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f68a59c558f 00007f689ea04571 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd
0709 13:13:24.645713 (+584392us) raft_consensus.cc:1685] finished
0709 13:13:24.645740 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.645905 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33519,"prepare.queue_time_us":24855,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6290,"raft.queue_time_us":221891,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":21971,"spinlock_wait_cycles":21090688,"thread_start_us":757828,"threads_started":2,"wal-append.queue_time_us":633002}
I20260709 13:13:24.613955 27297 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:24.652982 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.659626 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.663631 27219 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.643280 27294 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:24.643000 27380 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.665417 27061 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:24.666668 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50811.5us
W20260709 13:13:24.696321 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54366.8us
I20260709 13:13:24.667080 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.696501 27380 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.698273 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:24.667184 27428 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:24.699476 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:24.670004 27417 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.699676 27297 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.698298 27440 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:24.700850 22765 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:24.635663 27426 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.668740 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1902) took 1271 ms. Trace:
I20260709 13:13:24.701834 22833 rpcz_store.cc:276] 0709 13:13:23.397117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.415298 (+1018181us) service_pool.cc:224] Handling call
0709 13:13:24.432174 (+ 16876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.481906 (+ 49732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.668523 (+186617us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b54001bc450. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:24.668676 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38062336}
I20260709 13:13:24.654105 27364 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:24.704095 27322 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.704722 27272 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.701134 27380 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.701363 27428 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:24.639189 27347 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.647534 27321 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:24.694918 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.712286 22912 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:24.696213 27219 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:24.718418 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:24.719614 27351 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:24.716112 27321 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:24.707517 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 54638.4us
I20260709 13:13:24.700007 27389 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:24.693894 27329 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.754477 27235 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.756038 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2686) took 3092 ms. Trace:
I20260709 13:13:24.811805 22906 rpcz_store.cc:276] 0709 13:13:21.663983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.021018 (+357035us) service_pool.cc:224] Handling call
0709 13:13:22.021442 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.178991 (+1157549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.755739 (+1576748us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400041354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:24.755967 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651501312}
I20260709 13:13:24.755712 27247 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.756482 27308 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.581889 27427 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.757045 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:24.813892 22834 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:24.758483 27339 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.759517 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.756490 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.762953 27385 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.757047 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.817938 22914 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:24.763972 27398 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.766218 27003 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.729889 27428 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.798893 22836 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:24.805220 27389 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:24.751175 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:24.821033 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 276571us
I20260709 13:13:24.826858 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:24.822468 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 71433.3us
W20260709 13:13:24.871881 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45022.6us
W20260709 13:13:24.872473 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48027.6us
I20260709 13:13:24.874989 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:24.879678 22912 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:24.881883 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:24.889138 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.908632 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.923463 22832 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:24.898639 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:24.911135 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:24.871172 27398 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:24.928524 27398 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:24.929105 27398 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:24.930570 22910 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:13:24.934911 27249 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:24.935745 27249 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:24.938794 27298 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.941973 27249 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.943091 27329 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:24.888626 27066 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.943802 22914 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:24.945225 22906 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:13:24.945719 22906 pending_rounds.cc:85] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 8
I20260709 13:13:24.945960 22906 pending_rounds.cc:107] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:13:24.946211 22906 raft_consensus.cc:2889] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.948251 27160 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.951967 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1895) took 2738 ms. Trace:
I20260709 13:13:24.952427 22762 rpcz_store.cc:276] 0709 13:13:22.213415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.342582 (+129167us) service_pool.cc:224] Handling call
0709 13:13:22.343416 (+   834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.930640 (+1587224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.951679 (+1021039us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400002dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:24.951898 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3521363328}
W20260709 13:13:24.953145 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.948858 27160 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:24.926319 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:24.927078 27003 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:24.966374 22907 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.011s	sys 0.027s
I20260709 13:13:24.966483 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:24.967536 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.927525 27247 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:24.870806 27235 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:13:24.953706 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:24.957233 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:24.957736 22834 pending_rounds.cc:85] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 5
W20260709 13:13:24.958022 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.925215 23464 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:25.007567 22907 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.015s	sys 0.027s
I20260709 13:13:24.910385 27347 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:24.970016 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:25.008936 22806 leader_election.cc:400] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11.
I20260709 13:13:25.011382 22806 leader_election.cc:403] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:13:25.012048 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.013660 22834 pending_rounds.cc:107] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:13:25.013985 22834 raft_consensus.cc:2889] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:25.009650 22762 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.990968 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:24.985320 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:25.030138 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:24.912571 27219 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:25.034408 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:25.037472 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:25.038039 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80737.5us
I20260709 13:13:25.016111 27329 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:25.010181 27437 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:25.008793 26948 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:25.069032 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:24.966127 27003 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:25.025573 27141 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11.
I20260709 13:13:25.025902 27405 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:13:25.028427 27347 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.144431 22835 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:25.115829 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:25.122905 22761 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.067s
W20260709 13:13:25.156064 22761 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.012s	sys 0.071s
W20260709 13:13:25.149550 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80632.2us
I20260709 13:13:25.149333 22762 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:25.160830 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.059163 27421 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:25.115983 27424 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:13:25.162540 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.173343 22762 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:25.174401 22914 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:25.125675 27444 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.154227 27066 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.038836 27327 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 13:13:25.076354 27329 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:25.187604 27329 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:25.177522 22914 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:13:25.148514 27003 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:25.159456 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.148378 26948 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:25.199802 26948 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:25.180156 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:25.158079 27235 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:25.201150 27235 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:25.201689 27235 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:25.077982 27219 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:25.187882 27364 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:25.186177 23464 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:25.210156 27219 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.212303 27235 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.212324 26948 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.220368 27003 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.222044 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:25.225108 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45097.4us
W20260709 13:13:25.253711 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94367.7us
I20260709 13:13:25.282971 22762 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:25.048389 27389 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.177388 27421 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:25.251984 26948 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.256399 27233 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:25.253386 27399 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:25.180894 27297 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:25.283952 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.283962 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:25.312811 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.254101 27380 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:25.297999 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.305101 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:25.312883 23464 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:25.292277 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.316306 22762 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11.
I20260709 13:13:25.317648 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.318602 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:25.181967 27261 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.320197 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:25.321252 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.226063 27066 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.222903 27444 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.330528 27440 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:25.359807 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:25.369383 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:25.370069 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.373464 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.375759 23459 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:25.379362 27450 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:13:25.379709 27455 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:13:25.381567 27446 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:13:25.382860 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1916) took 1030 ms. Trace:
I20260709 13:13:25.383271 22758 rpcz_store.cc:276] 0709 13:13:24.352904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.532945 (+180041us) service_pool.cc:224] Handling call
0709 13:13:24.554516 (+ 21571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.554894 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.554925 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.570465 (+ 15540us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.828493 (+258028us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:24.828542 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.828622 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.382600 (+553978us) raft_consensus.cc:1685] finished
0709 13:13:25.382626 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.382788 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2846,"raft.queue_time_us":11541,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":34253,"thread_start_us":254118,"threads_started":1,"wal-append.queue_time_us":500019}
I20260709 13:13:25.335328 27297 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.390089 27297 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.391242 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1900) took 2000 ms. Trace:
I20260709 13:13:25.396818 27261 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:25.385326 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.339643 27380 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:25.404866 23459 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:25.391742 22837 rpcz_store.cc:276] 0709 13:13:23.390461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.363483 (+973022us) service_pool.cc:224] Handling call
0709 13:13:24.385589 (+ 22106us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.423507 (+ 37918us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:24.424403 (+   896us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.775813 (+351410us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b1800421f00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:13:25.067189 (+291376us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.067693 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.067850 (+   157us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:25.067897 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.067990 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.390897 (+322907us) raft_consensus.cc:1685] finished
0709 13:13:25.391014 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.391168 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16877,"prepare.queue_time_us":286,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":92098,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":22947,"spinlock_wait_cycles":20509952,"thread_start_us":319266,"threads_started":2,"wal-append.queue_time_us":893853}
I20260709 13:13:25.410526 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:25.407478 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:25.367062 27452 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:13:25.345558 27160 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.411954 22762 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:25.414270 22757 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:25.332821 27399 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.418815 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.420939 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.421108 22757 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:25.422271 22762 raft_consensus.cc:2368] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:25.427788 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.428625 22758 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 8.
I20260709 13:13:25.429715 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.432516 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.433228 22762 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:25.412467 26948 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:25.430423 22758 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11. Re-sending same reply.
I20260709 13:13:25.442452 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.450304 27347 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:25.420440 27440 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:25.451192 27321 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:13:25.451663 27321 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.426028 27398 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.456032 27398 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.456813 22759 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:25.458099 27469 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:25.463429 27066 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:25.415596 27339 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:13:25.465051 27321 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:25.466495 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.412674 27421 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.467924 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.468642 22758 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:25.469911 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.474278 27444 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:25.474674 22758 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:25.438654 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:25.478127 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.470738 27380 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.479326 22757 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:25.468717 22762 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:25.483363 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.484031 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.484282 22757 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:25.484879 22762 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:25.479708 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 586) took 1123 ms. Trace:
I20260709 13:13:25.486009 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.486613 22761 rpcz_store.cc:276] 0709 13:13:24.356691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.550461 (+193770us) service_pool.cc:224] Handling call
0709 13:13:24.550765 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.550989 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:25.156561 (+605572us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.479183 (+322622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:25.479256 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:25.479333 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.479434 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.479625 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":213157,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"thread_start_us":896977,"threads_started":2}
I20260709 13:13:25.489243 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.490592 22761 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:13:25.489209 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.496207 22762 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
W20260709 13:13:25.500020 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.507125 22762 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:25.508023 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.508083 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1917) took 1141 ms. Trace:
I20260709 13:13:25.511173 22757 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:25.511262 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.512507 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1920) took 1113 ms. Trace:
I20260709 13:13:25.509814 22762 rpcz_store.cc:276] 0709 13:13:24.366650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.506249 (+1139599us) service_pool.cc:224] Handling call
0709 13:13:25.506801 (+   552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.507743 (+   942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.508024 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:25.513101 22757 rpcz_store.cc:276] 0709 13:13:24.399145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.509678 (+1110533us) service_pool.cc:224] Handling call
0709 13:13:25.510706 (+  1028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.512126 (+  1420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.512426 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:25.515846 22762 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:25.502828 26948 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:25.517379 27399 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.517185 26948 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.519470 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1914) took 1178 ms. Trace:
I20260709 13:13:25.519774 26948 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:25.505326 27470 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:13:25.505060 27462 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:13:25.520800 27432 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.804s	user 0.002s	sys 0.001s
I20260709 13:13:25.522132 27329 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.525827 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1923) took 1104 ms. Trace:
I20260709 13:13:25.526424 22762 rpcz_store.cc:276] 0709 13:13:24.421752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.515158 (+1093406us) service_pool.cc:224] Handling call
0709 13:13:25.515574 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.516391 (+   817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.516605 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:25.519892 22760 rpcz_store.cc:276] 0709 13:13:24.340893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.525869 (+184976us) service_pool.cc:224] Handling call
0709 13:13:24.548651 (+ 22782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.549388 (+   737us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:24.742242 (+192854us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.186820 (+444578us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.187412 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.187574 (+   162us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:25.187626 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.188414 (+   788us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.519232 (+330818us) raft_consensus.cc:1685] finished
0709 13:13:25.519256 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.519404 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21615,"prepare.queue_time_us":180675,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":65239,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":20780,"thread_start_us":467097,"threads_started":2,"wal-append.queue_time_us":598687}
I20260709 13:13:25.530521 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1925) took 1015 ms. Trace:
I20260709 13:13:25.503163 27347 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.530757 27329 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.532809 22762 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:25.531708 22760 rpcz_store.cc:276] 0709 13:13:24.514980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.529544 (+1014564us) service_pool.cc:224] Handling call
0709 13:13:25.529920 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.530124 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.530157 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:25.530211 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:25.530252 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.530312 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.530455 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:25.536753 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 226954us
I20260709 13:13:25.538466 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1924) took 1063 ms. Trace:
W20260709 13:13:25.533653 27420 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.140s	user 0.009s	sys 0.006s
I20260709 13:13:25.539016 22762 rpcz_store.cc:276] 0709 13:13:24.475309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.531935 (+1056626us) service_pool.cc:224] Handling call
0709 13:13:25.532531 (+   596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.535972 (+  3441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.538382 (+  2410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:25.542776 22806 leader_election.cc:400] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11.
I20260709 13:13:25.544422 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:25.553300 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.559450 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.535049 27347 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.551785 27440 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:25.572139 22806 leader_election.cc:400] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:25.572613 22806 leader_election.cc:403] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:25.576946 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 700) took 1952 ms. Trace:
W20260709 13:13:25.569284 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 158922us
I20260709 13:13:25.577574 23465 rpcz_store.cc:276] 0709 13:13:23.625112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.625384 (+   272us) service_pool.cc:224] Handling call
0709 13:13:23.642773 (+ 17389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.954801 (+312028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.954857 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.957206 (+  2349us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.110580 (+153374us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:24.110627 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.110719 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.441736 (+1331017us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:25.571788 (+130052us) raft_consensus.cc:1685] finished
0709 13:13:25.571814 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.576867 (+  5053us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":169850,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":28455,"spinlock_wait_cycles":8204032,"thread_start_us":432561,"threads_started":2,"wal-append.queue_time_us":355683}
I20260709 13:13:25.563679 27327 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:25.587502 22760 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:25.585525 22761 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
W20260709 13:13:25.592538 27474 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:25.599246 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:25.570086 27440 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:25.610738 22760 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:13:25.617276 22760 pending_rounds.cc:85] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:13:25.618194 22760 pending_rounds.cc:107] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:13:25.618651 22760 raft_consensus.cc:2889] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:25.589834 27437 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:25.575601 27389 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:25.607518 27003 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:25.619174 27440 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:25.625183 27454 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:25.661149 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:25.661731 22834 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:25.662109 27321 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:25.662835 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:25.662937 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:25.663687 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 590) took 1305 ms. Trace:
I20260709 13:13:25.663797 22841 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:25.664103 22759 rpcz_store.cc:276] 0709 13:13:24.358076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.887227 (+529151us) service_pool.cc:224] Handling call
0709 13:13:24.887792 (+   565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.457286 (+569494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.663439 (+206153us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54001004d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:25.663625 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308105472}
W20260709 13:13:25.666294 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1728) took 1795 ms (client timeout 1529 ms). Trace:
W20260709 13:13:25.666795 22913 rpcz_store.cc:269] 0709 13:13:23.870708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.870906 (+   198us) service_pool.cc:224] Handling call
0709 13:13:25.666083 (+1795177us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54000cf654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:25.666220 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199799168}
I20260709 13:13:25.669134 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:25.682648 27475 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:13:25.683235 27347 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:25.640869 27233 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:25.663839 22832 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:25.686654 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1915) took 1340 ms. Trace:
I20260709 13:13:25.665450 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.687085 22764 rpcz_store.cc:276] 0709 13:13:24.346605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.527272 (+180667us) service_pool.cc:224] Handling call
0709 13:13:24.667580 (+140308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.668313 (+   733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.668348 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.668925 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.969575 (+300650us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:24.969638 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.969734 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.385615 (+415881us) raft_consensus.cc:1685] finished
0709 13:13:25.385664 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.686348 (+300684us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540035e650. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:25.686581 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1325,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":321889024,"thread_start_us":299974,"threads_started":1,"wal-append.queue_time_us":582802}
I20260709 13:13:25.688490 22759 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:25.692927 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 120930us
I20260709 13:13:25.693920 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.698292 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:25.693607 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1919) took 1298 ms. Trace:
I20260709 13:13:25.700538 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:25.702600 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1930) took 1082 ms. Trace:
I20260709 13:13:25.703941 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.712630 22759 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:25.713089 22764 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:25.714190 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.699131 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.703310 22758 rpcz_store.cc:276] 0709 13:13:24.394904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.504686 (+1109782us) service_pool.cc:224] Handling call
0709 13:13:25.505122 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.507509 (+  2387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.507543 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.525487 (+ 17944us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.693116 (+167629us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:25.693183 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.693286 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.693355 (+    69us) raft_consensus.cc:1685] finished
0709 13:13:25.693371 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.693528 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1960,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9063,"thread_start_us":156691,"threads_started":1,"wal-append.queue_time_us":136615}
I20260709 13:13:25.715904 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.716559 22758 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:25.717880 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:25.718609 22764 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:25.718649 23460 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:25.719281 23460 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:25.720227 23460 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.722693 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:25.723892 27447 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:25.725308 27447 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:25.728040 23460 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:25.735267 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:25.743201 27287 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.748092 27378 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:25.750912 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 588) took 1393 ms. Trace:
I20260709 13:13:25.705006 27389 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:25.702579 27003 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:25.751461 22765 rpcz_store.cc:276] 0709 13:13:24.357446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.709478 (+352032us) service_pool.cc:224] Handling call
0709 13:13:24.709994 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.711648 (+  1654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.711681 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.711747 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:24.711788 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.750631 (+1038843us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b5400222ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:25.750701 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.750844 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3116928}
I20260709 13:13:25.701927 27444 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:13:25.755795 27444 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:25.765071 22836 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1685) took 2185 ms (client timeout 1643 ms). Trace:
W20260709 13:13:25.778281 22836 rpcz_store.cc:269] 0709 13:13:23.579974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.804682 (+224708us) service_pool.cc:224] Handling call
0709 13:13:25.764844 (+1960162us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54000a5d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:25.764970 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283349376}
I20260709 13:13:25.791337 22758 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:25.720714 22764 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:25.776574 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:25.804109 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:25.806074 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:25.809677 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:25.810993 22839 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:25.756755 27444 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:25.817466 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:25.815797 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 115325us
I20260709 13:13:25.752003 27389 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:25.820436 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.856477 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:25.858319 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:25.858040 27489 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:25.860564 27482 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:25.866420 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2564) took 1730 ms (client timeout 1679 ms). Trace:
I20260709 13:13:25.861063 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:25.872557 22763 rpcz_store.cc:269] 0709 13:13:24.136074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.149779 (+ 13705us) service_pool.cc:224] Handling call
0709 13:13:25.866125 (+1716346us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400311d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:25.866326 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806706432}
I20260709 13:13:25.860811 27437 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:25.870815 27031 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:25.878312 27031 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:25.879141 27031 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:25.874897 27440 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.881973 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 705) took 1349 ms. Trace:
I20260709 13:13:25.882879 27444 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:25.882390 23457 rpcz_store.cc:276] 0709 13:13:24.532315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.533567 (+  1252us) service_pool.cc:224] Handling call
0709 13:13:24.534109 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.586795 (+ 52686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.586834 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:24.586883 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:24.586924 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.881673 (+1294749us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b5400514854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:25.881738 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.881907 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120464640}
I20260709 13:13:25.852306 27389 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:25.877199 27482 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:25.853819 27287 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:25.899438 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:25.889701 27444 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:25.903376 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:25.923523 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.826499 27399 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:25.894148 27187 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:13:25.818753 27378 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:25.911651 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:25.890978 27482 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:25.962441 27485 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:25.712334 22762 rpcz_store.cc:276] 0709 13:13:24.620763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.549573 (+928810us) service_pool.cc:224] Handling call
0709 13:13:25.549935 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.550192 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:25.554672 (+  4480us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.702172 (+147500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:25.702247 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:25.702303 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.702386 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.702529 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9232,"prepare.queue_time_us":190,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":11531,"thread_start_us":137802,"threads_started":1}
I20260709 13:13:25.936298 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:25.966437 22836 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:25.955953 27287 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:25.983863 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:25.986495 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:25.989087 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:25.994099 22839 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:25.959421 27482 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:25.815120 27003 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.006215 27003 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:25.980628 27454 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.014021 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.026250 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.034142 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.034916 23459 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:26.036779 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:26.041896 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:26.015713 27287 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.024456 27440 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.008021 27378 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:13:26.047901 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 325296us
I20260709 13:13:26.049747 27489 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:26.056701 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 195748us
I20260709 13:13:26.056701 27399 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:26.062007 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.072284 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:26.074474 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:26.082989 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 531) took 1847 ms (client timeout 2199 ms). Trace:
I20260709 13:13:26.064179 27399 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:26.084530 23462 rpcz_store.cc:269] 0709 13:13:24.235464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.235650 (+   186us) service_pool.cc:224] Handling call
0709 13:13:26.079923 (+1844273us) spinlock_profiling.cc:243] Waited 88.8 ms on lock 0x7b5400735fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:26.082914 (+  2991us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204311424}
I20260709 13:13:26.088215 27233 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:26.088759 27003 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.078824 27440 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:26.088976 27233 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.091668 27233 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:13:26.092398 27399 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.094560 27399 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:26.094794 27428 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:26.095045 27440 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:26.095403 27428 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:26.095899 27428 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.096251 27440 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.099040 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.075312 27287 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.061062 27421 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:13:26.102108 22839 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:13:26.116237 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:26.120352 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:26.121094 27426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.393s	user 0.002s	sys 0.011s
I20260709 13:13:26.122110 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:26.124929 27490 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:13:26.130093 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 836) took 1231 ms. Trace:
I20260709 13:13:26.130798 23458 rpcz_store.cc:276] 0709 13:13:24.898835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.899025 (+   190us) service_pool.cc:224] Handling call
0709 13:13:26.083538 (+1184513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.083799 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.083835 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.083889 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:26.083925 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.083988 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.129875 (+ 45887us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400735fd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:26.130019 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2723274112}
I20260709 13:13:26.135494 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:26.137449 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:26.117756 27003 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:26.140064 27287 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:26.146042 27364 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:26.123162 27426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.406s	user 0.002s	sys 0.011s
I20260709 13:13:26.154000 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2587) took 1195 ms. Trace:
I20260709 13:13:26.123518 27410 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:26.159452 27410 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:26.160030 27410 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:26.135195 27437 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.154137 26948 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.173933 27429 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:13:26.138927 27489 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.141376 27447 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:13:26.157359 27287 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:26.185341 27287 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:26.156697 22760 rpcz_store.cc:276] 0709 13:13:24.958886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.622370 (+663484us) service_pool.cc:224] Handling call
0709 13:13:25.634301 (+ 11931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.645947 (+ 11646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.645982 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.646535 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:13:26.062270 (+415735us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:26.138038 (+ 75768us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.153473 (+ 15435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.153623 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.153721 (+    98us) raft_consensus.cc:1685] finished
0709 13:13:26.153740 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.153929 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15551,"prepare.queue_time_us":12781,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1045,"raft.queue_time_us":16589,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":17373,"thread_start_us":381079,"threads_started":2,"wal-append.queue_time_us":415274}
I20260709 13:13:26.186852 27031 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.190128 22834 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:26.190853 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.191871 22766 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:26.189240 27482 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.193221 27389 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:26.195004 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.195056 27482 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.202117 27369 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:26.195654 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:26.203590 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.210371 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.211537 22766 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:26.195458 27389 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:26.223670 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48042.6us
I20260709 13:13:26.223766 22766 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:26.227799 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 1.24259e+06us
I20260709 13:13:26.229019 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.229327 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 574) took 2162 ms. Trace:
I20260709 13:13:26.229883 22911 rpcz_store.cc:276] 0709 13:13:24.066950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.566730 (+499780us) service_pool.cc:224] Handling call
0709 13:13:24.607048 (+ 40318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.607300 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:24.611533 (+  4233us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.228876 (+1617343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.229028 (+   152us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:26.229080 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.229147 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.229271 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1193848,"prepare.queue_time_us":14559,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"thread_start_us":426786,"threads_started":1,"wal-append.queue_time_us":552924}
I20260709 13:13:26.232421 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.233284 22766 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 8: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:26.233399 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:26.215663 27369 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:26.235649 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:26.225175 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.236958 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.242405 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.237183 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.243577 22764 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:26.244333 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2710) took 1601 ms. Trace:
I20260709 13:13:26.244843 22910 rpcz_store.cc:276] 0709 13:13:24.643156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.932735 (+289579us) service_pool.cc:224] Handling call
0709 13:13:24.933076 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.933297 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:24.937622 (+  4325us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.243687 (+1306065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.243907 (+   220us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:26.243992 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.244098 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.244265 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":986674,"prepare.queue_time_us":3852,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"thread_start_us":318941,"threads_started":1,"wal-append.queue_time_us":456170}
I20260709 13:13:26.234314 27284 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:26.243283 22763 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:26.247404 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.249383 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.247030 27284 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:26.252079 27284 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:26.255280 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.256076 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:26.257205 23460 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:26.259629 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.260392 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.261374 23458 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:26.263799 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.264766 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.236637 27294 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.238504 27219 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:26.277375 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.284654 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:26.289920 22760 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:26.294696 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:26.298110 23465 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:26.300328 22730 connection.cc:256] RPC call timeout handler was delayed by -1.2051204950000738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:13:26.247608 26948 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:26.247721 27031 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.315393 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.264552 27385 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.269979 27163 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.264890 27329 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:26.325940 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59240.6us
I20260709 13:13:26.322203 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1740) took 2335 ms. Trace:
I20260709 13:13:26.330176 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.331132 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.333053 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:26.334684 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:26.334933 27507 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:13:26.336592 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.329249 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.243719 27369 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.282368 27247 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:26.328326 22907 rpcz_store.cc:276] 0709 13:13:23.987155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.987506 (+   351us) service_pool.cc:224] Handling call
0709 13:13:23.988305 (+   799us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:23.992398 (+  4093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.992445 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:13:24.014731 (+ 22286us) log.cc:844] Serialized 78 byte log entry
0709 13:13:24.183079 (+168348us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:25.008365 (+825286us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.009350 (+   985us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.865132 (+855782us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18003f92e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:26.226549 (+361417us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b18003f92e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:13:26.274114 (+ 47565us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.321702 (+ 47588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.321819 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.321878 (+    59us) raft_consensus.cc:1685] finished
0709 13:13:26.321896 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.322108 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":453844,"prepare.queue_time_us":330846,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":92,"raft.queue_time_us":1217577,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":19980,"spinlock_wait_cycles":67999872,"thread_start_us":908178,"threads_started":3,"wal-append.queue_time_us":253648}
I20260709 13:13:26.301116 27235 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.344898 22836 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:26.297734 27272 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:26.353549 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.346673 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.349246 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:26.351660 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.355033 22832 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:26.356659 27247 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.358083 26948 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:26.358597 27410 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.359352 26948 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.360349 27247 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.317481 27378 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:26.361389 27272 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.361948 27378 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.363075 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.363752 22837 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:26.363893 27272 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:26.364516 27378 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.364720 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:26.354768 22836 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:26.366701 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.367727 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:26.374902 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 141638us
I20260709 13:13:26.376551 22837 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 8: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:26.284439 27429 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:13:26.320788 26952 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:26.361128 27410 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.381271 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.381686 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.383960 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:26.385847 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.391866 27247 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:13:26.345042 27163 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.400750 27163 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.386879 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.322854 27259 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:26.298019 27066 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:13:26.366055 27424 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:13:26.397943 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58832.1us
W20260709 13:13:26.392652 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98873us
I20260709 13:13:26.390753 27378 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:26.391104 27235 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.439944 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.427403 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.327692 27385 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.436192 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:26.345208 27329 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:26.436723 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55150.6us
I20260709 13:13:26.394098 26952 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:26.397639 27440 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.457350 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:26.455389 27444 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.010s
I20260709 13:13:26.459371 27247 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:26.459810 27444 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:26.460314 23457 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:13:26.463092 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:26.439675 27272 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.409829 27428 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.406528 27259 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.455389 26952 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:26.480324 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:26.459828 27247 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.487586 22911 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:26.452453 27235 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:26.484678 27510 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 13:13:26.461691 27293 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:13:26.500626 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 78378.1us
I20260709 13:13:26.484038 27329 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:26.492640 27247 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:26.503903 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.461079 27440 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:26.543856 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.544293 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.467444 27163 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:13:26.494700 27399 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.542007 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:26.548038 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 225766us
I20260709 13:13:26.558295 22911 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:26.588933 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:26.474570 27516 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 13:13:26.619187 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 152939us
W20260709 13:13:26.601670 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.549602 27437 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.640138 22907 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:26.659629 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:26.662053 27235 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.655664 27369 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.558293 27329 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:13:26.547935 27522 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:13:26.667959 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:26.670228 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1909) took 1702 ms. Trace:
I20260709 13:13:26.672792 27524 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:13:26.691694 27272 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:26.692680 22762 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:26.671509 27518 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:13:26.696120 27163 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:26.696360 27160 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:26.696583 27163 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:26.696883 27160 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:26.701116 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:26.702234 22766 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.677048 22833 rpcz_store.cc:276] 0709 13:13:24.968074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.968261 (+   187us) service_pool.cc:224] Handling call
0709 13:13:25.026644 (+ 58383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.040017 (+ 13373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.040052 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.057378 (+ 17326us) log.cc:844] Serialized 26 byte log entry
0709 13:13:26.198217 (+1140839us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b180044b480. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:13:26.234204 (+ 35987us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:26.234245 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.234323 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.643490 (+409167us) raft_consensus.cc:1685] finished
0709 13:13:26.643525 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.670148 (+ 26623us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12815,"raft.queue_time_us":19164,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":15532,"spinlock_wait_cycles":7070976,"thread_start_us":35768,"threads_started":1,"wal-append.queue_time_us":1195723}
I20260709 13:13:26.687659 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2719) took 1329 ms. Trace:
I20260709 13:13:26.705472 22914 rpcz_store.cc:276] 0709 13:13:25.357860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.358065 (+   205us) service_pool.cc:224] Handling call
0709 13:13:25.358506 (+   441us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:26.252408 (+893902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.252441 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:26.253084 (+   643us) log.cc:844] Serialized 52 byte log entry
0709 13:13:26.434658 (+181574us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:26.434751 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.434871 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.687261 (+252390us) raft_consensus.cc:1685] finished
0709 13:13:26.687301 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.687548 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":919505,"raft.queue_time_us":898419,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":34175,"thread_start_us":181090,"threads_started":1,"wal-append.queue_time_us":294060}
I20260709 13:13:26.616379 27399 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.665405 27440 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.697665 27163 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.732146 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
W20260709 13:13:26.733150 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 191253us
W20260709 13:13:26.735164 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 177829us
I20260709 13:13:26.675380 27369 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:26.744632 27448 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.448s	user 0.002s	sys 0.000s
I20260709 13:13:26.746003 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1750) took 2579 ms. Trace:
I20260709 13:13:26.747370 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:26.746441 22915 rpcz_store.cc:276] 0709 13:13:24.166744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.573958 (+407214us) service_pool.cc:224] Handling call
0709 13:13:24.574459 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.589441 (+ 14982us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.589490 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.590205 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.016593 (+426388us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:25.016710 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.016800 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.743986 (+1727186us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:26.745658 (+  1672us) raft_consensus.cc:1685] finished
0709 13:13:26.745687 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.745934 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14444,"raft.queue_time_us":16336,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"spinlock_wait_cycles":35072256,"thread_start_us":425874,"threads_started":1,"wal-append.queue_time_us":587701}
I20260709 13:13:26.747380 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:26.751035 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:26.751574 27458 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.593s	user 0.005s	sys 0.001s
I20260709 13:13:26.751626 26952 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.686357 27428 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.742035 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.739912 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.752481 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.754976 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.755425 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.757316 22914 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:26.726343 27454 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:13:26.716853 27385 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:26.716226 27522 consensus_queue.cc:1243] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:26.757936 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:26.792723 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:26.794762 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:26.794148 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 366884us
I20260709 13:13:26.796363 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:26.761826 27410 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:26.762930 27369 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:26.762123 27440 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:26.761691 27533 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:26.762899 27498 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.795449 27428 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:26.789319 27287 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.603s	user 0.014s	sys 0.006s
W20260709 13:13:26.807145 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 169789us
I20260709 13:13:26.811406 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.795014 27524 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:26.824716 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.809419 27369 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.810403 27518 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:26.804828 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:26.816924 27520 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:26.757946 27160 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.816743 27187 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:13:26.840467 27447 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:26.842288 27321 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:26.845335 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2588) took 1886 ms. Trace:
I20260709 13:13:26.846097 26952 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:26.838806 27369 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:26.813555 27533 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:26.820652 27529 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:26.853621 27530 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:26.800988 27434 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:13:26.822067 27428 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.788798 27527 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:13:26.827728 27440 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:26.863281 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68626.2us
I20260709 13:13:26.812855 27516 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:13:26.850023 27520 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:26.833864 27421 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:26.827073 27378 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:26.870965 22761 rpcz_store.cc:276] 0709 13:13:24.959275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.491317 (+532042us) service_pool.cc:224] Handling call
0709 13:13:26.845112 (+1353795us) spinlock_profiling.cc:243] Waited 73.2 ms on lock 0x7b54003403d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:26.845265 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168395136}
I20260709 13:13:26.824527 27431 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:13:26.836455 27427 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:26.855549 27284 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.603s	user 0.014s	sys 0.001s
I20260709 13:13:26.902568 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.836277 27493 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:13:26.898979 27428 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:26.919268 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.922170 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:26.829113 27287 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.874399 27187 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:26.907665 27440 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:26.907812 27247 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.870491 27447 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.870675 27489 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:13:26.872381 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58567.5us
I20260709 13:13:26.915572 22764 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:26.874249 27533 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:26.855798 27321 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:26.909288 27410 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:13:26.910087 27160 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.909473 27520 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:26.927001 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100767us
I20260709 13:13:26.925374 27399 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.928361 22765 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:26.934823 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.939142 22914 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:26.920950 27438 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:26.855283 27524 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:26.925293 27187 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:26.950331 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:26.956321 27447 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:26.957731 26952 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:26.961009 27247 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:26.968014 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.971306 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:26.971920 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:26.874601 27529 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:26.934880 27339 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.934690 27003 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:26.986508 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51778.5us
I20260709 13:13:26.922868 27284 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:26.947079 27382 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:13:26.993723 27426 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.994235 27522 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:27.013859 27484 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.142s	user 0.000s	sys 0.005s
I20260709 13:13:27.016923 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:27.010721 27515 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:27.027640 27515 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:26.902688 27530 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.002215 26952 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.028611 27515 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.031035 26952 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:26.983280 27493 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:13:27.037598 27433 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.038223 27433 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.015260 27003 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.991544 27489 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:13:27.039491 27003 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:27.030467 27515 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:27.043089 27444 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.001s
I20260709 13:13:27.043712 27444 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:27.043802 27426 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.049353 22763 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:27.052614 22762 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:27.053696 22766 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:27.054878 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62645.8us
W20260709 13:13:27.024930 27541 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.089s	user 0.001s	sys 0.001s
W20260709 13:13:27.026551 27540 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:13:27.047305 27542 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:13:27.062345 22832 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:26.993765 27378 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.064875 27431 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:27.055361 27247 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.079778 22832 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:27.084736 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:27.057302 27524 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.065464 27431 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.091010 27516 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:27.066085 27219 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.089128 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:27.058832 27444 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:13:27.059123 27447 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:13:27.032382 27529 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.097748 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:26.998631 27537 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:13:27.105042 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.085528 27493 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:27.064725 27354 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:27.066148 27247 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.094493 27431 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:27.118075 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.094453 27516 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.096356 27219 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:27.126250 23432 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.226s (SENT)
W20260709 13:13:27.126166 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109324us
W20260709 13:13:27.142992 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.987s (SENT) [suppressed 35 similar messages]
I20260709 13:13:27.115751 27529 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:27.123091 27436 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:27.092101 27524 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.135576 27516 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:27.169462 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67714.1us
I20260709 13:13:27.154217 27537 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:27.171926 27284 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.172112 27247 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.170846 27559 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:27.120399 27493 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.161468 27536 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:13:27.179163 27447 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:13:27.186707 27426 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.179631 27559 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:27.195802 27538 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:27.203025 27187 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.203845 27524 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.194938 27530 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.211443 27538 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.218566 27559 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.220345 27538 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.221169 27520 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.222272 27493 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.223682 27520 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.226987 27522 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:27.220381 27559 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:27.268215 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:27.272185 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.271611 27187 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:27.279589 22732 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.529s (SENT)
I20260709 13:13:27.211953 27428 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.290815 27160 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.291962 27160 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.293124 27428 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.300532 22835 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:13:27.303370 27160 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.304149 27160 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:27.304399 22732 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.601s (SENT)
I20260709 13:13:27.229472 27533 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.308748 27524 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.308538 27522 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:27.323577 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:27.326328 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.326776 27533 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.327023 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.328656 22766 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:27.330886 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 346789us
I20260709 13:13:27.332196 27061 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:27.209084 27003 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.345948 27287 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.305400 27160 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:27.354246 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:27.308207 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 122563us
I20260709 13:13:27.362715 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.373907 27438 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.372731 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.374845 27438 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.376562 27327 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.377705 27327 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.355610 27537 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:27.382102 27537 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:27.366324 27160 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.388031 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.390383 27327 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.386258 27537 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.392406 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:27.394739 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.398393 22907 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:13:27.400229 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:27.401235 22909 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:27.391021 27557 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.406451 27557 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.409255 27382 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.410391 27426 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.410475 27382 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.411769 27426 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.413971 27530 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:27.414042 27524 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.415274 27163 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:27.415089 27530 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.418370 22762 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:27.372080 27003 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.414976 27557 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.403403 27522 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.424840 27530 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.424065 27438 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.433529 27537 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:27.437129 27524 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.447587 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:27.449923 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:27.448984 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.455953 22761 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 13:13:27.434237 27327 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:27.472057 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.473340 22907 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:27.475841 22759 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:27.451936 27426 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.459703 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1925) took 1474 ms. Trace:
I20260709 13:13:27.477788 22841 rpcz_store.cc:276] 0709 13:13:25.985457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.986157 (+   700us) service_pool.cc:224] Handling call
0709 13:13:27.459104 (+1472947us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54000c71d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:27.459397 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751876992}
I20260709 13:13:27.469254 22911 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
W20260709 13:13:27.453881 27552 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.247s	user 0.000s	sys 0.005s
I20260709 13:13:27.480285 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:27.482300 27382 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.484134 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:27.485996 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.458695 27538 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:27.490916 27532 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.670s	user 0.002s	sys 0.000s
I20260709 13:13:27.497797 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.497408 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.498435 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.498677 22910 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:27.500932 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.502308 22906 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:27.503760 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.504689 22906 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:27.511482 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:27.512488 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.512504 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:27.497854 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:27.504247 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.513304 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:27.513041 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:27.522892 22908 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:27.524583 22906 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:27.525094 22906 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:27.526094 27498 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.526459 22911 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:13:27.527060 27498 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.532130 23457 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:27.519418 22838 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:27.558344 22761 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:27.550361 22909 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:27.567054 22841 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:27.566102 22760 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:27.573746 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:27.579223 22765 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:27.614705 22908 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:27.614742 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:27.616039 22838 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
W20260709 13:13:27.570472 27564 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:13:27.565788 27369 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.016s
I20260709 13:13:27.570978 27498 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.627862 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:27.563426 22906 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.566087 27522 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.581524 27399 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.643746 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.550971 27327 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:27.650970 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 139588us
I20260709 13:13:27.587423 27524 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.654294 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:27.655489 27529 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:27.655781 27235 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.656039 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:27.656414 27235 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.659919 27235 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.660995 22839 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 9: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:27.567198 27538 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.661018 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:27.662048 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:27.668819 22906 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:27.669859 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:27.672950 27259 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.674043 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:27.675047 22834 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:27.679543 27517 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.178s	user 0.000s	sys 0.004s
I20260709 13:13:27.682060 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:27.681866 27399 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.683836 23462 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:27.684309 23462 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:27.685334 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49775.6us
I20260709 13:13:27.685801 23462 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.687173 23462 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:27.689390 27233 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.689960 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.690146 27233 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.691478 22909 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:27.701800 22759 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:27.702314 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.703109 22762 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:27.704061 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:27.706311 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:27.634793 27369 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.708761 22906 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:13:27.703128 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.707054 23459 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:27.711694 22906 pending_rounds.cc:85] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
W20260709 13:13:27.723794 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69104.6us
I20260709 13:13:27.730338 22761 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:27.715085 27219 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.504s	user 0.004s	sys 0.016s
I20260709 13:13:27.736011 27327 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.735668 27219 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.736814 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:27.737886 27437 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.738564 27437 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.741816 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2624) took 1038 ms. Trace:
I20260709 13:13:27.742269 22763 rpcz_store.cc:276] 0709 13:13:26.703496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.703628 (+   132us) service_pool.cc:224] Handling call
0709 13:13:27.048806 (+345178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:27.049944 (+  1138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.741290 (+691346us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b540002e4d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:27.741570 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778824064}
I20260709 13:13:27.744571 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:27.748165 27516 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.006s
I20260709 13:13:27.749270 27321 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.750653 27321 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:27.751631 27538 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.752434 27498 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.753207 27498 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:27.735080 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.756873 23465 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:27.745440 22763 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:27.761466 22759 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:27.762226 27380 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:27.763420 27380 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.762478 27433 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:27.770910 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47428.4us
I20260709 13:13:27.777617 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.779819 22838 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:13:27.722736 22906 pending_rounds.cc:107] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:27.783593 22765 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:27.784200 22906 raft_consensus.cc:2889] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:27.787133 22765 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:27.799784 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2613) took 1542 ms. Trace:
I20260709 13:13:27.801374 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:27.803232 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70100.9us
I20260709 13:13:27.803506 22910 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:27.804378 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:27.805900 22766 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:27.806072 23458 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:27.809821 27498 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.811311 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:27.814253 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.800520 22758 rpcz_store.cc:276] 0709 13:13:26.254733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.262628 (+  7895us) service_pool.cc:224] Handling call
0709 13:13:26.263109 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:26.273113 (+ 10004us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:26.273171 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:26.273894 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:13:26.367521 (+ 93627us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:26.367575 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.367632 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.678013 (+1310381us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:27.707321 (+ 29308us) raft_consensus.cc:1685] finished
0709 13:13:27.707345 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.797371 (+ 90026us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9419,"raft.queue_time_us":10198,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":15010,"spinlock_wait_cycles":9572224,"thread_start_us":93238,"threads_started":1,"wal-append.queue_time_us":196896}
W20260709 13:13:27.817067 27559 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.587s	user 0.019s	sys 0.005s
I20260709 13:13:27.818079 22762 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:27.818434 27433 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:27.818779 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.819105 27433 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:27.819595 22907 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:27.820348 27433 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.821061 27399 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.822716 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:27.825043 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.826012 22907 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:27.827003 27538 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:27.827553 22757 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:27.828990 27380 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.830072 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:27.784742 27287 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.831681 27287 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.749022 27516 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:27.760802 27437 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:27.841053 23465 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:27.845835 23462 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:27.851087 23459 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:27.758002 27524 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.842592 23457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8"
dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
mode: GRACEFUL
new_leader_uuid: "776baf137f8745dd866379ca15ab8cee"
 from {username='slave'} at 127.0.0.1:33198
I20260709 13:13:27.855583 23457 raft_consensus.cc:606] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Received request to transfer leadership to TS 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:27.856047 23457 raft_consensus.cc:614] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:27.818531 27219 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.858906 22760 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:27.842923 27522 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:27.861166 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.861339 27558 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:27.861966 22759 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:13:27.866705 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:27.866864 22914 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:27.868947 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.879415 22832 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:13:27.882522 27421 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:13:27.882356 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:27.889111 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.893940 27219 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:27.862291 27522 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:27.900210 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104551us
I20260709 13:13:27.920316 27524 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:27.921474 27444 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:13:27.895082 27219 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:27.929214 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 124999us
I20260709 13:13:27.931229 27493 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:27.931704 27493 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:27.934228 22835 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:27.906059 27538 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:27.935621 27538 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.936849 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:27.937453 27428 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:27.906483 27522 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:27.941604 27431 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.011s
I20260709 13:13:27.938468 27428 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:27.936110 27538 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:27.938782 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:27.930306 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:27.978950 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:27.959439 27548 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:27.989395 22835 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:27.932907 27493 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:27.989078 22732 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.979s (SENT)
I20260709 13:13:27.974701 27066 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:28.001837 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.978107 27524 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.002825 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:27.973773 27380 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:28.006768 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 118617us
I20260709 13:13:28.001068 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:27.989144 27548 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:28.010212 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.974928 27431 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.000801 27066 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:28.018576 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 79908.8us
I20260709 13:13:28.009371 27160 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.011659 27399 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:28.013147 27548 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:28.030462 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.005513 27524 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.020831 27066 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:28.049623 27550 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:13:28.044027 27160 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.048203 27531 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:28.051718 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.053800 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:28.071081 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 110093us
W20260709 13:13:28.047468 27061 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.712s	user 0.016s	sys 0.003s
I20260709 13:13:28.067866 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:28.083091 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.046890 27380 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.086493 27380 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:28.047215 27516 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:28.098821 22911 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:13:28.098925 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 168594us
W20260709 13:13:28.102385 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 120159us
I20260709 13:13:28.102667 27431 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:28.106070 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:28.106802 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 83211.4us
I20260709 13:13:28.108299 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.112542 22834 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:28.114331 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.119331 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.120931 22764 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:28.122480 27219 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:28.122865 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 83729.4us
I20260709 13:13:28.124604 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.128254 27482 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:13:28.131489 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:28.128459 27571 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.380s	user 0.005s	sys 0.001s
I20260709 13:13:28.132861 27380 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:28.133868 27516 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:28.128296 27428 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.130200 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:28.137732 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:28.145584 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.110989 27534 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.146620 27160 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:28.148607 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.148660 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.149864 22915 raft_consensus.cc:2410] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:13:28.152379 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.140234 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.159348 27399 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.159242 27533 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:28.159893 27399 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.160328 27399 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:28.161736 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.163906 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.154881 27160 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.168515 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.171871 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.172451 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.170217 27003 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.184754 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.182482 22764 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:28.190078 27520 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.190197 27235 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:28.190766 27520 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.175474 27160 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.196614 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.198412 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.202116 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.197216 22761 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:28.181406 27321 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:28.181383 27437 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:28.207322 22906 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:13:28.209424 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.171365 27431 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:13:28.184042 27003 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:28.202222 27520 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.181661 27533 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:13:28.217932 27003 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.189494 27433 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:13:28.239809 22915 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:28.246580 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.247133 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.257987 22764 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:28.259616 27491 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.260066 27429 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:28.260658 27429 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:28.260582 27491 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.261264 27429 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:28.261920 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.263230 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.263465 27491 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.264477 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.264304 27066 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.266166 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.268097 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.271551 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.269086 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.272897 27321 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:28.273836 27321 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:28.274077 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.275337 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.257442 27524 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:28.275462 27321 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.276859 27524 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.277464 27524 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:28.279937 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.280427 27308 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:28.280840 27308 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.281174 27308 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:28.272701 27584 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:13:28.285349 27431 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:28.286697 27431 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:28.287082 27431 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:28.288520 27533 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:28.289179 27533 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:28.289737 27533 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:28.289897 22765 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:28.288226 27431 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.275643 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.305999 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:28.301546 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48034.5us
I20260709 13:13:28.323891 22911 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:28.317842 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:13:28.340938 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65413.5us
I20260709 13:13:28.306171 27369 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.348495 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:28.350081 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 98173.4us
I20260709 13:13:28.354955 27160 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.362653 27066 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.369059 27382 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:28.369940 27382 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:28.370110 27437 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:28.371251 27382 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.373646 27389 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:28.374204 27389 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.374634 27389 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:28.375200 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.385353 27429 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.391155 27287 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.407176 27558 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.408150 27558 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:28.410609 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 161990us
I20260709 13:13:28.411791 27558 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:28.411868 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.353262 27235 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:28.435215 27235 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:28.437590 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.438694 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.439208 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.440066 22907 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.440291 22912 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:28.440937 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.441416 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.441911 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.442770 22909 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.442679 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.443617 22912 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:13:28.443677 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.445013 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.445021 22912 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:13:28.445751 22912 pending_rounds.cc:85] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:13:28.395951 27061 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:28.426795 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:28.427274 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.427573 27428 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:13:28.425221 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.429303 27247 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:13:28.433826 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 162115us
I20260709 13:13:28.437812 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.419903 27233 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.265846 27548 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.422154 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.398362 22906 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:28.447722 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.458041 22912 pending_rounds.cc:107] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:28.463631 22912 raft_consensus.cc:2889] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:28.458825 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.462628 22841 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:28.405094 27431 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:28.468004 22915 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:28.470379 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58581.3us
I20260709 13:13:28.480491 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.481145 22910 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.481449 22908 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:28.482961 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67511.8us
W20260709 13:13:28.484453 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 212974us
I20260709 13:13:28.487417 22911 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:28.489128 22915 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:28.425182 27522 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.491473 23457 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:28.460829 27247 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:28.464265 27066 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.458563 27428 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:28.466511 27061 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:28.501895 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:28.500775 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:28.506160 27586 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:13:28.505775 27247 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.507426 27247 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:28.494997 27235 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.509838 27369 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:28.467471 27429 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.495159 27482 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.517851 27428 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.495201 27066 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:28.517858 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.527150 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.527992 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.528681 22906 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:28.532058 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.525405 22912 raft_consensus.cc:2376] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13. Re-sending same reply.
I20260709 13:13:28.528676 27066 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.537652 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.542909 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.543840 22910 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:28.525086 27287 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:28.550175 27548 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:13:28.549948 27287 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.552246 27287 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:28.552794 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.557742 27160 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:28.557943 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.558060 27369 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:28.558442 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.558733 22765 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:28.559612 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.559736 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.560031 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.560655 23464 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:28.560691 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.561866 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.562361 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.562999 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.564075 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.564640 22805 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.566006 27601 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.566505 22805 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.566782 27601 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:28.567534 27491 raft_consensus.cc:2764] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:13:28.568454 27160 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:28.561417 22915 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:28.523026 27591 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.542822 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.570204 27591 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:28.570742 27591 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.571063 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.572343 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.573679 22908 raft_consensus.cc:2410] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:13:28.543053 27428 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:28.575156 27061 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.576392 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.577013 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.538179 27482 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.569662 27160 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.571051 27601 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:13:28.569746 27594 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:28.580618 27594 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.565241 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.581012 27594 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:28.583418 27538 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 13:13:28.583977 27538 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.585656 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.586541 22910 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.588179 27308 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.585731 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.543188 27522 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:28.590938 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.544754 27595 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:13:28.592244 22915 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.548508 27533 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.543843 22912 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:28.592919 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:28.596953 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.593878 27595 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:28.594183 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 586) took 1201 ms. Trace:
I20260709 13:13:28.594393 22913 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:13:28.597980 22914 rpcz_store.cc:276] 0709 13:13:27.393168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.394043 (+   875us) service_pool.cc:224] Handling call
0709 13:13:28.593950 (+1199907us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54002107d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:28.594105 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068369280}
I20260709 13:13:28.598601 22909 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.599020 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:28.609681 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.612502 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:28.621291 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.602162 27520 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:28.624490 27520 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.624647 23460 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:28.624990 27520 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:28.612854 27287 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:28.629871 27287 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.630352 27287 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:28.629887 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.633139 27538 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.621162 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.634004 22914 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:28.634840 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.635764 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.636437 22914 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13. Re-sending same reply.
I20260709 13:13:28.637384 27219 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:28.652431 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.600150 27595 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:28.622258 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57585.5us
I20260709 13:13:28.637912 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.630149 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.615902 27548 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:13:28.624917 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.640672 27003 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.644786 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.633046 27066 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.646770 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.647122 22841 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:28.648039 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.648535 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.648917 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.649816 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.650141 22838 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:28.650723 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.657694 22910 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.659320 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.664741 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:28.651115 27539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.000s	sys 0.017s
I20260709 13:13:28.669864 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.622303 27524 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:28.669696 23462 pending_rounds.cc:85] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 9
I20260709 13:13:28.644165 27389 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.672480 23462 pending_rounds.cc:107] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:13:28.672830 23462 raft_consensus.cc:2889] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:28.674654 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.622151 27482 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.677623 23465 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:28.651813 27219 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:28.680688 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:28.671277 27539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.000s	sys 0.018s
I20260709 13:13:28.688422 22833 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:28.687454 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 76580.3us
I20260709 13:13:28.702991 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:28.706071 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81243.5us
W20260709 13:13:28.709707 22732 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.502s (SENT)
I20260709 13:13:28.714037 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.714787 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:28.715814 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.716815 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:28.717132 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.718117 22835 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55836
I20260709 13:13:28.718793 22835 raft_consensus.cc:606] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Received request to transfer leadership
I20260709 13:13:28.724503 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.725704 27595 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.727262 27595 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:28.710788 27399 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.000s
I20260709 13:13:28.731930 22835 raft_consensus.cc:614] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:28.739044 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.744231 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.745223 22908 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:28.750108 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.731657 27524 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:28.755208 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:28.756814 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.758262 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:28.759065 22911 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:28.738582 27421 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.761183 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.767375 22833 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:28.767074 27429 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:28.767184 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.770174 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:28.748943 27247 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:28.719437 27382 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.796159 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:28.783855 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 69913.5us
I20260709 13:13:28.796811 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.785445 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.789446 22911 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 13:13:28.742038 27160 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:28.715145 27594 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.809353 22908 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:28.796934 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.799517 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1921) took 4386 ms. Trace:
I20260709 13:13:28.789232 27003 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.755347 27529 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:28.738530 27399 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.760047 27601 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:28.813735 22912 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:28.808326 22765 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:28.815915 22832 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:13:28.773074 27308 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:28.816367 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.810985 22757 rpcz_store.cc:276] 0709 13:13:24.403563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.515106 (+1111543us) service_pool.cc:224] Handling call
0709 13:13:26.845949 (+1330843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:27.828103 (+982154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.089284 (+261181us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b54003403d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:28.758754 (+669470us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003403d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:28.789937 (+ 31183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5156633728}
I20260709 13:13:28.828599 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.829604 22760 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:28.832814 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:28.782644 27437 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.783783 27524 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:28.834082 22907 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:28.839210 22906 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:28.839816 23466 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:28.843153 23464 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:28.843211 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:28.843261 22763 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:28.844182 22911 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:28.826162 27596 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.188s	user 0.002s	sys 0.000s
I20260709 13:13:28.847954 26948 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:28.854014 27604 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:13:28.854508 26948 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.854738 27524 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.832352 27382 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.773000 27219 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.827641 27538 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.859827 27382 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:28.867904 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52506.8us
I20260709 13:13:28.869144 22915 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:13:28.865909 27421 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:28.868641 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 164618us
I20260709 13:13:28.822136 27533 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.894964 22757 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:28.896438 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.812961 27160 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:28.904459 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.904634 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.863381 27219 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.857468 26948 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.921006 22840 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:28.913852 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 1954) took 1429 ms. Trace:
I20260709 13:13:28.882339 27308 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.920778 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.921722 22759 rpcz_store.cc:276] 0709 13:13:27.484515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.762480 (+277965us) service_pool.cc:224] Handling call
0709 13:13:27.860349 (+ 97869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:27.862552 (+  2203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.913625 (+1051073us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b5400368b50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:28.913786 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224060672}
I20260709 13:13:28.846432 27437 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.924232 22760 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:28.843297 27529 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.940286 22911 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:28.941084 22832 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:28.882284 27538 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:28.945302 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:28.952422 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
W20260709 13:13:28.951876 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 229203us
I20260709 13:13:28.956283 22915 raft_consensus.cc:606] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:28.956858 22915 raft_consensus.cc:614] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:28.945336 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.918501 27235 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:28.966284 23458 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:28.971678 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54289.3us
W20260709 13:13:28.970338 22732 leader_election.cc:336] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.594s (SENT)
I20260709 13:13:28.981732 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.923903 27617 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:28.986490 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:28.984691 27437 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.982583 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:28.992726 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:28.996239 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 250720us
I20260709 13:13:28.934284 27308 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:28.997113 22832 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:29.009308 22834 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:28.945159 27520 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.007185 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:28.998853 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:29.013345 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55618.6us
I20260709 13:13:29.015753 27529 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:29.016520 27531 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.017064 27516 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:29.020177 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.029922 27620 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.030473 27160 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:28.993244 27533 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:29.033427 27520 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.033835 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:29.036747 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:29.038352 22912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:29.013180 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:29.007167 27612 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:13:29.041127 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:29.055892 22761 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.046198 22915 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:29.056252 22759 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:29.048403 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:29.008041 27539 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:29.057039 22834 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:29.052423 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.049863 22912 raft_consensus.cc:606] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.056500 22915 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:29.090622 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.092061 27623 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.039364 27516 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.132391 22839 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:29.134526 22840 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:29.058974 27399 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:29.096396 27548 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.150540 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.158229 22832 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:29.120390 27594 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:29.148536 27591 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.577s	user 0.016s	sys 0.000s
W20260709 13:13:29.174167 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 212546us
I20260709 13:13:29.147861 27444 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.179329 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:29.180300 23462 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:29.094403 27539 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.181203 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:29.092974 27533 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.198724 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.199792 22915 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.200820 27061 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.201658 27593 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.201939 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:29.203325 27622 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:13:29.203639 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:29.205063 27308 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:29.205576 27539 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.201107 22915 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.206049 27516 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.207255 27516 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:29.210037 27625 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:13:29.191218 27591 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.213750 27591 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.215560 27533 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.210146 27538 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:29.222230 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:29.247689 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:29.248405 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.245908 23458 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:29.250306 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 725) took 1041 ms. Trace:
I20260709 13:13:29.250694 27421 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:29.192281 27287 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.000s
I20260709 13:13:29.269613 27603 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:29.270146 27531 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:29.273037 23464 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:29.273717 23457 rpcz_store.cc:276] 0709 13:13:28.209286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.209468 (+   182us) service_pool.cc:224] Handling call
0709 13:13:28.212382 (+  2914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:28.493658 (+281276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.249843 (+756185us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400514854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:29.250179 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641215872}
I20260709 13:13:29.276113 22832 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:29.277192 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.279292 22834 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:29.280133 22840 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:29.280992 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.283046 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:29.284089 23458 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:29.286667 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.286137 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.289476 27431 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:29.282567 27287 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.292912 27287 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:29.294152 22761 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:29.294114 27603 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.297605 27531 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.300024 27531 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:29.304136 27621 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.171s
I20260709 13:13:29.305215 27621 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:29.305615 27621 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:29.306840 27621 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.308857 27621 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:29.311194 27594 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:29.312584 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:29.317050 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:13:29.283308 27428 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.708s	user 0.004s	sys 0.010s
W20260709 13:13:29.325053 27611 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.662s	user 0.004s	sys 0.005s
I20260709 13:13:29.325376 27482 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:29.327420 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:29.318321 27003 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:13:29.333626 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:29.328717 27389 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.329936 22807 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.297310 27603 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:29.337523 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:29.338564 22834 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:29.339382 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:29.340131 27531 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:29.340546 27522 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:29.343051 27522 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.343408 27522 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.344554 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.345899 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.342664 27531 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:29.346657 27531 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.346763 27066 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.347358 22912 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:29.348011 22912 raft_consensus.cc:606] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.345844 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.352733 27493 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.330142 27538 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:29.358340 27538 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:29.360177 22839 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:29.359931 27538 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.365832 27603 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:29.377589 27603 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:29.378003 27603 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.378880 27603 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.380805 27603 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.366710 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:29.383284 27061 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:29.384225 27061 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.368257 27558 raft_consensus.cc:993] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:29.368125 27529 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:29.390065 22915 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:13:29.391405 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72108.1us
I20260709 13:13:29.396714 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.397092 22761 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:29.394917 27382 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:29.400774 27529 raft_consensus.cc:1081] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:29.401094 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:29.403182 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:29.400797 22763 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:29.404366 27321 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:29.404852 27606 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:29.405278 23459 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:29.406853 22765 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:29.409425 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:29.385351 27061 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.410369 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.414839 22762 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:29.420763 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.423147 23461 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:29.414350 27534 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:13:29.431804 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 250628us
I20260709 13:13:29.415982 27382 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:29.416106 27321 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:29.434160 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.372912 27522 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.426481 26948 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.417820 27606 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.429422 27431 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.457127 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:29.459609 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 407253us
I20260709 13:13:29.441697 27382 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:29.436836 27534 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.472734 27531 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:29.473668 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.473946 27405 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.459686 27389 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:29.461431 27516 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:29.476282 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2745) took 1970 ms. Trace:
I20260709 13:13:29.479079 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:29.479646 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.464749 27601 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:29.480051 22913 rpcz_store.cc:276] 0709 13:13:27.499091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.499625 (+   534us) service_pool.cc:224] Handling call
0709 13:13:28.594098 (+1094473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:28.600355 (+  6257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.469751 (+869396us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002107d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:29.469890 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2515980928}
I20260709 13:13:29.467329 26948 raft_consensus.cc:2764] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:29.487234 23465 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:29.471629 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:13:29.488754 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78483us
I20260709 13:13:29.489151 27482 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:29.493144 27635 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:29.493850 27635 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:29.493781 27621 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:29.495656 27635 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:29.502344 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43716.9us
I20260709 13:13:29.505806 27636 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:29.506166 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.506834 27636 consensus_queue.cc:1243] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:29.507441 27636 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:29.507812 27636 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:29.489786 27219 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.003s
I20260709 13:13:29.508858 27636 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.518944 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.522450 27620 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.528172 27639 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:29.540555 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.541018 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.541885 27369 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:29.541646 27606 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.543422 27369 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:29.544852 27369 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:29.510187 27321 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.546598 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.546921 27321 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.547585 27620 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:29.491735 27389 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.552125 27389 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.554108 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:29.557964 27639 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:29.559253 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:29.559630 27593 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:29.560138 27636 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:29.513991 27534 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.562393 27531 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:29.562453 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:29.538023 27431 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.565793 27522 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:29.565917 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:29.567031 22759 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:13:29.567859 23463 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:13:29.545339 27369 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:29.581939 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1775) took 1142 ms. Trace:
W20260709 13:13:29.581753 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 75736.8us
I20260709 13:13:29.590822 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:29.592767 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49142.8us
I20260709 13:13:29.606596 22837 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:29.622821 22765 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:29.607025 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47858.3us
I20260709 13:13:29.627038 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:29.608155 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54123.5us
I20260709 13:13:29.631783 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:29.632572 22910 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.611923 27601 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.611923 27620 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:29.614835 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:29.621802 22764 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:29.638664 22912 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:29.639057 22760 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:29.626219 22907 rpcz_store.cc:276] 0709 13:13:28.439930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.440676 (+   746us) service_pool.cc:224] Handling call
0709 13:13:29.581756 (+1141080us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54001257d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:29.581877 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902561152}
I20260709 13:13:29.617353 27233 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:29.619767 27633 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:29.646559 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 167569us
I20260709 13:13:29.647770 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:29.660187 22913 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:29.648067 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:29.661006 22906 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.644430 22762 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:29.662082 23466 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:29.650156 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:29.608032 27160 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:13:29.661051 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.652333 27399 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.652249 27633 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:29.653986 27531 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.622365 27631 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:13:29.661664 27233 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:29.645080 27516 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:13:29.641047 27389 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:13:29.689697 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62764.3us
W20260709 13:13:29.690099 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99397.3us
I20260709 13:13:29.667148 27534 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:29.695473 22906 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:29.678995 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.696287 22765 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:29.675215 27529 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.700071 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.682068 27633 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:29.676998 27498 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:29.705736 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:29.705736 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:13:29.707978 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.677722 27621 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:13:29.712546 27640 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:29.705673 27529 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.716917 27639 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:29.717962 27613 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.719935 27382 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.747191 27640 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.748353 27640 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.722404 27482 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:29.723640 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:29.729836 27160 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:29.751257 27538 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:29.751674 27538 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.752197 27538 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.733637 27609 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:29.753321 27399 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:29.753683 27609 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:29.708638 27531 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:29.754587 27531 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.754792 27609 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.756158 27638 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:29.710592 27321 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:29.757225 27321 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:29.757956 26948 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.758419 27321 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.758544 26948 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.754849 27308 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:13:29.761101 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 1937) took 1045 ms. Trace:
I20260709 13:13:29.761647 27308 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:29.761540 22837 rpcz_store.cc:276] 0709 13:13:28.716143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.741633 (+ 25490us) service_pool.cc:224] Handling call
0709 13:13:29.760903 (+1019270us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540027ea54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:29.761038 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767980416}
I20260709 13:13:29.704351 27498 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.762301 27308 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.729925 27233 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.751053 27522 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:29.751718 27405 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:29.753746 27399 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:29.743496 27601 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.704291 27380 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 13:13:29.764297 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.736347 27606 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:29.706293 27618 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.746580 27639 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.730075 27369 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.746127 23466 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:29.749111 27382 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:29.789280 22763 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:29.766897 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55667.3us
I20260709 13:13:29.768550 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:29.751078 27160 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.763952 27493 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:13:29.753697 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.796161 22759 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:29.787088 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:29.796648 22759 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:29.763228 27308 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.757261 27638 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:29.781243 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 864) took 1056 ms. Trace:
I20260709 13:13:29.797087 22764 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:29.779987 27601 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:29.787559 22712 auto_rebalancer.cc:893] Removing replace marker failed: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:29.772064 27405 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.775000 27522 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:13:29.803057 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49470.1us
I20260709 13:13:29.808799 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.834590 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.778070 27531 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:29.839555 22712 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:13:29.774333 27529 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:29.840250 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 147871us
I20260709 13:13:29.840761 22913 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:29.803930 23460 rpcz_store.cc:276] 0709 13:13:28.719588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.720030 (+   442us) service_pool.cc:224] Handling call
0709 13:13:28.720407 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.572370 (+851963us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18004254a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:13:29.572548 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.572583 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.573464 (+   881us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.683373 (+109909us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:29.683407 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.689707 (+  6300us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.765704 (+ 75997us) raft_consensus.cc:1685] finished
0709 13:13:29.765736 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.775950 (+ 10214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18473,"raft.queue_time_us":94871,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":12572,"spinlock_wait_cycles":25995648,"thread_start_us":161285,"threads_started":2,"wal-append.queue_time_us":138127}
I20260709 13:13:29.851236 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:29.852383 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.862845 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:29.863027 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:29.795531 27613 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:29.842435 22712 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:13:29.802934 27638 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.839236 27516 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.841012 27522 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:29.867678 27531 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:29.787480 27618 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.908476 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.909682 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:29.930243 22837 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:29.868067 27380 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.875865 27498 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:29.838346 27620 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:13:29.938603 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.838755 27633 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:29.869657 27529 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.933625 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.944638 22836 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:29.933944 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.891726 27533 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.930895 22759 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:29.935747 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 101300us
I20260709 13:13:29.938252 22835 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:29.938185 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.877701 27308 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:29.921828 27558 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:29.878858 27613 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.939342 22839 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:29.949000 22910 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:29.965891 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:29.967989 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 124972us
I20260709 13:13:29.969666 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:29.938692 27431 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.892058 27482 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:13:29.938997 27498 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.943466 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:29.997092 22840 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:29.920812 27003 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:13:29.994076 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:29.952096 27233 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.002593 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:29.964540 27558 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.936479 27531 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.893190 27639 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:29.995048 27369 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:30.004586 27498 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:30.003994 27061 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.963596 27399 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.028759 27531 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:30.035607 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.075670 22764 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:30.036736 27219 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:30.042346 27639 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:30.044181 27618 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.045768 27613 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:30.045814 27529 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:30.004266 27431 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.049094 27534 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:29.991365 27638 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.045845 27656 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:13:30.051671 27647 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:30.053623 27649 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:13:30.054713 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 85175.5us
I20260709 13:13:30.096189 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:30.057924 27633 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.060822 26948 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.060726 27160 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.061972 27601 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:30.068073 22761 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:30.105371 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.014642 27558 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:30.067713 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.107275 22760 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:30.024690 27522 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.071265 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:30.072184 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.111106 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:30.113576 27613 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:30.080967 22759 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:30.048233 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:30.073979 27651 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.242s	user 0.000s	sys 0.002s
I20260709 13:13:30.113300 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:30.119782 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 71637.7us
I20260709 13:13:30.120843 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:30.086288 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:30.087260 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.011531 27233 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.091372 22837 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:30.079631 27219 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:30.129767 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:30.125335 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:30.132797 27369 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:30.134509 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:30.134810 27369 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:30.135479 27369 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.136966 27643 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:30.137229 27656 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:30.138841 27656 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:30.145735 27531 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:30.146263 27531 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:30.146716 27531 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:30.147533 27219 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.148891 27219 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:30.133945 27633 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:30.153208 27431 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.155376 27538 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.157869 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:30.131652 27635 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 13:13:30.160084 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46836.4us
I20260709 13:13:30.161149 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:30.161839 27635 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:30.162454 27635 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.148887 27405 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:30.166227 27635 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:30.166613 27595 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:30.167208 27595 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:30.168234 27595 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.131436 27522 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:30.170336 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.171003 22841 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
I20260709 13:13:30.172626 27369 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.131472 27534 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:30.178030 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:30.180166 22839 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:30.237746 27539 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:30.181484 27233 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:30.182019 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.137454 27643 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:30.186036 27638 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:30.247403 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.249740 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:30.189368 27529 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:30.190418 22841 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:30.192237 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.262029 22906 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:30.262125 22838 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:30.193444 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.195111 27538 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:30.196234 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66585.3us
I20260709 13:13:30.196838 27534 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:30.198055 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.198700 27618 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:30.199717 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.200850 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.201814 27516 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:30.273846 22911 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:30.203495 22758 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:30.206457 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.275023 22908 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:30.207784 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:30.202517 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:30.218361 27399 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:30.218627 27389 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:30.221194 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:30.279919 23465 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:30.225468 22910 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:30.228046 27609 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:30.284261 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.231748 27061 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:30.232295 26948 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:30.234771 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.188660 27380 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:30.255375 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59568.2us
I20260709 13:13:30.268923 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:30.270242 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:30.296090 22839 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:30.272225 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:30.282575 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:30.287096 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:30.298394 22915 raft_consensus.cc:606] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:30.298902 22915 raft_consensus.cc:614] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:30.287876 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:30.299479 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:30.264485 27603 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.272099 27640 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:30.287015 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:30.274271 27534 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.300318 23459 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:30.299674 23458 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:30.303260 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.311808 22759 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:30.299713 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:30.276422 27516 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:30.273131 27618 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:30.315621 23466 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:30.315961 22759 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:13:30.285537 27633 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:30.318951 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.323004 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:30.292323 27160 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:30.326406 22833 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:30.328783 22915 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:30.330482 22759 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:30.334275 22910 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:30.337924 27516 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.339538 27516 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:30.341818 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:30.343451 22759 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:13:30.343827 22759 pending_rounds.cc:85] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:13:30.344048 22759 pending_rounds.cc:107] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:13:30.344296 22759 raft_consensus.cc:2889] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:30.344719 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:30.345460 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.346335 22764 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:30.345860 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.347060 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.346611 22759 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:30.346266 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.347072 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.330195 27633 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.351604 27498 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.356950 27633 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:30.353147 27531 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.367501 27431 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:30.368263 27655 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.388019 22908 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:30.391091 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.369676 27643 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.371033 27618 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.409612 27618 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:30.412216 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:30.316238 27534 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:30.316521 27522 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:30.418196 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.417970 27522 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.419358 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:30.419926 22912 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:30.420783 27522 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:30.421605 22915 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:30.423686 22909 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:30.301041 26948 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.315267 27603 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.385039 27431 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:30.443398 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.447005 22759 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:30.335486 27640 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.453997 22908 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:30.458036 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:30.458858 22759 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:30.462613 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 192677us
I20260709 13:13:30.407887 27655 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.448632 27539 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.465595 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:30.467085 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:30.469036 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 156113us
I20260709 13:13:30.472416 22807 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:30.486222 22839 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:30.489938 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:30.523229 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.478104 27429 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:30.474102 27643 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:30.519830 27219 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:13:30.550268 27668 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:13:30.517232 27431 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.515157 27539 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:30.588522 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:30.673926 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 405071us
I20260709 13:13:30.489120 27389 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:13:30.478423 27405 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:30.804935 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.734050 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:30.489126 27639 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.731134 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:30.772694 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62168.4us
W20260709 13:13:30.809649 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 283772us
I20260709 13:13:30.809202 27405 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.741655 27429 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.769824 27529 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:30.821384 22759 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:30.822367 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:30.826119 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.827950 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:30.815971 27558 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.691s	user 0.017s	sys 0.003s
I20260709 13:13:30.830417 22913 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:30.810602 27595 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.819826 27522 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:30.825011 27405 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:30.844744 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:30.828914 27429 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:30.830624 27558 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.854528 22913 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:30.835175 27666 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:30.863787 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67956.3us
I20260709 13:13:30.875954 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:30.842401 27431 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:30.818492 27639 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:30.886181 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:30.828455 27623 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 13:13:30.887566 27516 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.548s	user 0.020s	sys 0.003s
I20260709 13:13:30.892524 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:30.894851 22914 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:30.879586 27666 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.924235 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:30.844652 27595 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:30.926795 27623 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:30.889038 27531 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:30.928121 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:30.930706 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53266.6us
I20260709 13:13:30.934571 27389 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:30.935359 27609 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.645s	user 0.017s	sys 0.003s
I20260709 13:13:30.928766 27640 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:30.938599 27534 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.525s	user 0.014s	sys 0.005s
I20260709 13:13:30.939597 27647 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:30.896670 27603 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:30.941977 27639 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:30.944525 27558 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:30.947757 27665 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.595s	user 0.000s	sys 0.003s
I20260709 13:13:30.949542 27516 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.922039 27606 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:30.954211 27663 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:13:30.956835 27498 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:30.958709 27638 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.716s	user 0.020s	sys 0.000s
I20260709 13:13:30.925777 27666 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:30.915369 27655 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:31.009270 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 90503.8us
I20260709 13:13:31.014715 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.015493 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.016629 22763 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:13:31.085291 22907 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:31.039388 27621 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.008076 22757 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:31.083753 22760 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:30.962946 27522 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:31.090147 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:31.016273 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:31.090950 22766 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.085122 27688 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:31.092712 27688 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.095597 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 637) took 1380 ms. Trace:
I20260709 13:13:31.095920 27673 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:13:31.096907 27595 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.085402 27516 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.087189 27606 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.098899 27539 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.001858 27389 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.099671 27606 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:31.100328 27680 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 13:13:31.099208 27618 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.689s	user 0.003s	sys 0.017s
I20260709 13:13:31.099718 26948 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.106097 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.109186 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:31.109848 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92122.9us
I20260709 13:13:31.110348 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.111512 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:31.112928 22758 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:31.115362 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:31.115756 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 99824.3us
I20260709 13:13:31.116427 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.117285 22758 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 13 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:13:31.096063 22765 rpcz_store.cc:276] 0709 13:13:29.714920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.733116 (+ 18196us) service_pool.cc:224] Handling call
0709 13:13:29.733903 (+   787us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:29.735344 (+  1441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.735415 (+    71us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:30.035354 (+299939us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:30.036105 (+   751us) log.cc:844] Serialized 52 byte log entry
0709 13:13:30.290417 (+254312us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:30.290478 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:30.290578 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:31.093441 (+802863us) raft_consensus.cc:1685] finished
0709 13:13:31.093465 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.095503 (+  2038us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":231,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":217,"spinlock_wait_cycles":16335232,"thread_start_us":247741,"threads_started":1,"wal-append.queue_time_us":435080}
I20260709 13:13:31.102229 27621 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:31.135808 22730 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.010s	sys 0.027s
W20260709 13:13:31.140792 22730 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.239s	user 0.011s	sys 0.027s
W20260709 13:13:31.141198 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 1.26541e+06us
I20260709 13:13:31.102240 27389 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.082858 27534 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:31.145473 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.099227 27516 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.150445 22912 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:31.152441 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:31.130573 27539 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:31.093174 27688 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:31.144991 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 638011us
I20260709 13:13:31.141037 27595 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:31.168138 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:31.157936 27617 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.164656 27693 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.177896 27522 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.186288 27621 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.166273 27539 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.187855 27405 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.190704 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2789) took 1014 ms. Trace:
I20260709 13:13:31.194312 27534 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.194871 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:31.195329 27399 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.908s	user 0.013s	sys 0.001s
I20260709 13:13:31.141336 27687 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:31.144155 27690 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:31.152117 27618 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.196944 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2782) took 1042 ms. Trace:
I20260709 13:13:31.197692 27688 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.202879 22766 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:31.147893 27655 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:13:31.210666 22765 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:31.224853 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.291617 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.292395 22839 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:31.294104 22834 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:31.295729 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.225647 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.297611 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.296990 22760 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:31.236510 22909 rpcz_store.cc:276] 0709 13:13:30.176190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.279676 (+103486us) service_pool.cc:224] Handling call
0709 13:13:30.299951 (+ 20275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:30.427501 (+127550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.190427 (+762926us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54001e32d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:31.190635 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283852544}
W20260709 13:13:31.236867 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 319518us
I20260709 13:13:31.249979 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.250113 22838 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:31.263950 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.301265 22840 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:31.266443 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.268294 22907 rpcz_store.cc:276] 0709 13:13:30.154625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.164542 (+  9917us) service_pool.cc:224] Handling call
0709 13:13:30.193279 (+ 28737us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:13:31.196762 (+1003483us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54000447d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:31.196876 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1914185344}
W20260709 13:13:31.207012 27593 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.012s	sys 0.008s
I20260709 13:13:31.240316 22915 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:31.304395 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.299775 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:31.305456 22836 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 11: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:31.307808 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.308952 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.309239 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.310276 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.310937 22915 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:31.311091 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:31.311973 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.307950 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.313334 22835 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:31.269100 27690 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.322829 22832 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:31.319267 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.323493 22837 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:31.235107 27595 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.320364 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.236325 27617 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:31.313812 27593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.014s	sys 0.009s
I20260709 13:13:31.346191 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2793) took 1061 ms. Trace:
I20260709 13:13:31.348018 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.349290 22915 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:31.340441 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.352375 22836 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:31.339305 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.341477 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.335438 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:31.336380 22834 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:31.346593 22910 rpcz_store.cc:276] 0709 13:13:30.284724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.284857 (+   133us) service_pool.cc:224] Handling call
0709 13:13:30.285153 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:30.334780 (+ 49627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.345939 (+1011159us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b54003ca0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:31.346129 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112508288}
I20260709 13:13:31.347038 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:31.368316 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104447us
I20260709 13:13:31.367806 22765 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:31.370966 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.358309 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.316474 27688 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:31.374359 22834 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:31.376346 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.239323 27219 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:31.234876 27693 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.407662 23459 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:31.377115 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.378281 23460 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:31.375201 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.379345 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:31.375677 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.409168 23461 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:31.331746 27699 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.407810 22836 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:13:31.339988 27389 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:31.331984 27534 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.269343 27687 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.409623 23463 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:31.334296 27690 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.411074 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.472380 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.472724 23461 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
I20260709 13:13:31.301833 27618 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.377202 27621 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:31.432396 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.463263 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 122164us
I20260709 13:13:31.439572 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:31.468729 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89995.1us
I20260709 13:13:31.355732 27695 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:31.408628 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 246928us
I20260709 13:13:31.410120 27061 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:31.476215 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.408591 27160 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.415021 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.372964 27529 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:13:31.484311 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.484930 27639 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:31.410300 27697 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:31.483315 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 106272us
I20260709 13:13:31.487902 23466 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:31.485285 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:31.501116 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.501745 27695 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.502285 27691 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:31.504078 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.488314 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:31.504954 27405 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.506443 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:31.496335 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.504343 27695 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.543787 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.544682 22908 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:31.546409 27691 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:31.379146 27700 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
W20260709 13:13:31.552103 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45786.7us
I20260709 13:13:31.554498 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:31.485857 27061 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.497974 27702 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.507222 27639 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.455507 27389 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.563449 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.577369 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.578064 22908 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:31.500087 27534 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.547760 27692 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:31.467473 27699 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.572748 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.498929 27621 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.563059 27061 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:31.528604 27160 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:31.592819 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.570346 27558 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:31.571038 27701 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:31.556977 26948 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:13:31.587818 27638 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:31.570535 27702 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.620718 27638 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:31.621614 27693 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.621805 27638 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.622301 27713 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:13:31.601410 27531 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:31.624140 27698 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:13:31.624384 27531 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.624953 27687 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:31.625219 27696 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:13:31.572400 27603 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.564003 27482 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:13:31.606266 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:31.624905 27531 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:31.604518 27701 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:31.616592 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107221us
I20260709 13:13:31.599534 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.623852 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2796) took 1287 ms. Trace:
I20260709 13:13:31.641397 22915 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:31.625306 23464 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:31.567803 27691 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:31.540503 27697 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.589536 27700 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:31.580740 27640 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:13:31.601193 27558 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.632184 22908 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:31.579272 27639 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:31.627060 27709 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.616447 22910 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:31.641829 22913 rpcz_store.cc:276] 0709 13:13:30.336283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.878855 (+542572us) service_pool.cc:224] Handling call
0709 13:13:30.899650 (+ 20795us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:31.083065 (+183415us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:31.195000 (+111935us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.486919 (+291919us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:31.506954 (+ 20035us) log.cc:844] Serialized 52 byte log entry
0709 13:13:31.507118 (+   164us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:31.507157 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.507231 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:31.623557 (+116326us) raft_consensus.cc:1685] finished
0709 13:13:31.623581 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.623786 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154749,"prepare.queue_time_us":113275,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":222915,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":14769,"thread_start_us":444489,"threads_started":2,"wal-append.queue_time_us":392457}
I20260709 13:13:31.594110 27219 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.632488 27617 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:31.652346 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76101.2us
I20260709 13:13:31.630293 27539 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.636724 27595 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.668779 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:31.648013 27666 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 13:13:31.671999 22915 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:31.673208 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.632632 27603 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.641886 27708 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:31.677435 27655 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:31.694069 27702 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.666846 27558 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:31.677604 27431 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:31.677058 27539 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.701990 27539 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.704376 27539 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.652004 27716 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.705878 27716 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.708184 27716 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.709676 27534 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.710284 27534 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.710752 27534 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.712775 27534 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.713735 27708 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.714372 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2786) took 1540 ms. Trace:
I20260709 13:13:31.714915 22914 rpcz_store.cc:276] 0709 13:13:30.173722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.173859 (+   137us) service_pool.cc:224] Handling call
0709 13:13:30.174287 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:30.896787 (+722500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.714022 (+817235us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54003c87d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:31.714288 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1656499712}
I20260709 13:13:31.716084 27708 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.713316 27534 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.716542 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:31.717089 27534 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.719005 27701 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.720541 27701 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:31.721117 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:31.723119 27677 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.916s	user 0.005s	sys 0.000s
I20260709 13:13:31.723364 27697 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.725746 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:31.650979 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.692314 22913 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:31.726409 22761 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:31.682600 27617 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.693466 27699 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.728101 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2787) took 1552 ms. Trace:
W20260709 13:13:31.731519 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 1989) took 1920 ms (client timeout 2010 ms). Trace:
I20260709 13:13:31.717388 22914 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
W20260709 13:13:31.733596 22757 rpcz_store.cc:269] 0709 13:13:29.768057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.768249 (+   192us) service_pool.cc:224] Handling call
0709 13:13:31.687859 (+1919610us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001b9c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:31.688033 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2793753344}
W20260709 13:13:31.736016 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 160385us
I20260709 13:13:31.731240 22906 rpcz_store.cc:276] 0709 13:13:30.175487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.268811 (+ 93324us) service_pool.cc:224] Handling call
0709 13:13:30.269164 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:30.286210 (+ 17046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:30.286256 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:30.295352 (+  9096us) log.cc:844] Serialized 26 byte log entry
0709 13:13:30.454389 (+159037us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:30.592564 (+138175us) log.cc:844] Serialized 14 byte log entry
0709 13:13:30.770462 (+177898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:30.770556 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:30.949889 (+179333us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b48000b0c58. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:13:31.722379 (+772490us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:31.727783 (+  5404us) raft_consensus.cc:1685] finished
0709 13:13:31.727816 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.728029 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216150,"prepare.queue_time_us":738852,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":5784,"raft.queue_time_us":92404,"raft.run_cpu_time_us":2988,"raft.run_wall_time_us":170763,"spinlock_wait_cycles":54711936,"thread_start_us":216786,"threads_started":2,"wal-append.queue_time_us":209090}
I20260709 13:13:31.748927 22906 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:31.751667 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100765us
I20260709 13:13:31.755744 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:31.753532 27647 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:31.752656 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:31.768446 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.729274 27389 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.774991 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:31.783167 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.784380 27482 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:31.784848 27482 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.785202 27482 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:31.789103 27621 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.789716 26948 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:31.790599 26948 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:31.790916 26948 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.791227 26948 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:31.779047 27380 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:31.793028 27380 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:31.793394 27380 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.793628 27691 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.671109 27709 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.797852 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:31.798522 27711 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:31.799063 27711 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.799876 27709 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.773981 27699 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:31.800722 27699 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:31.801110 27699 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.653260 27700 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.744005 27688 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.705430 27539 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.797734 27687 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:31.743585 27160 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.742925 27697 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:31.801188 27711 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.812429 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:31.816167 27431 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:31.831364 27431 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:31.832247 27431 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:31.816449 27539 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.833571 27688 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.835909 27688 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.836427 27688 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.836894 27688 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.688745 27666 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.751237 27617 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.819301 27700 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.825645 27697 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.837328 27539 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:31.812577 27429 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.949s	user 0.007s	sys 0.007s
I20260709 13:13:31.843793 27482 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.862427 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:31.863302 22909 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:31.864900 27482 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.841248 26948 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.862426 27429 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.867326 22833 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 10: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:31.867596 26948 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.839418 27380 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.870747 27429 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:31.873122 22659 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 245) took 41095 ms (client timeout 15000 ms). Trace:
I20260709 13:13:31.839579 27389 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:13:31.781124 27689 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:31.879184 27603 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:31.879976 27380 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:31.886982 27534 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:31.779304 27712 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:31.838788 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2669) took 1330 ms. Trace:
I20260709 13:13:31.813483 27687 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:31.897951 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:31.851375 27658 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:31.914713 22909 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
W20260709 13:13:31.853437 27670 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.047s	user 0.000s	sys 0.003s
I20260709 13:13:31.926457 22759 rpcz_store.cc:276] 0709 13:13:30.508438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.730792 (+222354us) service_pool.cc:224] Handling call
0709 13:13:31.838603 (+1107811us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54000811d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:31.838724 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24771456}
I20260709 13:13:31.759032 27713 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:13:31.938612 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.942651 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:31.945928 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:31.946491 23458 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:13:31.933461 27539 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:31.955998 27539 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:31.957315 27539 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.957739 22658 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:31.926417 27712 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:31.976689 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46379.4us
I20260709 13:13:31.979914 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:31.985069 23458 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:31.932430 27658 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:31.987704 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:31.957736 27380 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:31.899981 27699 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.965281 27429 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:31.929234 27689 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:31.987459 27712 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:31.993988 27609 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:32.008867 27389 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.992787 27558 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:31.933184 27690 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.028596 27522 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.033192 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:32.081713 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.025835 27700 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:32.022480 27701 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:32.136250 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.137290 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.138895 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:32.138876 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 409081us
I20260709 13:13:32.236152 22834 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:32.240209 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:32.237579 22841 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:32.251547 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.174026 27429 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:32.153620 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 165957us
I20260709 13:13:32.159332 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:32.158579 23431 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.865s (SENT) [suppressed 17 similar messages]
I20260709 13:13:32.007212 27687 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.170738 27636 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:32.287043 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.287477 22910 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:32.312755 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.313819 22909 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:13:32.233944 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:32.288720 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48581.4us
I20260709 13:13:32.180629 27719 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.266064 27389 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.291811 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.292542 27534 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:32.292896 27725 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:13:32.293363 27721 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:32.233400 27666 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.308990 27609 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:32.310272 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.310473 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:32.347546 22838 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:32.310272 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.319271 22912 raft_consensus.cc:2368] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13.
I20260709 13:13:32.160287 27724 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:32.317153 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:32.311329 27636 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.959867 22658 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 9, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 9 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:31.897012 22659 rpcz_store.cc:269] 0709 13:12:50.777713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.815565 (+ 37852us) service_pool.cc:224] Handling call
0709 13:12:51.720128 (+904563us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5b090 00007f68a1af9f3a 00007f68a78b01cc 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc 00007f68a5e6e392 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:31.872829 (+40152701us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:19.556278 (+     0us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b18001fccc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:19.556490 (+   212us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:19.557278 (+   788us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:19.557348 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:13:19.557379 (+    31us) tablet.cc:662] Decoding operations
0709 13:13:19.589143 (+ 31764us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:19.589275 (+   132us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:19.589306 (+    31us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:13:19.591542 (+  2236us) tablet.cc:701] Row locks acquired
0709 13:13:19.591595 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:13:19.591827 (+   232us) write_op.cc:270] Start()
0709 13:13:19.592128 (+   301us) write_op.cc:276] Timestamp: P: 1783602799591743 usec, L: 0
0709 13:13:19.592182 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 13:13:19.593169 (+   987us) log.cc:844] Serialized 8961 byte log entry
0709 13:13:22.561651 (+2968482us) op_driver.cc:464] REPLICATION: finished
0709 13:13:23.495682 (+934031us) write_op.cc:301] APPLY: starting
0709 13:13:23.495826 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:25.047887 (+1552061us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:25.074655 (+ 26768us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:25.859740 (+785085us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:25.883990 (+ 24250us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:13:25.884278 (+   288us) write_op.cc:312] APPLY: finished
0709 13:13:25.884877 (+   599us) log.cc:844] Serialized 204 byte log entry
0709 13:13:26.087207 (+202330us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:26.120658 (+ 33451us) write_op.cc:454] Released schema lock
0709 13:13:26.124521 (+  3863us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:26.150301 (+ 25780us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:26.150602 (+   301us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3087347,"reported_tablets":51,"spinlock_wait_cycles":293919104,"child_traces":[["op",{"apply.queue_time_us":878650,"apply.run_cpu_time_us":75503,"apply.run_wall_time_us":2667454,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100313,"num_ops":23,"prepare.queue_time_us":17626,"prepare.run_cpu_time_us":48935,"prepare.run_wall_time_us":1393658,"raft.queue_time_us":1168501,"raft.run_cpu_time_us":35184,"raft.run_wall_time_us":932664,"replication_time_us":2922327,"spinlock_wait_cycles":4533504,"thread_start_us":3213277,"threads_started":5,"wal-append.queue_time_us":1777590,"wal-append.run_cpu_time_us":36060,"wal-append.run_wall_time_us":3207958}]]}
I20260709 13:13:32.240024 27697 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.322355 27534 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:32.306866 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 147638us
I20260709 13:13:32.374874 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:32.376281 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:32.180045 27686 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.232071 27688 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:32.190016 27617 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
W20260709 13:13:32.322610 27638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.000s	sys 0.045s
I20260709 13:13:32.175753 27380 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.236011 27522 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.380909 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.236768 27700 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.364668 27636 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:32.237030 27701 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:32.386240 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69583.2us
I20260709 13:13:32.403455 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.404349 22759 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:32.379479 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:32.399094 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 100109us
I20260709 13:13:32.330581 27719 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.406800 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:32.388908 27638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.000s	sys 0.046s
I20260709 13:13:32.409901 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.380586 27534 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:32.414009 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.382257 27687 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:32.416260 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.416985 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.419070 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:32.386698 27609 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:32.381922 27529 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:32.424222 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.356659 27721 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.422384 22757 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:32.365015 27724 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.426509 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:32.426797 22836 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:32.354979 27429 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.238875 27658 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:32.422266 27692 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.831s	user 0.013s	sys 0.000s
I20260709 13:13:32.434310 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 1979) took 1137 ms. Trace:
W20260709 13:13:32.438506 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 279949us
I20260709 13:13:32.425817 27657 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:32.412824 27617 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:13:32.454583 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.455381 22834 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:32.457002 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.426249 27529 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:32.458248 22841 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:32.458576 23458 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:13:32.389539 27686 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.438832 22832 rpcz_store.cc:276] 0709 13:13:31.295608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.370462 (+ 74854us) service_pool.cc:224] Handling call
0709 13:13:31.370822 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:31.476227 (+105405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.476260 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:31.485852 (+  9592us) log.cc:844] Serialized 26 byte log entry
0709 13:13:31.735681 (+249829us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:32.134006 (+398325us) log.cc:844] Serialized 14 byte log entry
0709 13:13:32.147407 (+ 13401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.147511 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.432447 (+284936us) raft_consensus.cc:1685] finished
0709 13:13:32.432481 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.432693 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12997,"prepare.queue_time_us":576318,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":88376,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":36133,"thread_start_us":658018,"threads_started":3,"wal-append.queue_time_us":486070}
I20260709 13:13:32.431258 27638 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.462621 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:32.467288 23460 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:32.466241 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:32.476639 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69953.1us
I20260709 13:13:32.476907 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.477634 22837 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:32.446283 27720 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:32.443612 27692 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:32.482043 27734 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:32.483119 27738 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.494387 27733 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:32.481302 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.496213 22838 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:32.497418 27690 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.498695 27389 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.468883 27699 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.500165 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.447057 27657 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:32.501096 22906 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:32.490236 27734 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.502456 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.503232 22906 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:32.503844 27734 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:32.504407 27061 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.901s	user 0.008s	sys 0.004s
I20260709 13:13:32.504377 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.505311 27061 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.505759 22906 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:32.506543 22914 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:32.507457 27732 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:32.490950 27618 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.509341 27701 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.511373 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.514168 27724 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.508062 27732 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:32.478230 27723 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:32.515198 27732 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.514556 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:32.517177 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 751203us
I20260709 13:13:32.501111 27657 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:32.518102 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:32.480046 27638 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:32.520550 22912 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1813) took 1372 ms (client timeout 1508 ms). Trace:
W20260709 13:13:32.521111 22912 rpcz_store.cc:269] 0709 13:13:31.148341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.194564 (+ 46223us) service_pool.cc:224] Handling call
0709 13:13:32.520280 (+1325716us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540020b7d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:32.520465 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485085568}
I20260709 13:13:32.494313 27738 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.522773 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.516719 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.532572 22912 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:32.525133 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.528497 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:32.528705 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:32.534578 22841 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:32.539628 27701 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:32.540660 27699 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:32.532853 22909 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 10: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:32.548770 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 169467us
I20260709 13:13:32.504458 27689 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.480221 27429 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.522764 27720 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.507575 27061 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:32.514832 27724 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:32.566839 27700 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:32.568771 27737 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:13:32.569456 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.573612 22912 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:32.575224 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.578349 22912 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:13:32.564283 27720 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.581420 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:32.571378 27742 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:13:32.573483 27531 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.930s	user 0.014s	sys 0.002s
I20260709 13:13:32.603580 27399 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:32.605713 27749 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:13:32.607214 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 625) took 1495 ms. Trace:
I20260709 13:13:32.608153 27738 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.607621 22908 rpcz_store.cc:276] 0709 13:13:31.112101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.680433 (+568332us) service_pool.cc:224] Handling call
0709 13:13:31.759839 (+ 79406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:32.162668 (+402829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:32.162717 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:32.178940 (+ 16223us) log.cc:844] Serialized 52 byte log entry
0709 13:13:32.396729 (+217789us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:32.397072 (+   343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.397178 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.606926 (+209748us) raft_consensus.cc:1685] finished
0709 13:13:32.606956 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.607146 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9538,"raft.queue_time_us":51500,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":1480,"thread_start_us":470204,"threads_started":2,"wal-append.queue_time_us":212997}
I20260709 13:13:32.589629 22912 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:32.613416 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.577345 27724 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:32.604847 27399 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.573949 27722 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:32.621558 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.622597 27721 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.625562 27750 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.625918 27380 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.628046 27700 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:32.628873 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 887) took 1100 ms. Trace:
I20260709 13:13:32.609035 27712 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.635637 23464 rpcz_store.cc:276] 0709 13:13:31.516065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.547024 (+ 30959us) service_pool.cc:224] Handling call
0709 13:13:32.616020 (+1068996us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b5400375354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:32.616172 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111899904}
I20260709 13:13:32.634183 22908 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:32.586069 27061 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:32.618862 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.641160 22912 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:32.642185 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.643458 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.644004 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:32.644043 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:32.629161 27639 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.952s	user 0.009s	sys 0.004s
I20260709 13:13:32.573087 27529 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.638551 27721 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.639007 27722 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.645378 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.629817 27638 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:32.642388 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.633368 27522 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:32.647343 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 177500us
I20260709 13:13:32.649029 22912 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:32.605409 27636 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:32.679394 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 122558us
I20260709 13:13:32.688098 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:32.689127 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:32.689857 23460 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.690049 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:32.690672 22832 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:32.691037 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48734.2us
W20260709 13:13:32.691466 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.688323 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:32.690773 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:32.693725 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:32.615829 27531 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.693758 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:32.702592 22840 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.703486 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.653740 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.639678 27399 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.646569 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:32.695891 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:32.704253 22912 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:32.633040 27724 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.705245 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:32.711410 22908 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:32.704408 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.708045 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.712497 22912 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:32.639597 27686 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:32.713665 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.720634 22912 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:32.718282 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:32.679371 27736 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.712805 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.650820 27711 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:32.729857 27711 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:32.648768 27061 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.722325 22915 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:32.630232 22658 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 11, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 11 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:32.733222 22658 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 9, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 9 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:32.735428 22658 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:32.709321 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.693053 27534 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.738237 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1815) took 1589 ms. Trace:
I20260709 13:13:32.738660 22913 rpcz_store.cc:276] 0709 13:13:31.149031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.726830 (+577799us) service_pool.cc:224] Handling call
0709 13:13:31.758337 (+ 31507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:32.152133 (+393796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:32.152169 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:32.152669 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:13:32.342350 (+189681us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:32.342428 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.342596 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.737998 (+395402us) raft_consensus.cc:1685] finished
0709 13:13:32.738022 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.738173 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2134,"raft.queue_time_us":44094,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":493777,"threads_started":2,"wal-append.queue_time_us":386427}
I20260709 13:13:32.739920 27534 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:32.740298 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.740795 22910 raft_consensus.cc:2368] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:32.741933 27753 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:32.742360 27753 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.742502 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.743358 22910 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:32.743443 27723 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:32.743892 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:32.745997 27638 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:32.746407 27638 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.746743 27638 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:32.702230 27700 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:32.749081 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:32.750068 22913 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:32.637184 27750 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:32.650815 27389 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:32.728032 27699 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:32.728520 27745 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:13:32.721896 27751 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:32.729490 27741 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:13:32.730928 27711 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.742729 27753 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.153331 27405 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:33.153855 27711 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.154083 27405 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.156822 27405 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.726867 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.157367 27695 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:32.663978 27719 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.157873 22912 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:32.653165 27722 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.754134 22915 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:33.160550 27722 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:32.754921 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.742547 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:32.760953 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 114366us
I20260709 13:13:33.165822 22837 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.165423 22757 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.167644 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:33.168035 22732 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.508s (SENT)
I20260709 13:13:33.168282 22757 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.169226 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.169538 27389 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:32.770267 22840 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:32.708496 27522 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:32.780210 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.172673 22758 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.172399 27522 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.174019 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.174757 22758 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.175843 27522 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:32.664206 27558 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.175626 27389 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.719028 27399 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:32.777104 22807 leader_election.cc:400] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:33.180778 22807 leader_election.cc:403] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:32.778276 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.182216 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.774358 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1819) took 1605 ms. Trace:
I20260709 13:13:32.781318 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:32.783344 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.804683 22908 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:33.183400 22764 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.183502 22765 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:32.768962 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:32.771129 27736 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.184782 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:32.770287 27700 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.185797 27700 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:33.183121 22914 rpcz_store.cc:276] 0709 13:13:31.169267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.734383 (+565116us) service_pool.cc:224] Handling call
0709 13:13:31.761333 (+ 26950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:32.507028 (+745695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.774024 (+266996us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54001257d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:32.774276 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713241856}
I20260709 13:13:32.776501 27699 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.187372 27699 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.188076 27736 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:32.889923 22658 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 8 to 12, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:33.189716 22658 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 10, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 10 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:13:33.190665 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 413859us
I20260709 13:13:33.191471 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:33.191538 22658 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 4 to 6, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 6 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:33.199489 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:32.770565 27755 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:32.776647 27723 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:32.721264 27380 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:33.122394 27702 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.122545 27061 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.123528 22761 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:32.773757 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:13:33.252458 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 478786us
I20260709 13:13:33.253259 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:33.254179 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:33.125388 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 434673us
I20260709 13:13:33.126904 27712 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:33.127086 27657 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.609s	user 0.015s	sys 0.014s
I20260709 13:13:33.127214 27531 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.128432 27636 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:33.130167 27429 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.568s	user 0.015s	sys 0.008s
I20260709 13:13:33.130398 22913 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:33.237425 27751 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:33.131862 27731 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.566s	user 0.001s	sys 0.000s
W20260709 13:13:33.131992 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.140673 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:33.140825 27730 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.661s	user 0.005s	sys 0.005s
W20260709 13:13:33.145977 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.268216 22836 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:13:33.268657 22836 pending_rounds.cc:85] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:33.268867 22836 pending_rounds.cc:107] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:13:33.269125 22836 raft_consensus.cc:2889] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:33.150696 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 744) took 1013 ms. Trace:
I20260709 13:13:33.170994 27703 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:32.650311 27639 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:32.650959 27721 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.200451 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:33.201126 22658 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:32.781651 27655 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.200495 27699 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.202864 27700 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.284282 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2018) took 1709 ms. Trace:
I20260709 13:13:33.284746 22760 rpcz_store.cc:276] 0709 13:13:31.574873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.575177 (+   304us) service_pool.cc:224] Handling call
0709 13:13:31.592430 (+ 17253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:31.974446 (+382016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.974488 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:32.735026 (+760538us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:32.735705 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 13:13:33.116057 (+380352us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:33.116584 (+   527us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.239388 (+122804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.239492 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:33.283941 (+ 44449us) raft_consensus.cc:1685] finished
0709 13:13:33.284063 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.284219 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122469,"prepare.queue_time_us":264,"prepare.run_cpu_time_us":3877,"prepare.run_wall_time_us":4083,"raft.queue_time_us":208021,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":63878,"spinlock_wait_cycles":15862016,"thread_start_us":761356,"threads_started":2,"wal-append.queue_time_us":504041}
I20260709 13:13:33.286245 27700 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.287276 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.287866 22760 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.287712 22658 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 10, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 10 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:33.203740 22758 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:33.289989 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.290287 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.290617 22764 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:33.290969 22758 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:33.208384 22765 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:33.211624 27753 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.212776 27529 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.145380 22907 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:33.128403 27750 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:33.220115 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 460700us
I20260709 13:13:33.220824 27695 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:13:33.225581 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.226138 27666 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.223562 27658 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.301651 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:33.145581 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.303063 22838 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:33.229079 27724 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.229756 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 737) took 1197 ms. Trace:
I20260709 13:13:33.141777 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.305197 22833 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:13:33.149250 23460 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:33.305385 27666 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.239152 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.307348 27429 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:33.307548 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.308135 27666 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.308283 22763 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:33.303124 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:33.304697 22839 rpcz_store.cc:276] 0709 13:13:32.032157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.032273 (+   116us) service_pool.cc:224] Handling call
0709 13:13:32.032598 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:32.421297 (+388699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:32.421346 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:32.421406 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:32.421451 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.847862 (+426411us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54000c67d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:33.155010 (+307148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.229666 (+ 74656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893353856}
I20260709 13:13:33.309890 27712 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:33.309368 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:33.236917 27720 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.006s
I20260709 13:13:33.311304 27720 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.311858 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.256644 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.313442 27720 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.313823 22762 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:33.314806 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.314874 27686 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:33.315516 22762 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:33.256059 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:33.316398 27405 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:33.316797 27405 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:33.316865 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.317677 22762 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:33.317822 27405 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.318529 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.319007 27753 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.319144 22762 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.319648 27405 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:33.259003 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67619.4us
I20260709 13:13:33.259297 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:33.312992 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.320564 22763 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:13:33.320798 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:33.320811 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61672us
I20260709 13:13:33.322091 22760 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:33.264268 27399 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:13:33.233471 27687 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.809s	user 0.012s	sys 0.003s
I20260709 13:13:33.238569 27609 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:32.779470 22766 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:13:33.264232 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 495350us
I20260709 13:13:33.284332 27699 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:33.256687 27638 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.008s
I20260709 13:13:33.255582 27702 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.273347 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.258265 27617 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.343256 27617 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.343724 27617 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.344515 27702 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.344691 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.345080 27638 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:33.280890 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86428.9us
I20260709 13:13:33.281121 27534 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.281822 27655 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.292229 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.352820 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2814) took 1036 ms. Trace:
I20260709 13:13:33.265304 27713 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:33.270394 22835 rpcz_store.cc:276] 0709 13:13:32.137193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.137564 (+   371us) service_pool.cc:224] Handling call
0709 13:13:32.137869 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:32.138089 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:13:32.187964 (+ 49875us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.149850 (+961886us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b18000fe260. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:13:33.150099 (+   249us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.150318 (+   219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:33.150370 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:33.150421 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.150493 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.150627 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":737050,"prepare.queue_time_us":1016862,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":113,"raft.queue_time_us":1847,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":7735,"spinlock_wait_cycles":2346112,"thread_start_us":208318,"threads_started":2,"wal-append.queue_time_us":378002}
I20260709 13:13:33.262251 27657 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.273972 27558 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.297343 22908 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:33.298116 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.277187 27738 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:33.310638 27712 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:33.316016 22764 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
W20260709 13:13:33.240132 22832 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.549s	user 0.014s	sys 0.000s
I20260709 13:13:33.364678 22832 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.364949 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:33.323278 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.324517 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:33.367257 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:33.261745 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.325407 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:33.295130 27732 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.779s	user 0.013s	sys 0.006s
W20260709 13:13:33.334347 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 78374.1us
I20260709 13:13:33.368935 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:33.369918 22906 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:33.368534 22805 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:33.370133 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.370837 22758 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [12.6-13.7]
I20260709 13:13:33.242513 27380 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.336860 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:33.308063 27429 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:33.337296 27755 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:13:33.258482 27723 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:33.354719 27713 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.415498 27713 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:33.410197 27761 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:13:33.423082 27429 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.385298 27522 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:13:33.346071 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.331463 27724 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:33.430169 27764 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:13:33.438428 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 184294us
I20260709 13:13:33.328083 27666 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.278194 27703 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.357223 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:33.450196 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93070us
I20260709 13:13:33.450946 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:33.451066 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.358495 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.452680 22765 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:33.453761 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.353295 22912 rpcz_store.cc:276] 0709 13:13:32.316587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.158608 (+842021us) service_pool.cc:224] Handling call
0709 13:13:33.158950 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.159177 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:33.216232 (+ 57055us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.279444 (+ 63212us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.344955 (+ 65511us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b18003f2860. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219
0709 13:13:33.352297 (+  7342us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b18003f2860. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:13:33.352460 (+   163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:33.352519 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:33.352566 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.352631 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.352757 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54910,"prepare.queue_time_us":78050,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":54259,"spinlock_wait_cycles":67546496,"thread_start_us":65837,"threads_started":2,"wal-append.queue_time_us":53934}
I20260709 13:13:33.341864 27720 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.372617 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.373946 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.376194 22914 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:33.375160 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:33.378436 27750 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:33.345418 22761 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:33.462885 22761 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:13:33.462980 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 87886.8us
I20260709 13:13:33.360658 27558 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.356935 27534 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.388819 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.388715 22760 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:13:33.389850 27723 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.375927 27380 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:33.393942 27755 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:33.399909 27712 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.354398 27061 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.402820 27655 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.381212 27759 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:33.360379 27688 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.384768 27754 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:13:33.372151 27738 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.376729 27732 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.467751 27558 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.410260 22910 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:33.372905 27692 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.492264 27692 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.492735 27692 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.495983 27713 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.497720 22909 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:33.459667 22766 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:33.464330 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:33.469882 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.469276 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.501940 22836 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.383052 27699 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.501539 22760 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:33.473218 27755 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:33.446623 27657 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.455834 27720 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.460664 27603 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.410089 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2802) took 1159 ms. Trace:
I20260709 13:13:33.510269 22907 rpcz_store.cc:276] 0709 13:13:32.250633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.250794 (+   161us) service_pool.cc:224] Handling call
0709 13:13:33.409844 (+1159050us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54000fb754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:33.410015 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868712704}
I20260709 13:13:33.465273 27522 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.469223 27529 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:33.520457 27534 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.521427 27603 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:33.521962 27603 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.525619 27655 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:33.486843 27738 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:33.507596 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63286.3us
I20260709 13:13:33.513676 27695 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12.
I20260709 13:13:33.514518 27755 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.516361 27720 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.358973 27751 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.491671 27732 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.528369 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:33.530854 22763 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:33.531985 22913 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:33.504626 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:33.577064 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 72521.6us
I20260709 13:13:33.577921 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:33.580611 27753 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:33.581336 27655 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:33.535581 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1814) took 2386 ms. Trace:
I20260709 13:13:33.536151 27639 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.582100 22911 rpcz_store.cc:276] 0709 13:13:31.148726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.724783 (+576057us) service_pool.cc:224] Handling call
0709 13:13:31.725164 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:31.725592 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.725622 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:31.745241 (+ 19619us) log.cc:844] Serialized 52 byte log entry
0709 13:13:32.134784 (+389543us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:32.290934 (+156150us) log.cc:844] Serialized 14 byte log entry
0709 13:13:32.300841 (+  9907us) log.cc:844] Serialized 14 byte log entry
0709 13:13:32.317308 (+ 16467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.321178 (+  3870us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.907842 (+586664us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:33.535260 (+627418us) raft_consensus.cc:1685] finished
0709 13:13:33.535293 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.535480 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31338,"prepare.queue_time_us":333925,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":36267,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":33092,"spinlock_wait_cycles":52013312,"thread_start_us":448395,"threads_started":2,"wal-append.queue_time_us":657108}
I20260709 13:13:33.484109 27638 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.583770 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:33.537672 27721 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.486605 27688 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.584519 22906 raft_consensus.cc:2376] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14. Re-sending same reply.
I20260709 13:13:33.473366 27531 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:13:33.554347 22836 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
W20260709 13:13:33.567056 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 127341us
I20260709 13:13:33.506187 27702 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.508011 27699 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.481920 27061 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.532127 27529 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:33.590461 27772 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:33.559508 27666 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:33.590714 27529 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.591081 27772 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.593600 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.595878 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.597884 23461 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:33.605052 27699 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.580740 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:33.580747 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.586145 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:33.527395 27522 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:33.367133 22658 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:33.540252 27609 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:33.620548 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.622972 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.622987 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.624050 22840 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:33.608486 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.558535 27713 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.609831 27522 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.616873 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.626288 23461 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:33.571143 27617 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.607780 27755 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.627251 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.634315 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.636183 27639 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.636428 23461 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:33.634509 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:33.636772 27639 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.637600 23460 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:33.637792 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.638437 27655 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.639113 27687 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:33.639014 27655 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.639613 27687 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.612293 27638 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:33.640625 27638 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:33.641304 27655 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.638576 22837 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:33.641474 27638 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.642769 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:33.644905 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.646183 23458 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:33.647349 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:33.650010 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.650707 23460 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.652405 23463 raft_consensus.cc:2368] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11.
I20260709 13:13:33.654119 22907 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:33.655217 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.657269 27617 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.631723 27767 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:13:33.658399 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 206097us
I20260709 13:13:33.659024 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.659163 27753 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:33.659283 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:33.659848 27753 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.660221 27736 raft_consensus.cc:2764] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:33.611874 27702 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.663803 27702 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.663978 23464 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:13:33.664371 23464 pending_rounds.cc:85] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 11
I20260709 13:13:33.664587 23464 pending_rounds.cc:107] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:13:33.664801 23464 raft_consensus.cc:2889] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:33.665838 27429 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.665853 27534 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:33.666482 27429 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.666689 27534 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.668816 27429 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.623164 27732 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:33.668982 27534 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.669464 27732 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:33.642789 27638 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.670531 27732 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.642738 27655 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:33.674440 27655 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.675591 22913 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:33.678792 27061 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.682585 27655 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.625428 27609 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.628085 27389 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:33.631737 27733 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.607764 27777 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:33.655748 22836 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.703804 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.694855 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.705014 22908 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:33.630105 27772 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.697417 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.668010 27724 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:33.700196 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:33.716265 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.717240 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:33.717689 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.728523 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.729238 22766 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:33.729544 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.730995 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.732053 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.734776 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.694017 27687 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.733824 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.736752 22762 raft_consensus.cc:2368] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11.
I20260709 13:13:33.718122 27601 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.740970 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.740970 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.743157 22760 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:33.744493 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.745355 22762 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:33.745759 22766 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:33.646965 27692 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.741523 27531 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:33.748373 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.759372 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.703716 27389 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:33.714867 27719 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.748222 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.798480 22763 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:33.798601 22762 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 8.
I20260709 13:13:33.799414 27531 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:33.799585 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.748016 27601 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.800936 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.813982 22760 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
W20260709 13:13:33.802285 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.806160 22758 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.809067 22911 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:33.815932 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.817054 22762 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:33.704473 27733 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.741874 27655 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:13:33.716579 27777 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.839308 27777 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.747462 27380 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:33.840798 22832 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:33.841392 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:33.844134 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.845019 22763 raft_consensus.cc:2368] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
I20260709 13:13:33.849335 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:33.852707 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.852263 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.856412 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:33.812769 27732 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.859035 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.852348 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.862936 22765 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:33.863075 22762 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:33.861570 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.864216 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.859345 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.865648 23464 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:33.866014 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.865012 22765 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
W20260709 13:13:33.867604 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.869656 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.870829 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.871376 22766 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:33.874632 27380 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:33.839542 27703 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.879021 27759 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:33.879818 27531 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.883733 22906 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:33.878923 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.885061 22760 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:33.885682 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.867621 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.887071 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.887346 22765 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:33.890663 22764 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.895750 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.896663 22765 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.903146 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.903595 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.907214 22766 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:33.907326 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.814052 27719 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.908947 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.911504 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.868963 27732 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.915588 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.915974 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.927525 22836 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
W20260709 13:13:33.928082 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 210892us
I20260709 13:13:33.919447 22762 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:13:33.929270 22879 leader_election.cc:400] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11.
I20260709 13:13:33.929725 22879 leader_election.cc:403] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:13:33.924767 22764 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:33.925598 22833 raft_consensus.cc:2410] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:13:33.868754 27753 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:33.940526 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.890223 27712 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:33.949303 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.943092 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.950579 22763 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:13:33.950774 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.946317 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.946403 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.947327 22765 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:33.947710 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.953264 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:33.955299 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:33.956816 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.957736 23460 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:33.957108 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:33.905314 27429 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.975251 22834 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:33.960793 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:33.961882 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.981402 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.982553 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.962913 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.970496 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.990067 22762 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:33.971310 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.990945 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.991355 22836 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:33.991803 23458 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:33.971807 27755 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:33.972889 27601 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:33.973817 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.997887 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:33.998660 22763 raft_consensus.cc:2368] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
I20260709 13:13:33.999153 22841 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:33.998953 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.971705 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:33.981089 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.000824 22834 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:33.992899 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:33.982739 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:33.940292 27732 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.008379 22880 leader_election.cc:400] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 10 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11.
I20260709 13:13:34.003209 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:34.003373 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:34.007208 22836 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:34.007675 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:34.011595 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.012125 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:34.012230 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:34.012799 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:34.012912 22906 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.006546 27723 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:34.013106 22840 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:34.013628 27723 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:34.014072 27723 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:34.016877 27782 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11.
I20260709 13:13:34.017844 27713 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:34.017617 27721 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:34.018328 27713 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.006971 27751 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:13:34.019357 27754 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:33.960711 27712 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.020931 27754 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:34.021452 27754 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:33.974802 27703 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.025440 27703 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:34.026176 27774 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:13:34.001770 22760 raft_consensus.cc:2376] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14. Re-sending same reply.
I20260709 13:13:33.995899 27429 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.035939 22838 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:34.039644 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.041167 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:34.042447 22907 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:34.048938 27688 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:34.035063 27603 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.513s	user 0.015s	sys 0.000s
I20260709 13:13:34.019176 27713 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:34.089583 22759 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:33.992995 27755 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.091665 22759 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:33.912736 27389 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.000145 27777 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.080s
W20260709 13:13:34.099040 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 634) took 1552 ms (client timeout 1853 ms). Trace:
I20260709 13:13:34.098886 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:34.099434 22909 rpcz_store.cc:269] 0709 13:13:32.546799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.565469 (+ 18670us) service_pool.cc:224] Handling call
0709 13:13:34.098393 (+1532924us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400043654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:34.098974 (+   581us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129233792}
I20260709 13:13:34.010387 27639 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:34.103816 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 174690us
I20260709 13:13:34.107254 27719 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:34.108408 27703 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:34.108771 27609 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.110221 27692 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:34.111116 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:34.111796 27522 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:34.112316 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 109013us
I20260709 13:13:34.143079 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:34.113844 27754 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:34.125573 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:34.114552 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:34.144265 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 134781us
I20260709 13:13:34.146515 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:34.135084 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.136211 22907 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:34.137136 22766 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:34.094028 27782 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.141064 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.149351 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.150640 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:34.132227 22765 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:34.096812 27755 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.090014 27687 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:34.152848 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.153896 22907 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:34.158915 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:34.160233 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:34.155009 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:34.164275 22906 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:34.165521 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:34.166046 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 262480us
I20260709 13:13:34.166635 22909 raft_consensus.cc:2368] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:34.167204 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.168740 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.178570 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.180568 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.182680 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:34.093690 27603 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.188195 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.185333 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:34.190399 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43954.5us
I20260709 13:13:34.212129 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.194937 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:34.151651 27655 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:34.190265 22730 connection.cc:256] RPC call timeout handler was delayed by -1.2953552239998771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 13:13:34.218377 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.160677 27688 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:34.220086 22838 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:34.262970 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.263821 22838 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:34.203985 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:34.219367 22841 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:34.265360 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61531.4us
W20260709 13:13:34.269799 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50887.5us
I20260709 13:13:34.270882 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.138163 27736 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:13:34.159397 27687 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:34.181643 27783 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:34.181707 27692 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:34.281886 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:34.292752 22838 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.298154 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.299392 22835 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:34.221029 27061 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:34.219292 27753 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:34.313812 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 170819us
I20260709 13:13:34.159976 27782 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.288177 27687 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:34.297708 27767 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:34.328235 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 173295us
I20260709 13:13:34.288483 27783 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:34.341305 27721 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.343376 27721 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.334651 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.348760 27687 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.352169 27687 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:34.359761 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.324384 27688 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:34.315162 27639 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:34.312000 27753 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:34.374739 27638 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.664s	user 0.014s	sys 0.002s
W20260709 13:13:34.377070 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 165072us
W20260709 13:13:34.377375 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45622.2us
W20260709 13:13:34.375749 27405 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.043s	user 0.021s	sys 0.000s
W20260709 13:13:34.381327 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 114580us
W20260709 13:13:34.378186 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 107434us
I20260709 13:13:34.330729 27733 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:34.314916 27692 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:34.373366 27783 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:34.324290 27601 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:34.332633 27767 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:34.378726 27789 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.389609 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.391875 22912 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:13:34.395810 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.406258 22840 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:34.388011 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.386518 27753 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.427845 27603 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:34.432116 27738 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.860s	user 0.009s	sys 0.003s
I20260709 13:13:34.432659 27738 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.383459 27796 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:34.435478 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:34.420542 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:34.384083 27429 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:34.411234 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:34.379576 27688 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:34.422474 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:34.405572 27687 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.421557 27655 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:34.495509 22910 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:34.450901 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.442494 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:34.470474 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:34.472450 22764 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:34.497248 22761 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:34.422371 27405 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:34.422652 27767 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:34.441134 27699 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.835s	user 0.000s	sys 0.019s
I20260709 13:13:34.416751 27601 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.433952 27796 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:34.496097 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75662.7us
W20260709 13:13:34.450650 27720 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.864s	user 0.017s	sys 0.000s
W20260709 13:13:34.501446 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50661.8us
I20260709 13:13:34.542771 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:34.385128 27788 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 13:13:34.460649 27721 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:34.439604 27789 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.458709 27688 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.441754 27736 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:34.450312 27797 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:34.630664 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.494789 27603 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:34.632805 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 190411us
I20260709 13:13:34.598522 22838 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:34.665680 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:34.676797 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:34.629106 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86474.9us
I20260709 13:13:34.495383 27609 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:34.541473 27792 raft_consensus.cc:2764] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:13:34.634763 27558 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.124s	user 0.011s	sys 0.007s
W20260709 13:13:34.679370 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48781.3us
I20260709 13:13:34.634662 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:34.694891 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 506795us
I20260709 13:13:34.696182 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:34.683252 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:34.519423 27429 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:34.558990 27405 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:34.510218 27531 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:34.633729 27766 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.106s	user 0.007s	sys 0.000s
I20260709 13:13:34.498929 27687 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.697405 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.723469 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.698160 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.684473 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:34.510525 27798 raft_consensus.cc:2764] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:13:34.690759 27534 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:34.632278 27601 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.634315 27803 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:34.707765 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73151.4us
I20260709 13:13:34.702081 27405 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:34.751961 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:34.716605 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:34.761307 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:34.717603 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:34.762568 22761 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.719070 22757 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:34.637977 27799 raft_consensus.cc:2764] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:13:34.725365 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:34.689832 27686 raft_consensus.cc:2764] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:13:34.551405 27522 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:34.730633 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 46236.1us
I20260709 13:13:34.739073 22764 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:34.766904 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:34.767670 22764 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:34.552702 27699 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.704591 27788 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:34.694686 27603 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:34.750103 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.770784 22838 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:34.704488 27782 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:34.714718 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:34.595485 27655 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.755206 22837 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:34.757395 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:34.755368 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:34.775591 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 364433us
I20260709 13:13:34.728158 27702 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:34.763698 22757 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:34.772588 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.021s	sys 0.028s
W20260709 13:13:34.794626 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.022s	sys 0.028s
I20260709 13:13:34.699447 27558 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.743369 27792 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:34.565526 27692 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.740181 27609 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:34.787024 22766 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:34.789990 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:34.790964 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.739722 27703 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:34.797973 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:34.671754 27720 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:34.801802 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:34.755558 27736 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.753368 27803 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.807521 27818 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:34.793900 22757 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.799324 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:34.898281 22757 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:34.898677 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99450.9us
I20260709 13:13:34.899912 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.901120 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2837) took 1318 ms. Trace:
I20260709 13:13:34.907014 22910 rpcz_store.cc:276] 0709 13:13:33.582478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.583001 (+   523us) service_pool.cc:224] Handling call
0709 13:13:33.583332 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:34.496006 (+912674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.900851 (+404845us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54003020d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:34.901047 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097318144}
I20260709 13:13:34.818331 23464 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:34.812487 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:34.818871 22839 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:34.794729 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:34.772097 27405 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.770475 27788 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.838848 22761 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:34.842149 22836 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:34.837250 27719 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:34.815680 27792 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.860451 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:34.862244 27819 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:13:34.861407 27603 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.868331 27687 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:34.871217 27558 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:34.929169 27788 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.872975 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:34.873991 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:34.963297 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 265209us
W20260709 13:13:34.880797 27813 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
W20260709 13:13:34.882071 27794 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:13:34.808434 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:34.902035 27703 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:34.902441 27812 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.297s	user 0.000s	sys 0.000s
I20260709 13:13:34.902843 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:34.902940 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:34.969301 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67810.8us
I20260709 13:13:34.970343 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:34.902530 27657 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:34.909296 27736 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.813443 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.976210 22838 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:34.976531 23431 leader_election.cc:400] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
W20260709 13:13:34.909808 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 115174us
I20260709 13:13:34.908793 23463 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:34.814369 27772 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:34.942684 27702 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:34.977579 22732 leader_election.cc:400] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:34.810210 27738 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:34.857728 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 633) took 2375 ms. Trace:
I20260709 13:13:34.813074 27522 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:34.813151 27797 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:34.979242 22915 rpcz_store.cc:276] 0709 13:13:32.482393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.164365 (+681972us) service_pool.cc:224] Handling call
0709 13:13:33.164752 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:33.256079 (+ 91327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:33.256120 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:33.256645 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:13:33.362248 (+105603us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:33.362312 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.362414 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:34.421770 (+1059356us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:34.827986 (+406216us) raft_consensus.cc:1685] finished
0709 13:13:34.828017 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.857626 (+ 29609us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104745,"raft.queue_time_us":91722,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4168,"spinlock_wait_cycles":65432448,"thread_start_us":87775,"threads_started":1,"wal-append.queue_time_us":205744}
W20260709 13:13:34.965605 27723 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.951s	user 0.012s	sys 0.007s
W20260709 13:13:34.929232 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 203949us
I20260709 13:13:34.807948 27807 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:34.831941 27429 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:34.932029 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.986831 22910 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:34.988205 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.989003 22910 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:34.909940 27803 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:34.941450 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:34.932654 27732 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.896s	user 0.007s	sys 0.012s
I20260709 13:13:34.872792 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:34.929469 27818 raft_consensus.cc:2764] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:35.001475 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:34.977883 27777 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:34.923694 27806 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:34.874920 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.012640 22908 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:35.013296 22908 raft_consensus.cc:606] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
W20260709 13:13:34.963438 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 90595.6us
I20260709 13:13:34.964361 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:34.934333 27792 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:34.904855 27720 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:34.976955 23431 leader_election.cc:403] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:34.977941 22732 leader_election.cc:403] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:35.022691 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46334.2us
I20260709 13:13:34.985244 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:34.990968 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:34.974959 27529 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:34.960939 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 100584us
I20260709 13:13:34.986925 27797 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.024231 22910 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:35.020581 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:35.027738 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:35.028417 22835 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:35.029978 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44876.4us
I20260709 13:13:35.027763 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 637) took 1100 ms. Trace:
I20260709 13:13:35.028563 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:35.031311 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.004025 27657 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:35.033084 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:35.033525 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:35.033846 23464 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:35.031131 22910 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:13:35.030025 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65750.2us
I20260709 13:13:35.003602 27429 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:35.035195 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:35.025856 27753 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.000s
I20260709 13:13:35.045634 27822 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:35.045821 27755 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.881s	user 0.008s	sys 0.012s
I20260709 13:13:35.042156 27639 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:35.029816 27797 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.038206 27687 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.050096 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:35.035984 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:35.068272 23463 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:35.042771 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.057148 22835 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:35.025517 27529 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.011575 27806 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:34.991596 27807 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:34.992776 27522 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:35.062654 22911 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.000s	sys 0.017s
W20260709 13:13:35.096897 22911 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.002s	sys 0.018s
I20260709 13:13:34.995666 27723 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.026106 27789 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.003369 27732 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.034195 27808 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:35.088614 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:35.100066 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57379.4us
I20260709 13:13:35.107254 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:35.100528 22915 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:13:35.104295 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:35.067612 27657 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.083591 27529 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.116569 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.097566 27558 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:35.103920 27639 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:35.115264 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 80277.8us
I20260709 13:13:35.094487 27699 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.067939 27720 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 10. Current term is 11.
I20260709 13:13:35.068684 27753 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:35.113835 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 90430.9us
I20260709 13:13:35.103097 27772 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.136524 27838 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:35.095780 27534 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:13:35.123315 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:35.080334 27755 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.105805 27832 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:35.137948 27753 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.147794 23463 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:35.136618 22909 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.142863 27841 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:35.143240 27755 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.143878 22763 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.144440 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:35.151226 27699 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:35.151443 27753 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:35.124907 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.151994 27753 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.152182 27699 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:35.152468 27753 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.153875 27534 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:35.154414 27841 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.157606 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.137485 27772 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.161329 27841 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.161546 27720 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.165196 27720 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.165887 27657 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.168292 27720 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.162310 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:35.171958 27713 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.072s	user 0.010s	sys 0.002s
I20260709 13:13:35.172178 27720 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:35.172444 27713 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.173190 27720 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.174408 27720 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.177670 27753 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.178177 27609 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:35.180838 27753 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.181537 27753 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.182080 27753 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.182101 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.183321 22835 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:35.159016 27699 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:35.185585 27829 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:13:35.188885 27753 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:35.192344 27810 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.586s	user 0.002s	sys 0.000s
W20260709 13:13:35.195007 27835 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:13:35.199519 27801 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.618s	user 0.003s	sys 0.000s
I20260709 13:13:35.187515 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.214627 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.222551 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:35.228053 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 750) took 1856 ms (client timeout 1786 ms). Trace:
W20260709 13:13:35.228674 22839 rpcz_store.cc:269] 0709 13:13:33.371728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.372277 (+   549us) service_pool.cc:224] Handling call
0709 13:13:35.227732 (+1855455us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400411ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:35.227948 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3133647744}
I20260709 13:13:35.116158 27703 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.246034 27639 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.247421 27703 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.233204 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2058) took 1037 ms. Trace:
I20260709 13:13:35.216533 27429 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.249696 22765 rpcz_store.cc:276] 0709 13:13:34.195936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.196073 (+   137us) service_pool.cc:224] Handling call
0709 13:13:34.196396 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:34.196753 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:34.196783 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:34.217359 (+ 20576us) log.cc:844] Serialized 26 byte log entry
0709 13:13:34.380448 (+163089us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:34.380498 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.380584 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:35.191709 (+811125us) spinlock_profiling.cc:243] Waited 7.94 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:35.201104 (+  9395us) raft_consensus.cc:1685] finished
0709 13:13:35.201128 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.233134 (+ 32006us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":171494,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"spinlock_wait_cycles":18272512,"thread_start_us":68275,"threads_started":1,"wal-append.queue_time_us":314094}
I20260709 13:13:35.239562 27820 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:35.120594 27558 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.235534 27821 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:35.246121 27522 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.123889 27828 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:35.200629 27839 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:35.256335 27736 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.138044 27838 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:35.257530 27807 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.258450 27796 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.259415 27732 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.259650 22906 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:35.261135 22910 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:13:35.262975 27814 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.664s	user 0.000s	sys 0.001s
I20260709 13:13:35.265223 27751 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:35.124697 27822 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:35.124189 27687 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.241451 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.281562 27839 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.312094 27839 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.279307 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 638) took 1352 ms. Trace:
I20260709 13:13:35.313647 22906 rpcz_store.cc:276] 0709 13:13:33.927220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.199369 (+272149us) service_pool.cc:224] Handling call
0709 13:13:34.199724 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:35.259410 (+1059686us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400378d20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af31c0 00007f68a1af4231 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c
0709 13:13:35.277450 (+ 18040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.279204 (+  1754us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242248064}
I20260709 13:13:35.273932 27838 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:35.315721 22912 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1832) took 1619 ms (client timeout 1920 ms). Trace:
I20260709 13:13:35.304257 27751 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:35.316262 22912 rpcz_store.cc:269] 0709 13:13:33.695894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.697157 (+  1263us) service_pool.cc:224] Handling call
0709 13:13:35.315475 (+1618318us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b540020a3d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:35.315653 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553271552}
I20260709 13:13:35.310920 27822 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:35.315786 27838 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.320493 27838 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:35.321623 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:35.237892 27767 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.009s
I20260709 13:13:35.322860 27767 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.276888 27736 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.324256 27736 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.325783 22837 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:35.326412 27736 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.123971 27806 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.327688 27806 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.328038 27832 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:35.295835 27783 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.857s	user 0.019s	sys 0.000s
I20260709 13:13:35.031016 22907 rpcz_store.cc:276] 0709 13:13:33.926982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.183938 (+256956us) service_pool.cc:224] Handling call
0709 13:13:34.260177 (+ 76239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:34.260500 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:13:34.617721 (+357221us) log.cc:844] Serialized 14 byte log entry
0709 13:13:35.026636 (+408915us) log.cc:844] Serialized 14 byte log entry
0709 13:13:35.027418 (+   782us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:35.027471 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:35.027519 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.027595 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.027706 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":489835,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":3152,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":3938,"spinlock_wait_cycles":82406656,"thread_start_us":737633,"threads_started":2}
I20260709 13:13:35.330065 27783 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.332304 27783 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.333130 22906 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:35.268065 27828 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:35.333829 27828 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:35.252743 27719 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.268754 27558 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.302368 27723 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.307490 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2011) took 1159 ms. Trace:
W20260709 13:13:35.310917 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69580.5us
I20260709 13:13:35.341751 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.291227 27712 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.340088 22840 rpcz_store.cc:276] 0709 13:13:34.149238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.149363 (+   125us) service_pool.cc:224] Handling call
0709 13:13:35.307218 (+1157855us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:35.307386 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516794112}
I20260709 13:13:35.387163 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:35.387878 22912 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.387576 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.390529 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 644) took 1395 ms. Trace:
I20260709 13:13:35.391013 22909 rpcz_store.cc:276] 0709 13:13:33.994924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.289879 (+294955us) service_pool.cc:224] Handling call
0709 13:13:34.290264 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.148438 (+858174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.390264 (+241826us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54000fad54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:35.390465 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1946015488}
I20260709 13:13:35.392153 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.394208 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.395960 22907 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.406023 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 649) took 1555 ms. Trace:
I20260709 13:13:35.406483 22763 rpcz_store.cc:276] 0709 13:13:33.850323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.999372 (+149049us) service_pool.cc:224] Handling call
0709 13:13:34.039242 (+ 39870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.150173 (+1110931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.405782 (+255609us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400369054. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:35.405962 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539905920}
I20260709 13:13:35.420873 22909 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:35.424015 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.373128 27822 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:35.429651 22907 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:13:35.431143 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43721.9us
I20260709 13:13:35.435237 22912 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:35.433985 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:35.433087 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.441968 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:35.443544 27824 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.444092 27824 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.379364 27380 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:13:35.451412 27380 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:35.451880 27380 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:35.395578 27802 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
W20260709 13:13:35.461119 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 254906us
I20260709 13:13:35.405274 27736 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.470139 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.373852 27797 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.316854 27751 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.477263 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.486879 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.466972 27833 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:35.475191 27695 raft_consensus.cc:2764] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:35.473606 27655 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:35.478168 27534 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.481896 27732 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.440570 27522 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:13:35.489231 27738 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:35.488487 27733 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:35.496482 27601 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.005s
I20260709 13:13:35.466720 27719 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:13:35.481454 27753 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:13:35.498502 27833 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.496472 27751 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.453370 27380 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.488860 27712 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:13:35.488667 27832 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:13:35.474560 27788 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.483918 27736 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:35.522858 27803 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.006s
W20260709 13:13:35.537266 27688 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.863s	user 0.010s	sys 0.014s
I20260709 13:13:35.539288 23464 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.541812 27830 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:13:35.558760 27830 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:35.559135 27830 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.559993 27830 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.563061 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:35.563845 27603 raft_consensus.cc:993] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:35.529826 27601 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.564411 27603 raft_consensus.cc:1081] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
W20260709 13:13:35.565011 27405 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.633s	user 0.002s	sys 0.007s
I20260709 13:13:35.529317 27713 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:13:35.565986 27830 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:35.567071 27822 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:35.567521 27822 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:35.568997 27736 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.568779 27822 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.573890 27788 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.574772 27601 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.576057 22836 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:35.577101 27803 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.578750 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 636) took 1652 ms. Trace:
I20260709 13:13:35.579553 27803 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.580375 22832 raft_consensus.cc:2368] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12.
I20260709 13:13:35.581586 27767 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.584017 27558 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.535948 27833 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.588415 27824 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:35.590669 27792 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.003s
I20260709 13:13:35.592236 27558 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:35.550792 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64041.7us
W20260709 13:13:35.576121 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.595774 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:35.596421 23460 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:35.599469 27849 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:13:35.557219 27688 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.604074 27688 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.605031 27822 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:35.605569 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.606267 22762 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:35.606717 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.607260 27791 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:35.607888 27791 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.608708 23460 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:35.610062 27791 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:35.611462 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 634028us
I20260709 13:13:35.607419 27818 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.613549 27818 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.615808 27818 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.616478 22913 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.619746 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.620204 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.620522 22841 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:35.620764 22840 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:13:35.621152 22840 pending_rounds.cc:85] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:35.621366 22840 pending_rounds.cc:107] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:13:35.621625 22840 raft_consensus.cc:2889] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:35.616539 27831 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
W20260709 13:13:35.622538 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.622669 27831 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:35.622998 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.623219 27831 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.625887 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.622763 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.626308 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:35.626554 22757 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:35.626533 22760 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:13:35.627622 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.621479 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.629149 22841 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:35.629830 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.630007 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.630441 22764 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.630755 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.631177 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.631364 22760 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:35.633462 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.634107 22757 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:35.634905 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.636087 22913 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:35.637688 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.638473 22757 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:35.640316 22912 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:35.642464 23458 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:35.643107 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:35.644439 22910 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:35.579121 22911 rpcz_store.cc:276] 0709 13:13:33.926700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.187699 (+260999us) service_pool.cc:224] Handling call
0709 13:13:34.283155 (+ 95456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:34.312278 (+ 29123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:34.312316 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:34.324954 (+ 12638us) log.cc:844] Serialized 52 byte log entry
0709 13:13:34.440588 (+115634us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:35.097795 (+657207us) log.cc:844] Serialized 14 byte log entry
0709 13:13:35.102028 (+  4233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.578305 (+476277us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:35.578352 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:35.578410 (+    58us) raft_consensus.cc:1685] finished
0709 13:13:35.578523 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.578690 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13584,"prepare.queue_time_us":479086,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":85570,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":32262,"spinlock_wait_cycles":43610880,"thread_start_us":612142,"threads_started":2,"wal-append.queue_time_us":270390}
I20260709 13:13:35.646241 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.660720 22765 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:35.646759 22760 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:13:35.662767 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.663172 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:35.663369 22760 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.664036 22907 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:35.530529 27655 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:35.650682 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:35.674896 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.683043 22765 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 9.
I20260709 13:13:35.684356 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.685082 22765 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:35.664788 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.650568 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.662767 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.647193 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.669370 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.658644 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.656510 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:35.690451 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.691068 22757 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.694137 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:35.695467 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.696247 27830 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:35.696493 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.697445 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.698117 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.728425 22762 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:35.698525 22915 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:35.591233 27792 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.698908 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.674034 27767 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:13:35.632182 27405 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.699766 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:35.732430 23460 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:13:35.730253 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.700866 27788 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:35.735270 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.702102 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:35.702888 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44381.9us
I20260709 13:13:35.737289 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:35.706296 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:35.661616 27754 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 13:13:35.706929 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44279.9us
I20260709 13:13:35.741055 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:35.741549 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.708000 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:35.722348 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 66007us
W20260709 13:13:35.719169 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 233801us
I20260709 13:13:35.723037 22757 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:35.739574 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.744045 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.675310 27839 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:13:35.701789 27655 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.725350 22760 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:35.709033 22764 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:35.703172 27797 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.731034 22763 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:35.753059 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.753975 22763 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:35.733719 27688 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:35.733985 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:35.711310 27821 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.705816 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:35.760789 22834 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.760866 22911 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:35.731799 27818 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.729846 27824 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:35.744668 22757 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.768092 22763 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:35.768730 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.769908 27840 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:35.770143 27816 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.770406 27840 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:35.771014 27858 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:35.771549 27858 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.677079 27699 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:35.776648 27699 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:35.777215 27699 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.772927 27818 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.778496 27832 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:35.778546 27825 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:35.779999 27601 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:35.762423 27798 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.745318 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.746465 27603 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:35.774364 27755 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:13:35.781924 27603 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:35.744830 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.783049 27603 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.784091 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.746798 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:35.785413 27429 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:35.785784 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:35.787070 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:35.787616 22840 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.749361 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:35.727085 27522 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:35.789803 27522 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.789878 27738 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:13:35.790300 27522 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:35.741068 27791 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:13:35.796087 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:35.791167 27061 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:35.798852 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:35.799090 27061 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.799487 22911 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.607175 27529 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:35.735539 27788 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.771379 27840 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.740785 27753 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:35.675165 27609 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:35.780409 27832 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:35.785808 27719 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:35.747512 27639 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:35.789820 22913 raft_consensus.cc:606] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:35.824132 22913 raft_consensus.cc:614] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:35.751996 22766 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:13:35.740993 27803 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:35.793905 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.794526 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:35.785332 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.798296 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:35.757718 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.752749 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:35.801874 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.802495 22757 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:35.802742 23460 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:35.759902 27721 raft_consensus.cc:2764] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:35.811684 22840 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:35.814002 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.773030 27759 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:35.806712 27788 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.828724 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:35.908923 22914 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.799513 27061 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.830989 22908 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.909485 22764 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:35.909909 22757 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:35.813928 27825 consensus_queue.cc:1243] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:13:35.910494 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 96653.4us
I20260709 13:13:35.829569 22915 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:35.816557 27797 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:13:35.834097 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48824.9us
I20260709 13:13:35.927466 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:35.928158 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.928972 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.929735 27609 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:35.818909 27831 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:35.813314 27753 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:35.910429 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 108656us
I20260709 13:13:35.824767 27807 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.925103 22757 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:13:35.792286 27824 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:35.824558 27723 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 13:13:35.908363 27798 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:13:35.930338 22911 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
W20260709 13:13:36.104470 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175556us
I20260709 13:13:36.000556 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 764) took 1158 ms. Trace:
I20260709 13:13:36.068152 22761 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:35.911458 27719 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:35.924228 27840 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.114425 27824 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:36.113191 27724 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:13:36.026914 27732 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.006s
I20260709 13:13:36.115602 27724 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.116006 27380 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:36.116537 27724 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.117403 22757 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:36.117761 27783 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:36.117895 27724 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:36.118180 27783 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:35.931800 27534 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:36.119437 27534 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:36.120165 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2857) took 1256 ms. Trace:
I20260709 13:13:36.120376 27534 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.120579 22906 rpcz_store.cc:276] 0709 13:13:34.863460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.314592 (+451132us) service_pool.cc:224] Handling call
0709 13:13:35.314924 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.333596 (+ 18672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.119947 (+786351us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b540010e5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:36.120108 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41381632}
I20260709 13:13:36.121546 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:36.122793 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:36.123390 22909 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.960359 22841 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:35.992728 27805 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:36.124894 27805 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:36.125770 27805 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:36.047006 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.931939 27797 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.119059 27783 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.128047 27783 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:35.993567 27405 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:35.834050 27828 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:36.126150 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.126958 27859 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:36.132191 27859 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:36.099066 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:36.133286 27859 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:35.994161 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67215us
I20260709 13:13:36.125578 27753 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.118345 22757 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:13:36.132231 22914 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:36.141682 22757 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:36.142184 22913 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:36.112823 22834 rpcz_store.cc:276] 0709 13:13:34.841849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.968766 (+126917us) service_pool.cc:224] Handling call
0709 13:13:34.969114 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.761384 (+792270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.000212 (+238828us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b5400322dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:36.000476 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820285824}
I20260709 13:13:35.968176 22763 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:35.937687 22764 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:36.142344 22757 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:36.089737 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 758) took 1301 ms (client timeout 1624 ms). Trace:
I20260709 13:13:36.135718 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:36.146242 22832 rpcz_store.cc:269] 0709 13:13:34.788721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.790681 (+  1960us) service_pool.cc:224] Handling call
0709 13:13:36.089327 (+1298646us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54000398d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:36.089625 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752607616}
I20260709 13:13:36.148231 27733 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:36.148991 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.139108 27721 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.150594 27721 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.135084 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:36.159271 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.160228 23461 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.162144 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2026) took 1289 ms. Trace:
I20260709 13:13:36.162719 22836 rpcz_store.cc:276] 0709 13:13:34.872432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.925017 (+ 52585us) service_pool.cc:224] Handling call
0709 13:13:36.161907 (+1236890us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b5400320ad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:36.162079 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1342348160}
I20260709 13:13:36.164014 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.164767 23466 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.167114 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:36.171326 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:36.173368 27859 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.183275 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:35.922971 27796 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:35.997028 27863 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:13:36.207224 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.210302 22837 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:36.210791 27759 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:35.998703 27817 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.256s	user 0.002s	sys 0.002s
I20260709 13:13:36.028051 27833 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.217818 27816 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:36.220212 22765 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.986655 27811 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:35.970952 27529 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:35.972509 27792 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:36.147370 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:36.233852 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:35.933264 27609 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.295s
W20260709 13:13:36.252494 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 323352us
I20260709 13:13:36.245224 22909 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:36.261480 22757 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.262585 22765 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:36.209053 27796 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.245781 27639 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:36.230353 27811 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:36.268694 27796 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:36.269008 27811 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:36.269342 27639 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.115972 27732 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.267230 22838 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:36.273666 27807 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:36.274191 27807 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.275594 27807 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.282703 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:36.283569 27831 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.284123 27609 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:36.284601 27609 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.285604 27609 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.286863 27751 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:36.287606 27609 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:36.289405 27858 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:36.289659 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:36.290661 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 908) took 1236 ms. Trace:
I20260709 13:13:36.290916 27792 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:36.291070 23464 rpcz_store.cc:276] 0709 13:13:35.054557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.055411 (+   854us) service_pool.cc:224] Handling call
0709 13:13:35.055805 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.557972 (+502167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.290395 (+732423us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54002c9554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:36.290594 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111362304}
I20260709 13:13:36.296177 22837 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:36.297984 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.300956 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:36.302877 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.303169 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 155915us
I20260709 13:13:36.304776 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:36.307250 23461 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:36.301885 22837 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:36.314752 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.312780 23464 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:36.316876 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:36.326802 22832 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:36.079412 27699 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.327400 22757 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:36.229071 27759 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:36.321324 27860 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.040s	sys 0.063s
I20260709 13:13:36.340627 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:36.364576 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.365353 22757 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:36.365754 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61397.4us
I20260709 13:13:36.361963 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:36.114457 27719 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.340032 22838 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:36.263652 27808 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.266417 27791 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:36.381896 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:36.386261 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:36.297981 27824 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.387694 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:36.397976 22837 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:36.344775 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:36.394371 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:36.236614 27522 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.399858 22911 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:36.308959 27873 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:36.396572 22761 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:36.396116 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:36.409868 22761 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:36.411176 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:36.413108 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46032.4us
I20260709 13:13:36.415097 27858 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:36.416157 27522 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:36.417057 27874 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:36.417769 27874 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.418848 27874 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.420318 27825 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:36.420847 27825 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:36.421306 27825 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:36.421953 22910 raft_consensus.cc:606] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:36.395517 22757 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:36.401079 22838 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.402555 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:36.383109 27860 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.040s	sys 0.064s
I20260709 13:13:36.343828 27688 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.228145 27816 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.423426 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:36.321988 27838 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:13:36.468381 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2009) took 2313 ms. Trace:
W20260709 13:13:36.461953 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47893.9us
I20260709 13:13:36.484925 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:36.469558 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66990.8us
W20260709 13:13:36.472391 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57762.4us
I20260709 13:13:36.327690 27878 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:36.364578 27724 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.392053 27808 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:36.344403 27601 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 13:13:36.477133 22833 rpcz_store.cc:276] 0709 13:13:34.147071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.147254 (+   183us) service_pool.cc:224] Handling call
0709 13:13:34.147634 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:34.157257 (+  9623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:34.157306 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:34.160475 (+  3169us) log.cc:844] Serialized 26 byte log entry
0709 13:13:34.620036 (+459561us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:34.620079 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.620152 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:35.997878 (+1377726us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:36.406526 (+408648us) raft_consensus.cc:1685] finished
0709 13:13:36.406561 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.460318 (+ 53757us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9142,"raft.queue_time_us":117320,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5055,"spinlock_wait_cycles":24996608,"thread_start_us":402615,"threads_started":1,"wal-append.queue_time_us":473901}
I20260709 13:13:36.498266 22838 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:36.385721 27792 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:13:36.528980 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:36.460443 27873 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.399814 27831 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:13:36.476358 27798 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:36.494148 27405 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:36.493891 27878 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:36.395516 27783 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 13:13:36.474462 27558 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:13:36.412307 27824 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.496392 27879 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:36.567543 27699 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:36.569482 27833 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:36.572491 27783 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.573079 27783 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.574599 27838 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:36.575433 27783 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:36.579276 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.575176 27838 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.389335 27796 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.496658 27429 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:13:36.492462 27688 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.611303 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:36.631486 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.622082 23466 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:36.634886 22762 pending_rounds.cc:85] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:13:36.635509 22762 pending_rounds.cc:107] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:36.552009 27724 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:36.637087 22762 raft_consensus.cc:2889] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:36.556445 27831 consensus_queue.cc:1243] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:13:36.649727 22766 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:36.539625 27759 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.604512 27879 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:36.647030 27788 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.001s
I20260709 13:13:36.660291 27429 consensus_queue.cc:1243] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:36.685684 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:36.690165 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.692706 23465 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:13:36.693996 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:36.696715 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 914) took 1134 ms. Trace:
I20260709 13:13:36.702827 27720 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:36.703348 27720 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:36.704339 27720 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.710168 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1839) took 1927 ms. Trace:
I20260709 13:13:36.710683 22908 rpcz_store.cc:276] 0709 13:13:34.783186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.263801 (+480615us) service_pool.cc:224] Handling call
0709 13:13:35.264110 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.932662 (+668552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.709799 (+777137us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b540010e0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:36.710072 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303167744}
I20260709 13:13:36.697225 23458 rpcz_store.cc:276] 0709 13:13:35.562285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.562721 (+   436us) service_pool.cc:224] Handling call
0709 13:13:36.696483 (+1133762us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b540063f954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:36.696636 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78828800}
I20260709 13:13:36.718276 27700 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:36.719980 27818 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:36.720949 27818 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:36.722077 27818 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:36.722494 27818 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.724468 27818 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.728170 27755 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:13:36.728555 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.729141 27755 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:36.729749 27755 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:36.731130 27429 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:36.730912 27755 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:36.734531 27865 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.707s	user 0.003s	sys 0.001s
I20260709 13:13:36.735392 27833 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.736106 27833 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.736886 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 664) took 1133 ms. Trace:
W20260709 13:13:36.737987 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.739528 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.740207 22766 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.740653 27720 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.741660 27798 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:36.742120 27818 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.747499 27755 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:36.742245 27798 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:36.748371 27429 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.749716 27791 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.750231 27798 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:36.751260 27429 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:36.736943 27712 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:36.747956 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:36.592068 27805 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.755218 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.766438 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.755400 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:36.737444 22758 rpcz_store.cc:276] 0709 13:13:35.603679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.604728 (+  1049us) service_pool.cc:224] Handling call
0709 13:13:35.605137 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:35.605575 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:13:35.729151 (+123576us) log.cc:844] Serialized 14 byte log entry
0709 13:13:35.912353 (+183202us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:35.912860 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:13:35.912996 (+   136us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:35.913218 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 13:13:35.915575 (+  2357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.915656 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.733917 (+818261us) spinlock_profiling.cc:243] Waited 924 us on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:36.736638 (+  2721us) raft_consensus.cc:1685] finished
0709 13:13:36.736664 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.736811 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2075,"prepare.queue_time_us":256529,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":4708,"raft.queue_time_us":11173,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":7971,"spinlock_wait_cycles":2125056,"thread_start_us":276929,"threads_started":2,"wal-append.queue_time_us":286060}
I20260709 13:13:36.771584 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:36.759660 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 227153us
I20260709 13:13:36.654409 27828 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:36.784668 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:36.785717 22838 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:36.773646 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:36.788535 22833 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.774780 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:36.777532 22837 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:36.790118 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.799024 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:36.799345 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:36.800803 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:36.801857 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54013us
W20260709 13:13:36.802883 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47608.1us
I20260709 13:13:36.906338 22908 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:36.823559 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:36.756521 27712 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:36.810686 27724 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:36.821765 27405 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:36.754446 27788 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.750281 27791 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.821900 27888 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:36.801086 22764 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:36.928272 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:36.930289 22764 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:36.821403 27831 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:36.933297 22764 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:36.902470 22911 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:36.652396 27878 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:36.902911 22758 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
W20260709 13:13:37.024497 23431 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.006s	sys 0.022s
W20260709 13:13:36.908084 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.908618 22908 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:36.909425 22766 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:36.909615 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:37.026172 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 116644us
I20260709 13:13:36.915876 22841 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:36.918391 22906 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:13:36.901957 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:37.027603 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 225367us
I20260709 13:13:37.028731 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:36.902369 22912 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:37.030023 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:36.809839 27719 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.944379 27796 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:36.945039 27873 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:36.946481 22835 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:36.948979 22765 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:36.955041 22838 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:13:36.919329 27828 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:36.914705 27724 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.958631 22836 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:13:36.940582 27609 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.003s
W20260709 13:13:36.940716 27887 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
W20260709 13:13:36.968943 27753 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.766s	user 0.012s	sys 0.001s
I20260709 13:13:36.972856 27897 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:37.015972 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:37.024971 23431 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.006s	sys 0.022s
I20260709 13:13:37.025388 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:37.026080 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:37.064177 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 758148us
I20260709 13:13:37.026724 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.065155 23431 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:36.905575 27839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.010s	sys 0.029s
I20260709 13:13:37.031075 22840 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:36.876004 27879 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:37.066860 27839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.010s	sys 0.029s
I20260709 13:13:37.067463 27839 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:37.067962 27839 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.039700 22832 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:13:37.040500 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:37.071625 22906 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:37.034123 27700 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.035110 27655 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.037209 27798 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.035471 27899 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.041227 27858 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:37.006197 27787 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:13:37.045660 27738 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:37.073696 22764 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:37.040413 27751 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:37.046350 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:37.060894 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 375365us
I20260709 13:13:37.092461 22758 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:13:37.058820 27724 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:37.058120 27687 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.094732 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 119686us
I20260709 13:13:37.095424 22758 pending_rounds.cc:85] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 6
I20260709 13:13:37.096042 22758 pending_rounds.cc:107] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:37.049717 27898 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:37.096868 22758 raft_consensus.cc:2889] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:36.998383 27712 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.075743 27833 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.076606 27824 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.070348 27904 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:37.039768 27816 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.087865 27805 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:37.055205 27719 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.097560 27838 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.070137 27753 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.036945 27405 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.098899 27858 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:37.088451 27639 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:13:37.070466 27895 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:13:37.130260 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2735) took 1337 ms. Trace:
I20260709 13:13:37.146088 22760 rpcz_store.cc:276] 0709 13:13:35.793104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.255048 (+461944us) service_pool.cc:224] Handling call
0709 13:13:36.255418 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.263824 (+  8406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.263861 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.291845 (+ 27984us) log.cc:844] Serialized 26 byte log entry
0709 13:13:36.465518 (+173673us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:36.465567 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.465642 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.130027 (+664385us) raft_consensus.cc:1685] finished
0709 13:13:37.130053 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.130181 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7650,"raft.queue_time_us":15827,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"thread_start_us":173334,"threads_started":1,"wal-append.queue_time_us":263887}
W20260709 13:13:37.129494 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.150171 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.116596 27796 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:37.136582 27893 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:13:37.141853 27891 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:13:37.098377 27522 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:13:37.100942 27687 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.085948 27700 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.144146 22911 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:37.089222 27899 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.148979 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:37.109799 27751 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:37.154685 27838 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:37.201836 27838 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:37.167284 27788 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.202411 27912 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:37.094067 27655 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.075528 27897 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.169569 27796 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.128131 27686 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.141201 22833 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:37.169288 27839 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.155612 22837 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:13:37.167626 27859 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.994s	user 0.012s	sys 0.003s
W20260709 13:13:37.126801 27061 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.190s	user 0.020s	sys 0.011s
I20260709 13:13:37.197392 27753 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:37.091238 27798 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.207552 27061 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:37.208185 27896 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:13:37.209424 27798 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.153863 27712 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:37.210337 27892 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 13:13:37.210672 27061 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.211539 27522 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:37.211931 27522 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:37.214227 27908 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:37.166183 27880 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
W20260709 13:13:37.172683 27380 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.056s	user 0.010s	sys 0.005s
I20260709 13:13:37.203841 27912 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:37.218941 27912 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:37.202430 27838 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:37.219951 27912 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:37.220867 27840 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.106s	user 0.011s	sys 0.001s
W20260709 13:13:37.221206 27733 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.072s	user 0.007s	sys 0.003s
I20260709 13:13:37.167099 27824 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.209933 22766 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:37.205826 27686 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.221298 27912 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:37.198951 27700 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.222506 22914 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:37.234601 27825 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.813s	user 0.011s	sys 0.003s
I20260709 13:13:37.235276 27825 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.238457 27825 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:37.239595 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 193436us
W20260709 13:13:37.242507 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 140724us
I20260709 13:13:37.234133 27791 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.153280 27902 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:13:37.230407 27733 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:37.205783 27796 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.263962 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:37.271620 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:37.212844 27522 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.264708 22757 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:37.265206 27405 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.205327 27897 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:37.261516 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:37.272797 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:37.273110 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:37.272797 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:37.275602 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:37.230470 27824 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:37.262174 27900 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:13:37.278467 22757 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:37.281165 22763 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:37.285444 23458 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:37.289994 27914 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:37.290815 27914 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:37.288228 27534 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:13:37.291455 27898 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:37.291917 27898 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:37.292286 27921 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:37.292841 27921 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.299038 27899 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.290295 27910 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.264094 23460 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:37.287478 22915 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:37.308504 22915 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 13:13:37.284384 27888 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:37.311684 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:37.292856 27898 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.320708 27910 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.321904 27879 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:37.322427 27753 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:37.322811 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.339517 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
W20260709 13:13:37.336809 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 56933.7us
I20260709 13:13:37.340704 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:37.374102 22833 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:37.375746 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:37.377030 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:37.380932 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.387316 22764 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:37.392617 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:37.393546 22832 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:37.395252 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2870) took 1593 ms. Trace:
I20260709 13:13:37.287717 27878 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:13:37.326260 27888 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.403299 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.334110 27788 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:37.287184 27405 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.401909 22836 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:37.379979 27899 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:37.337273 27828 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:37.386590 27755 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.005s
I20260709 13:13:37.401516 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:37.416337 22841 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:37.296154 27751 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.406843 27888 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:37.268477 27858 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.370656 27753 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:37.390738 27898 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:37.395699 22907 rpcz_store.cc:276] 0709 13:13:35.801602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.801740 (+   138us) service_pool.cc:224] Handling call
0709 13:13:35.823954 (+ 22214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:35.878491 (+ 54537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.878545 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:35.879298 (+   753us) log.cc:844] Serialized 26 byte log entry
0709 13:13:35.894688 (+ 15390us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:35.894751 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.379522 (+1484771us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b54001b9ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:37.379597 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.379650 (+    53us) raft_consensus.cc:1685] finished
0709 13:13:37.379662 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.395181 (+ 15519us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":751,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":79159,"spinlock_wait_cycles":5602048,"thread_start_us":66264,"threads_started":2,"wal-append.queue_time_us":115717,"wal-append.run_cpu_time_us":5743,"wal-append.run_wall_time_us":1005841}
I20260709 13:13:37.325140 27902 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.078s
W20260709 13:13:37.415391 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.409610 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.386415 27733 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:13:37.432299 27922 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.414535 27899 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:37.434891 22806 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.063s (SENT) [suppressed 9 similar messages]
I20260709 13:13:37.440955 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:37.441917 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.444756 27905 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:13:37.445132 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:37.446333 27927 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.447325 27927 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.424219 27840 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:13:37.439376 27899 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:37.450837 27839 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:37.500766 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 160153us
I20260709 13:13:37.465040 27898 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:37.475639 27888 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.433019 27922 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.416597 27788 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.449990 27686 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.497004 27713 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:37.512116 27934 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.496272 27828 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.455600 27897 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:13:37.502144 27753 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:37.513756 27919 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:13:37.754462 27686 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.513801 27916 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.372s
I20260709 13:13:37.564102 27910 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.756072 27686 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.513520 27923 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.559226 27917 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
I20260709 13:13:37.767014 27923 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.559638 27927 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:37.706862 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.574785 27886 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.738s	user 0.003s	sys 0.000s
I20260709 13:13:37.503921 27719 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.514000 27934 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.512813 27713 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:37.800015 27429 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:37.593914 27922 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:37.802942 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 374985us
I20260709 13:13:37.807238 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:37.808574 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:37.811637 27405 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.514292 27839 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:37.811849 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:37.813541 27824 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:37.595388 27888 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:37.803026 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 367989us
W20260709 13:13:37.832930 27720 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.088s	user 0.016s	sys 0.000s
I20260709 13:13:37.836243 27897 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:37.837582 27897 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.838274 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:37.839812 23464 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:37.842347 27897 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:37.846439 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.846156 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:37.631202 27899 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.852041 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2089) took 2058 ms. Trace:
I20260709 13:13:37.863128 22834 raft_consensus.cc:2368] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:37.864444 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:37.865029 27910 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.866451 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.866154 27910 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.852605 22759 rpcz_store.cc:276] 0709 13:13:35.793879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.258900 (+465021us) service_pool.cc:224] Handling call
0709 13:13:36.259277 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:36.285434 (+ 26157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.285470 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:36.286175 (+   705us) log.cc:844] Serialized 52 byte log entry
0709 13:13:36.394599 (+108424us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:36.414816 (+ 20217us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.645889 (+231073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.645982 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.560723 (+914741us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:37.851339 (+290616us) raft_consensus.cc:1685] finished
0709 13:13:37.851505 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.851786 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13665,"prepare.queue_time_us":26878,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":67521,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":20045,"spinlock_wait_cycles":120403840,"thread_start_us":329118,"threads_started":2,"wal-append.queue_time_us":373493}
I20260709 13:13:37.875933 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.878201 22762 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:37.512949 27926 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.431s
I20260709 13:13:37.568580 27929 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.270s
W20260709 13:13:37.884303 27811 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.609s	user 0.020s	sys 0.002s
I20260709 13:13:37.891785 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:37.894136 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2872) took 1582 ms. Trace:
I20260709 13:13:37.895313 27916 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.896414 27916 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.897607 27788 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.901365 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:37.903024 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46252 (request call id 39) took 1902 ms. Trace:
I20260709 13:13:37.903985 22910 rpcz_store.cc:276] 0709 13:13:36.000506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.098880 (+ 98374us) service_pool.cc:224] Handling call
0709 13:13:37.902388 (+1803508us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540010e5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aeb853 00007f68a6ab960a 00007f68a003f7d4 00007f68a003f762 00007f68a003f691 00007f68a003f60c 00007f68a003e8c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:37.902608 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662618496}
I20260709 13:13:37.908006 27824 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:37.891288 27429 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:37.908438 23460 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 583) took 1773 ms (client timeout 1541 ms). Trace:
I20260709 13:13:37.909144 27824 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:37.909103 23460 rpcz_store.cc:269] 0709 13:13:36.135243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.135389 (+   146us) service_pool.cc:224] Handling call
0709 13:13:37.907948 (+1772559us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540068ef54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:37.908157 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464704896}
I20260709 13:13:37.911710 27429 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.888631 27755 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.913034 27755 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.916792 27755 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.905109 27839 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.922027 22760 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:37.931594 27601 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:37.932159 27601 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:37.922979 27751 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:37.937865 27751 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.940680 27751 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:37.924638 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.942008 22836 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:37.942529 27720 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:37.943439 27720 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:37.944547 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.947089 27720 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:37.950004 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:37.950971 27923 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.951089 23457 raft_consensus.cc:1217] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Deduplicated request from leader. Original: 9.8->[11.9-11.9]   Dedup: 11.9->[]
I20260709 13:13:37.951730 23457 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:13:37.954763 27910 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.956931 27928 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:37.957623 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:37.968375 27380 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:37.971926 27380 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.972891 27380 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:37.976549 27688 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.976899 27380 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:37.894697 22913 rpcz_store.cc:276] 0709 13:13:36.311190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.312019 (+   829us) service_pool.cc:224] Handling call
0709 13:13:36.364363 (+ 52344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.656441 (+292078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.656489 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.666243 (+  9754us) log.cc:844] Serialized 26 byte log entry
0709 13:13:37.005269 (+339026us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:37.005329 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.893184 (+887855us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54001e2dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:37.893320 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:37.893431 (+   111us) raft_consensus.cc:1685] finished
0709 13:13:37.893447 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.893892 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":8947,"spinlock_wait_cycles":670470528,"thread_start_us":102048,"threads_started":1,"wal-append.queue_time_us":362207}
I20260709 13:13:37.993991 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.983503 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.996397 27902 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:13:38.067481 27700 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.068295 27754 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:38.068346 27700 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.069384 27754 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.067369 22760 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:38.011535 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2050) took 1090 ms. Trace:
I20260709 13:13:37.992127 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:38.007131 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:37.999796 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2102) took 1044 ms. Trace:
I20260709 13:13:38.024431 27687 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.008876 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2101) took 1058 ms. Trace:
I20260709 13:13:38.081722 22838 rpcz_store.cc:276] 0709 13:13:36.920996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.921132 (+   136us) service_pool.cc:224] Handling call
0709 13:13:36.921538 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:37.046748 (+125210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.010783 (+964035us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b54000c62d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:38.011265 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76439296}
W20260709 13:13:38.084242 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 362490us
I20260709 13:13:38.084688 27926 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.085210 27655 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.028808 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.081940 22766 rpcz_store.cc:276] 0709 13:13:36.954974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.026295 (+ 71321us) service_pool.cc:224] Handling call
0709 13:13:37.999110 (+972815us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540002dd54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:37.999480 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332434304}
I20260709 13:13:38.095945 22762 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:38.097057 27926 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.061007 27601 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:38.033524 27885 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.237s	user 0.000s	sys 0.006s
I20260709 13:13:37.986572 27824 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.086550 22765 rpcz_store.cc:276] 0709 13:13:36.950548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.101025 (+150477us) service_pool.cc:224] Handling call
0709 13:13:38.007643 (+906618us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:13:38.007948 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.008351 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.008381 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:38.008441 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:38.008472 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.008526 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.008674 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10440960}
I20260709 13:13:38.027559 27899 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:37.996330 27720 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.153s
I20260709 13:13:38.098918 27601 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:38.165387 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 69786.5us
W20260709 13:13:38.156024 27838 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.638s	user 0.014s	sys 0.000s
I20260709 13:13:38.169729 27943 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.169660 27838 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.170794 27943 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.163779 22759 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:37.967993 27811 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.173353 27838 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.010651 27936 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.158s
I20260709 13:13:38.029595 27797 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.188082 22764 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:38.185978 23466 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.041s	sys 0.053s
I20260709 13:13:37.989250 27713 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.009298 27937 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.168s
I20260709 13:13:38.190176 23465 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:38.192078 23466 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.041s	sys 0.053s
I20260709 13:13:38.176261 27902 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:38.195647 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.175067 27943 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.210379 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:38.211263 27700 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.211916 22764 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:13:38.213179 27754 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.216573 27811 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:38.217737 27811 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:38.218549 27902 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.222568 27751 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.223466 27792 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:38.223609 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:38.232483 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 275285us
I20260709 13:13:38.234742 22912 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:38.235049 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.237854 27686 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.242326 27792 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.242803 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:38.244689 27751 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.249917 27926 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.241351 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:38.260313 22765 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:38.267726 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.273460 23462 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:38.272552 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:38.263983 27872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.009s	sys 0.009s
I20260709 13:13:38.273181 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.280170 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.284119 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.286638 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.289816 22837 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:38.290733 22841 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:38.291877 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.292578 22841 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:38.266175 27755 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.279217 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.293586 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:38.277145 27872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.009s	sys 0.011s
I20260709 13:13:38.298837 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.299019 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.315609 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.325667 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.328066 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.291898 27902 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.330989 22836 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 17 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:13:38.333318 23461 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:38.336054 27754 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.339533 27754 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.343382 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.345054 27522 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:38.344735 27755 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.345161 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.283624 27797 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:38.350543 27792 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.351198 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:38.351490 22832 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:38.351830 27753 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:38.352662 27840 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:38.355634 27912 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:38.357806 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2876) took 1422 ms. Trace:
I20260709 13:13:38.361195 22838 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:38.362602 27807 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:38.370051 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 189754us
I20260709 13:13:38.330276 22834 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
W20260709 13:13:38.313020 27913 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.053s	user 0.000s	sys 0.005s
I20260709 13:13:38.347288 27754 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:38.348680 27808 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.806s	user 0.013s	sys 0.015s
W20260709 13:13:38.401597 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70288.9us
I20260709 13:13:38.303081 27811 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.400894 22912 rpcz_store.cc:276] 0709 13:13:36.935803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.030285 (+ 94482us) service_pool.cc:224] Handling call
0709 13:13:37.030555 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.272395 (+1241840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.357295 (+ 84900us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000447d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:38.357710 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2691886336}
I20260709 13:13:38.334836 27926 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:38.416229 22839 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:38.419317 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:38.412380 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61371.8us
I20260709 13:13:38.427309 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:38.430317 27912 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.416285 27721 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.433372 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:38.438820 27732 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.440182 27732 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.400327 27840 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:38.412273 27792 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:38.451006 22836 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:38.452876 27751 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.426383 27928 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:13:38.455029 27724 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.344s	user 0.020s	sys 0.005s
I20260709 13:13:38.455451 27721 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:38.443271 27808 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.442713 27840 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.443009 27754 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.459805 27755 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.460283 27732 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:38.461549 27934 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.487134 22757 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:38.488217 27953 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.467564 22912 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:38.488963 27953 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.468253 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:38.476649 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.491119 23462 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:38.491570 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.492511 27840 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.494773 27686 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:38.495600 27686 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.480600 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:38.498052 27724 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.499176 22765 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:38.482717 27754 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.501102 27724 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.502094 27754 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.504393 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:38.506142 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:38.506457 27828 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:38.486704 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:38.506809 22758 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:38.522332 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:38.521490 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:38.520994 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:38.512470 22841 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:38.527259 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.477767 27721 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.535517 22832 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:38.524296 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:38.543565 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 56940.5us
I20260709 13:13:38.548573 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.524865 27699 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:38.547308 27721 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:38.548202 27792 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.558996 22764 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:13:38.560129 27805 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:38.560623 27805 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.560989 27805 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:38.565095 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.566325 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.566597 27732 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.567309 27732 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.567654 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:38.569013 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.571825 27724 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:38.572194 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:38.573079 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.577894 22765 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:38.577944 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:38.578982 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:38.579051 23460 raft_consensus.cc:2376] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13. Re-sending same reply.
I20260709 13:13:38.579902 22765 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:38.581157 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:38.582149 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 107106us
I20260709 13:13:38.582665 22759 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.585776 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:38.585958 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:38.586648 27921 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.587364 23463 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:38.531148 27811 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:38.521008 27828 consensus_queue.cc:1243] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:38.589730 27957 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:38.591243 27912 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.591987 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.592181 22910 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:38.510637 27755 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.593595 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.593952 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.594693 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:38.594874 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.522511 27686 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.595059 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.547168 27807 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:38.627408 23457 raft_consensus.cc:2376] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13. Re-sending same reply.
I20260709 13:13:38.596074 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:38.634423 23460 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:38.599145 22915 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:38.599895 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.600445 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.600834 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.601153 27724 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:38.599154 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:38.622874 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.635023 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:38.630865 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.630579 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:38.582175 27945 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:38.561303 27792 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.595777 23459 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:38.637312 22760 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:38.638434 22914 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:38.640955 22833 raft_consensus.cc:2376] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13. Re-sending same reply.
I20260709 13:13:38.644055 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.644606 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.644739 27755 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.645172 22912 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:38.679745 22832 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:38.619457 27732 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.646847 27945 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.649855 23465 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:38.651432 27807 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.615186 27934 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:38.652834 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 128797us
I20260709 13:13:38.660303 27724 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.635861 27912 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:38.644652 27868 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:13:38.639053 22907 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:38.669055 27808 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.668890 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.719039 22841 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:38.640962 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:38.672825 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.645949 27957 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.724169 27792 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.724452 27957 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.643198 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:38.681473 23464 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:38.725266 27792 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:38.726773 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 89586.5us
I20260709 13:13:38.728204 27792 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.729027 27754 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:38.734822 22765 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:38.686537 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60369.1us
W20260709 13:13:38.687985 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 102141us
I20260709 13:13:38.738559 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:38.663516 27914 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.740025 27914 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:38.694659 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:38.697328 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.744149 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:38.692346 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:38.745165 22915 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:38.745057 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:38.753310 27811 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:38.762480 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:38.765378 27603 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:38.640427 27601 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:38.772051 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.775789 22757 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:38.776847 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:38.777158 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 145608us
I20260709 13:13:38.687819 27945 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.783583 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.783614 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.785938 22912 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:38.792649 23464 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:38.773662 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:38.808590 23460 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:38.792950 22908 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:38.819698 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.795706 27720 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:38.694546 27934 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.843551 22759 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:38.844316 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70754.5us
I20260709 13:13:38.693658 27753 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:13:38.846800 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.839745 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.855325 23464 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:38.814036 27937 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:38.828469 27720 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.793720 27797 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.854131 23459 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
W20260709 13:13:38.870158 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 125203us
W20260709 13:13:38.859450 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 121065us
W20260709 13:13:38.869050 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.874529 22906 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
W20260709 13:13:38.875124 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:38.861153 27952 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:13:38.874091 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:38.856097 27686 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:38.876719 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:38.877902 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.874336 22915 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:38.862015 27522 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.868274 27720 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.860839 27899 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:38.893594 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:38.888499 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 193978us
I20260709 13:13:38.889011 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:38.872038 27808 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:38.887857 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.892927 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.900697 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:38.898986 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.902598 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:38.904592 23459 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:13:38.900527 22834 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
W20260709 13:13:38.913444 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.882930 27686 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:38.890450 27912 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.899369 27808 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.922654 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.923069 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.930778 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:38.913002 27957 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:38.918118 27686 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:38.936432 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.938246 22761 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:38.939888 22759 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:38.938571 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:38.942009 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:38.942373 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:38.944169 22841 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:38.892109 27522 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.901906 27797 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:38.922129 27807 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:38.948755 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74777us
I20260709 13:13:38.928673 27808 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:38.926319 27751 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.948860 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:38.933521 27873 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:38.951010 27899 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:38.956754 27724 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:38.948064 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2056) took 1501 ms. Trace:
W20260709 13:13:38.948935 27954 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:13:38.922021 27828 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:38.919346 27921 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:13:38.979532 23464 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:38.943171 22764 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:38.943936 27713 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:38.974179 22840 rpcz_store.cc:276] 0709 13:13:37.374480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.375397 (+   917us) service_pool.cc:224] Handling call
0709 13:13:37.375669 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:37.752476 (+376807us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:37.752561 (+    85us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:37.752630 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:37.752723 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.862710 (+1109987us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540028b4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:38.862764 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.875757 (+ 12993us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809438976}
I20260709 13:13:38.939062 27732 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:39.000613 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2734) took 3129 ms. Trace:
I20260709 13:13:38.945044 27753 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
W20260709 13:13:39.013727 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 140943us
I20260709 13:13:38.994910 22915 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:39.016610 22763 rpcz_store.cc:276] 0709 13:13:35.792817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.222887 (+430070us) service_pool.cc:224] Handling call
0709 13:13:36.253354 (+ 30467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:37.296617 (+1043263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.877069 (+1580452us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400002dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:38.922261 (+ 45192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363332992}
I20260709 13:13:39.021793 23465 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:39.018726 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.015798 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 925) took 1658 ms. Trace:
I20260709 13:13:38.980120 27873 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:39.079242 27873 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:38.984822 27755 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:38.950451 27805 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.084306 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.084702 27805 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:39.085150 22907 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:39.025110 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:39.025724 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:39.086483 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 137744us
I20260709 13:13:38.939026 27912 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:39.031163 27947 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.795s	user 0.001s	sys 0.009s
I20260709 13:13:39.032130 27529 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:39.087956 22879 leader_election.cc:400] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:39.088291 22879 leader_election.cc:403] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:39.088335 27529 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:39.090384 27958 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:13:39.091656 27529 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.034596 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.035774 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:39.014670 27721 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.023404 23466 rpcz_store.cc:276] 0709 13:13:37.346466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.346631 (+   165us) service_pool.cc:224] Handling call
0709 13:13:37.399238 (+ 52607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.243294 (+844056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.243371 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.244143 (+   772us) log.cc:844] Serialized 26 byte log entry
0709 13:13:38.295866 (+ 51723us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:38.295922 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.315000 (+ 19078us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:38.975202 (+660202us) raft_consensus.cc:1685] finished
0709 13:13:38.975240 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.005195 (+ 29955us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":306,"raft.queue_time_us":78206,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7302,"thread_start_us":844437,"threads_started":2,"wal-append.queue_time_us":286497}
I20260709 13:13:39.037479 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2103) took 2076 ms. Trace:
I20260709 13:13:39.038338 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.040578 27806 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.041291 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.094267 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:39.098104 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 76982.6us
I20260709 13:13:39.044261 22759 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:39.048158 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 675) took 2705 ms. Trace:
W20260709 13:13:39.050300 27950 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.608s	user 0.003s	sys 0.000s
I20260709 13:13:39.051764 27732 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:39.055543 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:39.056152 27792 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:39.095788 22763 raft_consensus.cc:2376] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13. Re-sending same reply.
I20260709 13:13:39.102938 22765 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.057696 27929 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.060437 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.060676 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.061805 27933 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:39.062988 27921 consensus_queue.cc:1243] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:13:39.065896 27934 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:38.995945 27807 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:39.133852 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.002552 27751 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.106321 27792 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.108299 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.143664 22764 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:39.098564 22766 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:39.096467 22758 rpcz_store.cc:276] 0709 13:13:36.961252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.101446 (+140194us) service_pool.cc:224] Handling call
0709 13:13:37.123809 (+ 22363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.569553 (+1445744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.037240 (+467687us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400369554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:39.037421 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3168503552}
I20260709 13:13:39.100581 22761 rpcz_store.cc:276] 0709 13:13:36.342319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.411466 (+ 69147us) service_pool.cc:224] Handling call
0709 13:13:36.411845 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.938738 (+2526893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.047773 (+109035us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b540035f054. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:39.048085 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5431082240}
I20260709 13:13:39.108441 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.116493 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.116679 22757 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:39.117249 27754 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:39.122381 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.124092 27916 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:39.124006 27929 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.094244 27805 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:39.155153 22762 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:13:39.137373 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:39.161368 27792 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:39.092583 27529 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:13:39.148175 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.164337 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.084673 27687 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:13:39.115702 27755 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.151973 27934 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:39.153306 27929 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.162734 27959 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.177749 27755 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.178066 27959 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.148658 27807 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.180886 27755 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:39.148844 27754 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:39.138796 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.189415 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.181136 27807 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:39.151541 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53506us
I20260709 13:13:39.135260 27753 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:39.092484 27828 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.174492 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.207650 22763 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:39.148487 27724 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:39.182063 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.165333 23464 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:39.125883 27700 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:39.190148 22766 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:39.190407 22759 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:13:39.167733 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.181005 27959 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.191233 22761 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:39.192065 27754 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:39.192869 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.149039 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.202533 27934 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:13:39.176334 27953 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.223572 22915 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:39.195181 27529 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:39.196236 27687 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.224267 22911 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:39.147172 27806 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.236181 22759 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:39.248394 22763 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:13:39.248438 22914 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:13:39.255483 22914 pending_rounds.cc:85] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 12
I20260709 13:13:39.256414 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:39.229472 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81574.8us
I20260709 13:13:39.204293 27753 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.230154 22841 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:39.216674 27700 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:39.258544 22909 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:39.270748 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.272276 22759 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:13:39.275138 27529 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:39.276011 27721 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:39.277937 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 163984us
W20260709 13:13:39.279422 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.722s (SENT)
I20260709 13:13:39.279881 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.255802 22914 pending_rounds.cc:107] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:13:39.281718 22914 raft_consensus.cc:2889] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:39.221349 27724 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.223750 27916 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:13:39.247601 27687 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:39.287482 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:39.294680 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 297000us
W20260709 13:13:39.295670 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.297884 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.296845 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.271592 27753 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:39.287547 22760 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.593s	user 0.014s	sys 0.000s
I20260709 13:13:39.304766 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:39.235173 27720 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.290212 27921 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:39.309361 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.310416 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:39.294273 27929 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:39.294786 27687 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.328078 23464 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:39.300225 27805 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:39.339498 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.358945 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:39.360525 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:39.362673 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99932.3us
I20260709 13:13:39.361189 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.361732 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:39.367225 22765 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:39.369609 22766 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:39.369870 23466 raft_consensus.cc:2410] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:13:39.371028 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.371923 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:39.372301 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 596) took 1140 ms. Trace:
I20260709 13:13:39.331775 27899 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.372654 23458 rpcz_store.cc:276] 0709 13:13:38.231466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.231625 (+   159us) service_pool.cc:224] Handling call
0709 13:13:39.372125 (+1140500us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54006112d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:39.372240 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571133312}
I20260709 13:13:39.373311 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.351039 27805 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:39.406894 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.303079 27732 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.380640 27953 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:39.421591 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 142285us
I20260709 13:13:39.413528 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.387869 27805 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:39.438119 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 128859us
I20260709 13:13:39.442636 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.444360 27977 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:39.432255 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 96235.8us
W20260709 13:13:39.452102 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78953.5us
I20260709 13:13:39.445233 27713 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:39.453042 27807 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.453572 27692 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.454201 27692 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.454741 27692 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.455155 27389 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:39.458205 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 267941us
I20260709 13:13:39.461321 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.430411 27953 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:39.494304 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:39.467573 27713 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.513692 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.389196 27700 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.484277 27953 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:39.484202 27828 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.481096 27921 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:39.483683 27603 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:39.525861 27974 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.476491 27914 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:13:39.527357 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.511821 27934 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:39.449923 27977 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.527153 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:39.530380 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 246950us
I20260709 13:13:39.507010 27811 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:39.471741 27686 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.534857 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.537599 27972 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:39.552335 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.495836 27389 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:13:39.540944 27967 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:13:39.530548 27808 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.003s
I20260709 13:13:39.532931 27700 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.530339 27974 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:39.554057 27945 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.005s
I20260709 13:13:39.532366 27806 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.554092 27972 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:39.535817 27828 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:13:39.557307 27959 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.533016 27921 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.573854 27720 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.574456 27974 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.637624 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 788) took 1084 ms. Trace:
I20260709 13:13:39.570254 27811 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:39.638124 22839 rpcz_store.cc:276] 0709 13:13:38.608137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.616116 (+  7979us) service_pool.cc:224] Handling call
0709 13:13:38.616510 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.715294 (+ 98784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.715343 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.794754 (+ 79411us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.224731 (+429977us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:39.224784 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.224863 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.637357 (+412494us) raft_consensus.cc:1685] finished
0709 13:13:39.637380 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.637556 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94564,"raft.queue_time_us":69438,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":9964,"thread_start_us":360317,"threads_started":1,"wal-append.queue_time_us":462922}
W20260709 13:13:39.639420 27873 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.559s	user 0.005s	sys 0.006s
I20260709 13:13:39.639513 27811 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.640385 27937 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:39.640792 27912 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.640894 27937 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:39.606740 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 79448.9us
I20260709 13:13:39.641923 27937 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.642951 27937 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.642877 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:39.610950 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76208.2us
I20260709 13:13:39.619082 22759 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 16. Ops: [14.9-14.9]
W20260709 13:13:39.622033 27983 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:13:39.621930 27899 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.645067 27899 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.645526 27899 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.616629 27908 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:13:39.623528 27979 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:13:39.584095 27945 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:39.625967 27975 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:13:39.627557 27601 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.628274 27868 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:39.629112 27968 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:13:39.629594 27966 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.243s	user 0.001s	sys 0.001s
I20260709 13:13:39.631178 27389 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:39.632308 27522 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:39.634388 27991 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:39.635191 27807 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:39.637015 27751 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.639914 27873 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:39.725592 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 173396us
I20260709 13:13:39.644603 27792 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.622553 27989 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:13:39.594823 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2128) took 1017 ms. Trace:
I20260709 13:13:39.599606 27724 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.589564 27934 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:39.662375 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.675510 27686 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.665271 27868 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:39.575666 27808 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:39.671315 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.671839 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.675182 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.743388 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:39.650985 27806 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:39.681138 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.686367 22909 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:39.660867 27914 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:39.691650 27942 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:39.764525 27942 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:39.764791 27914 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.765301 27914 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.765627 27942 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.766716 27942 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.767402 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2121) took 1238 ms. Trace:
I20260709 13:13:39.767802 22762 rpcz_store.cc:276] 0709 13:13:38.528833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.155964 (+627131us) service_pool.cc:224] Handling call
0709 13:13:39.156481 (+   517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:39.303781 (+147300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.303821 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:39.368319 (+ 64498us) log.cc:844] Serialized 52 byte log entry
0709 13:13:39.480987 (+112668us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:39.481028 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.481110 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.767130 (+286020us) raft_consensus.cc:1685] finished
0709 13:13:39.767168 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.767334 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8349,"raft.queue_time_us":15768,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6875,"thread_start_us":203261,"threads_started":2,"wal-append.queue_time_us":135141}
W20260709 13:13:39.698033 27957 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.736s	user 0.007s	sys 0.008s
I20260709 13:13:39.769932 27957 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:39.709218 27754 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.699857 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 789) took 1070 ms. Trace:
I20260709 13:13:39.771030 22838 rpcz_store.cc:276] 0709 13:13:38.629704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.722896 (+ 93192us) service_pool.cc:224] Handling call
0709 13:13:38.723487 (+   591us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:38.828690 (+105203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.828723 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:38.866532 (+ 37809us) log.cc:844] Serialized 52 byte log entry
0709 13:13:39.289183 (+422651us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:39.289225 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.289299 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.699445 (+410146us) raft_consensus.cc:1685] finished
0709 13:13:39.699562 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.699782 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69488,"raft.queue_time_us":116514,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11929,"thread_start_us":420958,"threads_started":1,"wal-append.queue_time_us":467723}
I20260709 13:13:39.772460 27754 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.772871 22761 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:39.773764 22764 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:39.774253 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.775125 22762 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:39.775174 22906 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:39.776410 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.777045 22762 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:39.703258 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2123) took 1173 ms. Trace:
I20260709 13:13:39.706861 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:39.779065 27945 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.707684 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2130) took 1110 ms. Trace:
I20260709 13:13:39.700258 27921 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.707661 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.709029 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:39.660645 27979 consensus_queue.cc:1243] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:13:39.713616 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.724758 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.622604 27959 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.726800 23431 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.604161 27993 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.675181 27797 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:39.674959 27389 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:39.809357 22911 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:39.744055 22912 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:39.749810 27806 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.750285 27991 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:39.734253 27792 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:39.756467 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 102285us
I20260709 13:13:39.757555 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:39.758478 27999 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:39.758824 27934 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.760347 27868 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.761593 28002 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:13:39.762928 27808 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:39.763912 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 108523us
W20260709 13:13:39.827322 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 300248us
I20260709 13:13:39.707724 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.780313 22766 rpcz_store.cc:276] 0709 13:13:38.597604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.217229 (+619625us) service_pool.cc:224] Handling call
0709 13:13:39.367821 (+150592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.374543 (+  6722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.707480 (+332937us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400222ed0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:39.707623 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307500288}
I20260709 13:13:39.771159 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:39.828619 22759 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.765323 27873 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.829936 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.801208 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:39.803810 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.732321 22757 rpcz_store.cc:276] 0709 13:13:38.577235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.213251 (+636016us) service_pool.cc:224] Handling call
0709 13:13:39.226923 (+ 13672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.260659 (+ 33736us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:39.427768 (+167109us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.590625 (+162857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:39.590714 (+    89us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:39.594231 (+  3517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.594354 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.594564 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":316749,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":118,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":8991,"thread_start_us":224048,"threads_started":2,"wal-append.queue_time_us":214052}
I20260709 13:13:39.722394 27807 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:39.810617 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83971.6us
I20260709 13:13:39.816533 22760 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:39.778653 22758 rpcz_store.cc:276] 0709 13:13:38.529461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.173142 (+643681us) service_pool.cc:224] Handling call
0709 13:13:39.218531 (+ 45389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.225157 (+  6626us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:39.225203 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:39.359785 (+134582us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:39.378049 (+ 18264us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.447928 (+ 69879us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:39.448057 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.448256 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.620919 (+172663us) raft_consensus.cc:1685] finished
0709 13:13:39.620944 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.703037 (+ 82093us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54001004d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:39.703187 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":24032,"spinlock_wait_cycles":41642752,"thread_start_us":61285,"threads_started":1,"wal-append.queue_time_us":229290}
I20260709 13:13:39.822992 22914 raft_consensus.cc:1217] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-11.9]   Dedup: 11.9->[]
I20260709 13:13:39.837867 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.838501 22914 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:13:39.823825 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:39.810151 27968 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:13:39.839318 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.818656 27921 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:39.774459 27903 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.806579 27977 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:39.782059 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.832319 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:39.828128 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:39.830943 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:39.834802 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.816291 27806 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:39.837361 22912 raft_consensus.cc:1217] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Deduplicated request from leader. Original: 10.8->[12.9-14.10]   Dedup: 12.9->[14.10-14.10]
I20260709 13:13:39.849864 27908 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:39.841490 22911 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:39.843292 27792 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.844934 27868 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.845831 27912 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.849612 28005 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:13:39.850021 22910 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:39.803475 27699 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:13:39.852972 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2062) took 1562 ms. Trace:
I20260709 13:13:39.805112 27692 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:39.824771 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 108896us
I20260709 13:13:39.862437 27999 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.862843 22912 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 13:13:39.862926 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.863624 28012 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:13:39.850567 27753 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.003s
W20260709 13:13:39.889183 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57392.3us
I20260709 13:13:39.890201 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:39.868083 22759 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:39.870503 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:39.871667 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 43630.9us
I20260709 13:13:39.851117 27921 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:39.814782 27993 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.866206 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.897907 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:39.876657 27995 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.900099 27921 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:39.900209 27808 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:39.900774 27732 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:39.901751 27979 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.902063 27713 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:39.902761 28006 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:39.902616 27942 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:39.903357 27873 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:39.888672 27999 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.888269 28001 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.949201 28001 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.951826 28001 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:39.904346 27753 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.907696 28005 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:39.880092 22837 rpcz_store.cc:276] 0709 13:13:38.290726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.301852 (+ 11126us) service_pool.cc:224] Handling call
0709 13:13:38.324612 (+ 22760us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.475732 (+151120us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:38.906163 (+430431us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.679896 (+773733us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:39.681882 (+  1986us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.682677 (+   795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:39.682804 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.683074 (+   270us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.852473 (+169399us) raft_consensus.cc:1685] finished
0709 13:13:39.852698 (+   225us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.852891 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8079,"prepare.queue_time_us":21542,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":80157,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":20328,"thread_start_us":1114141,"threads_started":3,"wal-append.queue_time_us":564360}
I20260709 13:13:39.893939 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:39.962062 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.895144 27805 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.886545 27389 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:39.975595 27389 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:39.980165 27753 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.932978 27686 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:39.928352 22907 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:39.878639 27991 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:39.989316 22914 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:39.934630 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:39.878382 27912 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.932701 27700 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:39.997582 27700 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:39.944564 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.886001 27914 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:39.954947 22915 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:39.939733 27979 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:39.984444 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61334.5us
I20260709 13:13:39.986570 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:39.991257 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.996167 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:39.955720 27807 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.013207 22907 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:40.014840 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:39.929944 27995 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:40.005904 27699 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:39.955591 27999 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.953745 27792 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.007464 28012 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:40.021853 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87365.5us
I20260709 13:13:39.933801 27984 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:13:39.988687 27686 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:39.925894 27899 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.935431 27953 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:39.987893 28005 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.000874 27912 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:39.926563 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87337.3us
I20260709 13:13:40.053640 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.016849 27979 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.020406 27995 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:40.063431 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 140254us
I20260709 13:13:40.032723 27686 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:40.032007 28012 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:40.069492 22763 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:40.057526 27713 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.080828 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:40.082543 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.026855 27792 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.063460 27991 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:40.072692 27751 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:40.106585 27991 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.107162 28012 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.107703 28020 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.108408 28020 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.110251 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.111164 22907 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:40.123636 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:40.126874 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.094751 28017 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:40.128219 28017 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:40.128827 28017 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:40.130106 22907 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.135771 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.136459 22759 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.139221 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:40.087379 27713 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.129010 27751 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.169611 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:40.160794 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.170564 22765 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.158772 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.171479 27713 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:40.162237 28005 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:40.174474 27974 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.004s
I20260709 13:13:40.159565 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.176200 22910 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.171528 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:40.177392 22757 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:40.171502 22909 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.139884 22758 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 8.
I20260709 13:13:40.178694 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.189157 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:40.135248 27687 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.005s
I20260709 13:13:40.201391 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:40.201961 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.203177 22907 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.216290 27805 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:40.217818 27977 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.223934 22906 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:40.226789 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.230044 22910 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:40.230371 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.231184 22908 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:40.133678 27908 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:40.162206 28011 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:40.235474 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.236158 22909 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:40.157387 27959 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.133397 28001 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:13:40.175211 27974 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.243429 28022 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:40.257789 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 168386us
I20260709 13:13:40.259698 22759 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:40.258718 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.222755 27687 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:40.259805 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 191967us
I20260709 13:13:40.264110 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:40.247305 27993 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:40.232349 27977 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.245873 27908 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.262408 27899 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.279374 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.291090 23458 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.290148 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.262244 27953 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.292915 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.248756 28011 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.302896 23464 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:40.303020 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.302915 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.300385 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:40.306525 23463 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.305337 23460 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:40.318246 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.280280 28023 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.286988 28012 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:40.262565 28022 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.298738 27903 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:40.355667 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.319108 28026 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:40.364132 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.351933 27792 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.337613 28029 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:40.377254 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.382885 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53107.8us
I20260709 13:13:40.383754 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:40.343071 27912 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.336951 28023 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.388003 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.301599 22908 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8670cf017371429ea31529b6f247493e"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:40.388269 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:40.390440 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.410686 22908 raft_consensus.cc:606] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:40.411168 22908 raft_consensus.cc:614] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:40.390754 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.392066 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.371634 27921 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.412912 22909 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.412583 27995 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.414215 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.415329 27995 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:40.416873 27921 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.417743 27972 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.418370 27686 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.419368 27972 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.420917 27686 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.421429 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.422921 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:40.424145 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:40.393672 28031 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:13:40.394588 28024 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:13:40.395183 27974 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.351951 28012 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.443106 28012 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.396899 28011 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.389292 27751 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:40.444377 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.445606 28003 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.603s	user 0.005s	sys 0.001s
I20260709 13:13:40.388819 27899 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:40.446306 27899 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:40.446491 27921 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:40.446790 27899 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:40.447142 27921 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.448553 28005 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:40.446497 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:40.449805 27993 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:40.379829 28029 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.450312 27993 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.450601 28029 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:40.451292 27993 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.399312 22835 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:40.453949 22837 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:13:40.455282 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.455757 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.456413 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.456492 27811 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:40.457242 22835 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:13:40.385758 28016 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:13:40.457890 28005 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:40.458349 28005 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.400746 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.458815 27999 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.459152 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.401796 27982 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:40.459940 22841 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:40.460238 27982 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.460638 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.461383 27982 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.462531 22913 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:40.462532 27982 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:40.462879 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.404132 22834 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:40.464316 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.465600 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.466920 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.467557 22834 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:40.424873 27792 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:40.470201 27792 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.470639 27792 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.425230 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.459180 28005 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.425582 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.474443 28005 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.474574 22759 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:13:40.475823 28027 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:13:40.476243 27811 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:40.477849 22841 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:40.427448 22907 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:13:40.481065 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:40.428869 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.392473 27912 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.428150 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.434152 27686 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:40.435878 28038 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.466792 28030 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:13:40.436529 22911 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:40.436813 28026 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:40.436941 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.436543 27991 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.438275 27687 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.497874 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:40.505332 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81257.8us
I20260709 13:13:40.542660 22835 raft_consensus.cc:2410] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:40.506387 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 677) took 1773 ms. Trace:
I20260709 13:13:40.543989 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.518206 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:40.440457 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.402519 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.547255 22840 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:40.478423 28022 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.479944 27811 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:40.434507 27953 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:40.518983 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:40.433372 23458 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:40.553877 23463 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:40.491370 22909 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:40.543655 22761 rpcz_store.cc:276] 0709 13:13:38.732669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.245385 (+512716us) service_pool.cc:224] Handling call
0709 13:13:39.262159 (+ 16774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.773356 (+511197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.506050 (+732694us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:40.506306 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174039296}
I20260709 13:13:40.556372 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:40.485869 27806 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.001s
I20260709 13:13:40.530169 22839 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:13:40.515687 22914 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:40.574580 22839 pending_rounds.cc:85] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 6
I20260709 13:13:40.576056 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:40.578631 22839 pending_rounds.cc:107] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:13:40.579109 22839 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:40.560402 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.562875 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.524991 23459 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:40.504770 28023 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.583874 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:40.584863 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:40.584662 28023 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:40.585865 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:40.586285 28023 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:40.586926 28023 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:40.587100 28005 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:40.586853 27982 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:40.587935 28005 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.587208 27942 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:40.489447 27753 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:40.580083 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.486690 27977 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.591638 22759 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:40.501422 28010 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:40.596223 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:40.505137 28011 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:40.604696 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.604821 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.489168 27908 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.554766 27995 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:40.506543 28020 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.489797 27868 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:13:40.594242 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:40.589005 28005 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.485419 27979 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.489653 27873 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:13:40.608072 23465 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:40.602598 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.607404 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.605357 22835 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:40.626276 22761 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:40.591538 27805 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:40.505918 27686 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.594901 27806 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.504638 27700 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.600160 28010 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.607453 27687 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:40.627941 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.590510 28022 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:13:40.557822 27991 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.634424 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.634588 22841 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:13:40.506018 28038 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.673110 28038 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.673931 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.674979 22832 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:13:40.675854 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.507176 28026 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.679167 27807 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.679582 28026 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.683477 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.649922 22837 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.612479 28011 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:40.685452 28010 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.685716 28011 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.686286 22832 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:40.686721 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.687273 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.687407 22837 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.598914 28023 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.690155 28022 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:40.690622 28022 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:40.690865 27699 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.691409 27699 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.691633 28022 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.692961 28022 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.653895 27953 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:40.693588 27699 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:40.662985 27601 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.996s	user 0.015s	sys 0.000s
I20260709 13:13:40.699103 27601 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:40.699712 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.700848 22840 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:40.664245 27991 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.701647 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:40.664623 27806 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.628118 27805 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:40.702943 28001 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:40.703226 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.703361 27805 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.703527 28001 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.662849 27999 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.670446 27686 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:40.636165 27977 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:40.615047 27995 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.617890 28017 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.727722 22834 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:40.686892 28010 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:40.693951 27700 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:40.648887 27687 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.694242 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.654381 22836 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:40.693686 27953 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.696153 27921 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:40.661259 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:40.751118 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89996.7us
I20260709 13:13:40.752727 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:40.748535 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:40.754204 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.668596 22835 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.755033 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.670356 27979 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:40.711328 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 690) took 1330 ms (client timeout 1751 ms). Trace:
I20260709 13:13:40.716249 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:40.719801 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.734828 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.758339 22762 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:40.735262 23459 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:13:40.758250 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.759167 22834 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:13:40.756790 22908 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:40.759790 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.760319 22832 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:40.741977 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:40.761382 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.761533 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.761611 22758 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.742890 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:40.745012 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.735643 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.762897 22764 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:40.756503 22763 rpcz_store.cc:269] 0709 13:13:39.328361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.381663 (+ 53302us) service_pool.cc:224] Handling call
0709 13:13:40.658932 (+1277269us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54002dcdd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:40.659065 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468252544}
I20260709 13:13:40.760533 23466 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:40.765683 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.722715 27942 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.763772 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.762383 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.745066 28023 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.748519 27687 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:40.756883 27389 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.781s	user 0.016s	sys 0.000s
I20260709 13:13:40.770484 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.771252 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.782315 22835 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.771353 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.723482 27982 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:40.777318 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.770324 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.757447 27873 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.779843 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:40.753160 27934 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.810312 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.750519 27953 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:40.794988 22761 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:40.819877 22761 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:40.795228 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.723594 28029 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.763176 27807 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:40.760488 27912 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:40.792388 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.836544 22835 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:40.804320 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.714375 27993 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:13:40.760874 27601 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.809193 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.766824 27921 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:40.815547 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.777256 27977 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:40.848708 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:40.849963 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:40.851053 22840 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.852874 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:40.860445 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.833726 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:40.791594 28001 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:40.844692 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.863179 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.864805 22839 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.866828 23462 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:40.790949 27899 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.842170 27959 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.840942 27908 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.882752 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 127773us
I20260709 13:13:40.826661 27934 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.879256 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.845995 27993 consensus_queue.cc:1243] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:13:40.881322 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.886320 22915 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.846155 28045 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:40.896188 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.816769 27873 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.888064 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.910157 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.885654 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.846383 27921 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:40.911846 27921 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:40.913708 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:40.904520 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.918270 22914 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:40.910754 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:40.926314 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:40.926174 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:40.927237 23466 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:40.879462 28038 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.905827 28040 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:13:40.937639 23464 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:40.849840 27601 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.936226 22908 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:40.945379 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:40.951751 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.905735 27529 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.957103 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.874030 27792 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:40.959827 22837 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:40.965437 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.976560 22835 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.978669 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:40.979341 22759 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.868170 28044 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 13:13:40.967829 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 201950us
I20260709 13:13:40.985111 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:40.985409 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99812.7us
I20260709 13:13:40.928756 27751 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:40.989976 22834 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:40.987336 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.940094 28013 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:40.993793 22835 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:40.934525 28045 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:40.997154 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:40.889212 27959 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.004415 22915 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:41.012498 22761 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:40.905223 27908 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.020450 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:41.022698 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:41.028334 22760 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:41.038462 22835 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:40.990770 27929 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:41.044101 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:41.005263 28013 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.927932 27982 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:40.996551 28052 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:41.061488 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:41.061812 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 228210us
I20260709 13:13:40.947368 28038 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.086390 22840 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:13:41.047801 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:41.088685 22759 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:41.088876 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:40.986860 27529 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.095695 22833 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:13:41.029783 27977 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:41.105106 22835 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:41.102732 22833 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:41.104020 22836 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:13:41.106365 22836 pending_rounds.cc:85] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 7
I20260709 13:13:41.106626 22836 pending_rounds.cc:107] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:13:41.106853 22836 raft_consensus.cc:2889] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:40.939369 27942 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.060942 28017 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.062325 28013 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:41.114034 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.116729 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.117347 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.119571 22913 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:41.121340 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:41.106880 28001 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:41.122119 22915 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:41.123261 22835 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:41.130503 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.087648 27807 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:41.132382 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.137771 22912 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:41.042538 27751 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.133904 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:41.141763 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:41.148844 22835 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:41.150770 22836 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:41.156071 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.174975 22835 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:41.124176 27868 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:41.201901 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:41.207505 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 118719us
W20260709 13:13:41.156626 28042 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.332s	user 0.001s	sys 0.001s
I20260709 13:13:41.145555 27995 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.155827 27686 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:41.155498 27972 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:41.176127 28029 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:41.225260 28049 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
W20260709 13:13:41.241413 28046 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:13:41.241709 28065 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:41.225977 28063 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.247088 28065 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.247570 28063 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:41.244637 28051 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:13:41.243046 28047 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:13:41.142133 27979 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:13:41.244068 27934 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:41.242724 27977 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:41.286170 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.309868 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132968us
I20260709 13:13:41.178033 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.202309 28044 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.320751 22915 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:41.322491 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.328143 23461 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:41.180845 27868 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:41.344694 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.224562 28064 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:41.221689 28059 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:41.349025 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.350057 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:41.353300 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.259354 27686 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.361217 28033 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:41.310362 27995 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:41.363106 27995 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.349788 28068 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:13:41.326706 27972 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:41.368422 27972 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.367322 28065 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:41.369323 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 435885us
I20260709 13:13:41.369776 27686 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:41.170483 28045 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:41.370975 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.371243 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:41.372496 28069 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:41.372527 28065 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:41.373596 28064 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.374734 27986 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:41.375152 27986 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:41.375998 28064 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.369477 28067 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:41.373620 27807 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.391458 27868 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.392795 27868 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:41.349805 28044 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.398976 28033 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.404625 28033 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:41.400607 28044 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:41.391141 28058 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:13:41.389925 27995 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.402609 28063 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.402122 27986 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.403805 28059 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.407255 27529 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:41.399773 28060 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.380968 28067 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:41.360620 27873 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:41.420177 27805 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.005s
W20260709 13:13:41.422988 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 421752us
W20260709 13:13:41.397243 28010 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.003s
W20260709 13:13:41.442047 22766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.003s	sys 0.011s
W20260709 13:13:41.449856 22766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.003s	sys 0.012s
I20260709 13:13:41.420303 28058 consensus_queue.cc:1243] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:13:41.424173 28005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.001s	sys 0.011s
W20260709 13:13:41.451285 22758 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.676s	user 0.019s	sys 0.001s
I20260709 13:13:41.456981 22758 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:41.419423 27792 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.459095 22907 raft_consensus.cc:606] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
W20260709 13:13:41.458850 23465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.028s	sys 0.027s
I20260709 13:13:41.460075 22907 raft_consensus.cc:614] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:41.460356 23465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.028s	sys 0.028s
I20260709 13:13:41.460657 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:41.433197 22764 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.655s	user 0.015s	sys 0.004s
I20260709 13:13:41.449867 27986 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:41.466691 22764 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:41.461349 28005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.001s	sys 0.011s
I20260709 13:13:41.471505 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:41.437088 28052 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:41.466357 27959 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.449997 28060 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.455029 28001 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:41.452303 27805 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:41.489758 27953 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.650s	user 0.004s	sys 0.008s
I20260709 13:13:41.490232 28044 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:41.488224 28010 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:41.493203 27912 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.010s
I20260709 13:13:41.494216 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:41.494170 28066 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:41.513842 27751 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:41.514878 28062 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:13:41.473923 27873 raft_consensus.cc:2764] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:41.472856 27908 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.449060 28059 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.546303 27953 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.513113 27686 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.525839 27929 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:41.526335 22913 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
W20260709 13:13:41.529223 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.559383 28011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.000s	sys 0.019s
I20260709 13:13:41.553685 27529 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:41.580842 22762 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:41.524744 28033 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.490279 28038 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:41.601054 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.550760 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 56628.7us
W20260709 13:13:41.552449 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 381902us
I20260709 13:13:41.562233 28013 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.549765 27912 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:41.556126 27921 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.013s
I20260709 13:13:41.591023 27601 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.584098 28052 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:41.578532 28011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.001s	sys 0.019s
I20260709 13:13:41.601336 28038 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:41.600893 28033 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:41.602643 28059 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.603717 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46252 (request call id 41) took 1086 ms. Trace:
W20260709 13:13:41.524849 27700 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.000s
I20260709 13:13:41.596964 28001 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.611438 27692 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:41.611756 27686 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.614665 27953 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.572341 27929 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:41.608837 27687 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.006s
I20260709 13:13:41.767683 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2100) took 1225 ms. Trace:
W20260709 13:13:41.647042 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 269200us
I20260709 13:13:41.646512 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:41.619990 27529 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:41.672142 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.552807 28067 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:41.673727 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 302586us
I20260709 13:13:41.669627 22907 rpcz_store.cc:276] 0709 13:13:40.517814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.518844 (+  1030us) service_pool.cc:224] Handling call
0709 13:13:41.603504 (+1084660us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400044cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aeb853 00007f68a6ab960a 00007f68a003f7d4 00007f68a003f762 00007f68a003f691 00007f68a003f60c 00007f68a003e8c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:41.603646 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077920128}
I20260709 13:13:41.686972 22760 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:13:41.753051 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.662595 27751 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.689337 28038 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:41.689236 22838 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:13:41.644045 27921 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:41.765441 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.617s (SENT)
W20260709 13:13:41.767345 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.019s	sys 0.019s
I20260709 13:13:41.621551 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:41.788096 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 166709us
I20260709 13:13:41.788944 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:41.780717 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 134289us
I20260709 13:13:41.787250 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:41.795640 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:41.796663 22836 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:41.798108 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:41.799860 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:41.800575 22835 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:13:41.789816 28067 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:41.808106 27529 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:41.768980 28078 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.101s	user 0.000s	sys 0.005s
I20260709 13:13:41.811955 23464 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:41.812359 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:41.787683 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.019s	sys 0.019s
I20260709 13:13:41.813326 23458 raft_consensus.cc:2368] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
W20260709 13:13:41.813400 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 601130us
I20260709 13:13:41.818394 27692 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:41.819325 27692 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:41.820106 28036 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:41.821014 28036 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:41.783689 28022 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.090s	user 0.014s	sys 0.012s
I20260709 13:13:41.787602 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:41.803179 22763 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:41.779688 27687 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.837416 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:41.838204 22766 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:41.839085 22837 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:41.840137 27687 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:41.840516 28038 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.840888 27687 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:41.828729 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:41.842965 27912 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:41.843427 27912 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:41.843822 27912 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:41.844174 22730 leader_election.cc:400] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:41.848811 22730 leader_election.cc:403] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:41.849699 28085 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:41.850185 28013 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:41.824949 28060 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:41.820667 27692 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:41.825088 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:41.831045 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.769117 22833 rpcz_store.cc:276] 0709 13:13:40.541165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.118247 (+577082us) service_pool.cc:224] Handling call
0709 13:13:41.141606 (+ 23359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.141883 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:41.418965 (+277082us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.766507 (+347542us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:41.766577 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:41.766628 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.766722 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.766879 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":438876,"prepare.run_cpu_time_us":249,"prepare.run_wall_time_us":244,"raft.queue_time_us":118,"raft.run_wall_time_us":5240,"thread_start_us":623822,"threads_started":2}
W20260709 13:13:41.860221 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.832007 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44858.1us
I20260709 13:13:41.832379 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:41.787763 27953 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:41.836143 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:41.864087 22765 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:41.836436 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71073.5us
I20260709 13:13:41.857916 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:41.866550 22758 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:41.859143 22836 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:41.769825 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2155) took 1021 ms. Trace:
I20260709 13:13:41.869000 22762 rpcz_store.cc:276] 0709 13:13:40.748157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.761031 (+ 12874us) service_pool.cc:224] Handling call
0709 13:13:41.769597 (+1008566us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b5400221ad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:41.769762 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780295552}
I20260709 13:13:41.785904 28072 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.876075 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:41.824460 27942 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:41.877307 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:41.876217 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:41.886747 23458 raft_consensus.cc:2410] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:41.885743 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:41.886922 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:41.887863 22915 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:41.887167 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:41.888720 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.897965 27953 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:41.899873 27953 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:41.907063 28076 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:13:41.909551 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 939) took 1202 ms. Trace:
I20260709 13:13:41.829483 27805 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:41.918248 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 56249.6us
I20260709 13:13:41.920099 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:41.873869 28023 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.083s	user 0.010s	sys 0.014s
I20260709 13:13:41.918538 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:41.859997 27977 raft_consensus.cc:2764] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:13:41.964093 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.980880 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.990165 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72538.7us
I20260709 13:13:41.992385 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:41.993016 22840 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:41.996708 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:41.910251 23465 rpcz_store.cc:276] 0709 13:13:40.707163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.707846 (+   683us) service_pool.cc:224] Handling call
0709 13:13:40.743965 (+ 36119us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:40.761283 (+ 17318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.761317 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:13:40.806970 (+ 45653us) log.cc:844] Serialized 78 byte log entry
0709 13:13:41.462284 (+655314us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:41.462330 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.462402 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.909220 (+446818us) raft_consensus.cc:1685] finished
0709 13:13:41.909245 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.909453 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7156,"raft.queue_time_us":7365,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8283,"thread_start_us":612708,"threads_started":1,"wal-append.queue_time_us":817941}
I20260709 13:13:41.791323 27751 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.015808 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2092) took 1592 ms. Trace:
I20260709 13:13:42.018909 22839 rpcz_store.cc:276] 0709 13:13:40.405253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.702908 (+297655us) service_pool.cc:224] Handling call
0709 13:13:41.992621 (+1289713us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001ff4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:41.997355 (+  4734us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369749760}
I20260709 13:13:42.020553 28058 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:42.020968 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:42.025144 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:42.026202 22765 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:42.023166 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:42.036608 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:41.980109 27959 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:41.978998 27942 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:42.050352 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:42.068619 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 71985.2us
I20260709 13:13:42.070320 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.073622 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:42.074489 22906 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:42.074388 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:42.097106 22915 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:42.099582 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:42.102695 22765 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:42.011608 28094 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:42.174321 22908 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:42.108421 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58153.3us
I20260709 13:13:42.115595 28079 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.106000 28013 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:42.116544 28010 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:42.117337 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:42.239300 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.122260 28011 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:42.066581 28060 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:42.150031 22906 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:13:42.167243 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.166625 27942 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.021656 28058 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:42.031970 27753 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:42.032316 27805 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:42.260468 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:42.235302 22835 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:42.044065 27982 raft_consensus.cc:2764] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:42.121716 27977 raft_consensus.cc:2764] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:41.913753 28072 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:42.240101 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 372605us
I20260709 13:13:42.240830 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:42.286162 22908 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:13:42.241757 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:42.287207 22909 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:42.248643 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.249622 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.205622 28085 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:42.159494 27751 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.199446 28094 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:42.247687 28022 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.253042 28071 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:42.306259 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 196461us
I20260709 13:13:42.311141 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.255261 27753 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.261252 27805 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.281898 27687 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:42.332741 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.248936 28013 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.238660 28079 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.291368 27959 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:42.329229 27929 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.546s	user 0.007s	sys 0.007s
I20260709 13:13:42.304056 28094 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:42.292281 28054 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:42.349695 28079 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:42.253746 28010 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.345484 22732 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.337267 27753 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:42.344695 27805 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:42.357631 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.363620 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:42.364138 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2078) took 1983 ms (client timeout 1633 ms). Trace:
W20260709 13:13:42.364743 22838 rpcz_store.cc:269] 0709 13:13:40.381000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.381121 (+   121us) service_pool.cc:224] Handling call
0709 13:13:42.363857 (+1982736us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003219d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:42.364048 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2934220544}
I20260709 13:13:42.344367 27687 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:42.365654 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:42.346589 27751 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.319626 28022 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.357609 27792 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:42.371892 27899 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.319082 28071 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.381829 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:42.388103 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.382992 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:42.386379 27977 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.379977 28011 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:13:42.377738 27792 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.366324 27687 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.375880 27807 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.387709 27959 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:42.406080 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.401321 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.397770 27977 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:42.411485 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1815) took 1972 ms. Trace:
I20260709 13:13:42.414654 22840 rpcz_store.cc:276] 0709 13:13:40.439049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.092460 (+653411us) service_pool.cc:224] Handling call
0709 13:13:41.992796 (+900336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:42.009666 (+ 16870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.411127 (+401461us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54001ff4d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:42.411380 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027648896}
I20260709 13:13:42.351151 27929 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.427377 22909 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:42.413528 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.354312 28054 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.400293 27792 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:42.436704 28097 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:13:42.442154 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.437150 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:42.444577 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.445433 22832 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:42.445621 23461 raft_consensus.cc:2410] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:13:42.449000 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:42.367413 28058 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.455839 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.457005 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2154) took 1721 ms. Trace:
I20260709 13:13:42.457437 22763 rpcz_store.cc:276] 0709 13:13:40.736045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.790890 (+ 54845us) service_pool.cc:224] Handling call
0709 13:13:40.791379 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.829768 (+1038389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.456678 (+626910us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b540037fad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:42.456935 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296983936}
I20260709 13:13:42.460485 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:42.461400 22840 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:42.466564 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:42.469612 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2157) took 1666 ms. Trace:
I20260709 13:13:42.470263 22761 rpcz_store.cc:276] 0709 13:13:40.803428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.834536 (+ 31108us) service_pool.cc:224] Handling call
0709 13:13:40.835020 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.013152 (+178132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.469153 (+1456001us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:42.469461 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407670016}
I20260709 13:13:42.475580 28038 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.477191 28071 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.479846 27929 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:42.481492 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:42.482041 28099 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:13:42.450141 27807 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.489044 22758 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:42.490610 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.491053 28011 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:42.491680 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.492538 28036 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.492877 22841 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:42.493628 28054 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.494102 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.517762 22836 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:42.494935 28072 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.493651 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.521790 22833 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:42.496711 28010 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.491099 28022 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.512511 28011 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.523249 28011 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:42.523420 28013 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:42.519799 23430 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 2.028s (SENT) [suppressed 15 similar messages]
I20260709 13:13:42.502347 22759 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:42.507205 22832 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:42.454568 27977 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.527908 27977 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:42.529099 27792 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.532471 27792 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.534772 28094 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.537251 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.538331 22836 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:42.538898 28094 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:42.516446 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.518381 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 954) took 1571 ms. Trace:
I20260709 13:13:42.539327 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:42.539474 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2160) took 1649 ms. Trace:
I20260709 13:13:42.539763 23464 rpcz_store.cc:276] 0709 13:13:40.947030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.948291 (+  1261us) service_pool.cc:224] Handling call
0709 13:13:40.949142 (+   851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.812742 (+863600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.518032 (+705290us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54002c6d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:42.518297 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983503104}
I20260709 13:13:42.535460 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.542327 22832 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:42.536104 28060 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:42.546298 28068 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:42.546953 28068 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.550141 28054 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:42.550813 28054 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:42.551164 22911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2932) took 1670 ms (client timeout 1584 ms). Trace:
W20260709 13:13:42.551812 22911 rpcz_store.cc:269] 0709 13:13:40.881167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.881309 (+   142us) service_pool.cc:224] Handling call
0709 13:13:42.550867 (+1669558us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400210cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:42.551043 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785836416}
I20260709 13:13:42.553993 28054 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.555601 28010 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.556126 28010 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.558429 28010 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:42.549439 28068 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:42.561403 28094 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.561564 28036 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.562162 28094 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.562180 28036 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.565677 28068 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:42.567250 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1814) took 2128 ms. Trace:
I20260709 13:13:42.565397 28036 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.511333 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:42.568377 28068 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:42.569401 28022 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:42.570509 28010 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:42.571470 28010 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:42.572660 28010 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.575915 28010 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:42.576517 27921 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:42.578008 28022 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.578634 28022 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.579233 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:42.581065 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 2930) took 1700 ms (client timeout 1921 ms). Trace:
I20260709 13:13:42.581718 28022 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.582002 28017 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:42.581735 22913 rpcz_store.cc:269] 0709 13:13:40.880403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.880687 (+   284us) service_pool.cc:224] Handling call
0709 13:13:42.580884 (+1700197us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54000cf654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:42.580999 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438047616}
I20260709 13:13:42.583142 28017 consensus_queue.cc:1243] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:13:42.584080 27700 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:42.584556 27700 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:42.533329 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:42.568260 22837 rpcz_store.cc:276] 0709 13:13:40.438823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.061901 (+623078us) service_pool.cc:224] Handling call
0709 13:13:41.106570 (+ 44669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.839494 (+732924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.566569 (+727075us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54000c6cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:42.567170 (+   601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684205312}
W20260709 13:13:42.589632 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.587985 22839 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 11.
I20260709 13:13:42.601370 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:42.605427 22836 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:42.606752 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:42.570467 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 59287.5us
I20260709 13:13:42.615919 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:42.610291 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:42.618316 22913 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:42.611296 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:42.620581 22908 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:42.621891 22915 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:42.540019 22760 rpcz_store.cc:276] 0709 13:13:40.890243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.042038 (+151795us) service_pool.cc:224] Handling call
0709 13:13:41.092047 (+ 50009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.781723 (+689676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.539110 (+757387us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400369554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:42.539393 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367319424}
I20260709 13:13:42.626652 22761 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:42.511981 28023 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.515466 27529 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.520295 27873 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.629210 28017 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:42.638772 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:42.640341 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:42.675282 22834 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:42.671134 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.040s	sys 0.056s
W20260709 13:13:42.676816 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.040s	sys 0.056s
I20260709 13:13:42.678452 23466 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:42.581815 27972 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:42.691030 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:42.565397 28094 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.697084 23461 pending_rounds.cc:85] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 9
I20260709 13:13:42.697713 23461 pending_rounds.cc:107] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 14.10
W20260709 13:13:42.633845 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63961.3us
I20260709 13:13:42.698143 23461 raft_consensus.cc:2889] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:42.701494 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.701795 23464 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:42.687919 27692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.007s	sys 0.017s
I20260709 13:13:42.707888 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:42.671505 27993 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.667268 28038 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:42.693037 28036 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.719493 23466 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:42.585911 27700 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:42.721397 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.730252 28001 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:42.730899 28001 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.704640 27972 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:42.669098 28060 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:13:42.712545 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:42.741008 23464 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:42.724758 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:42.726795 28038 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.708667 27986 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:13:42.673967 27529 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.688090 28033 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:42.736977 22840 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:42.737514 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:42.781983 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:42.695590 28067 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.888s	user 0.012s	sys 0.003s
I20260709 13:13:42.712476 28023 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:13:42.629796 28022 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
W20260709 13:13:42.723644 27912 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.879s	user 0.017s	sys 0.001s
W20260709 13:13:42.720247 27692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.007s	sys 0.017s
I20260709 13:13:42.779259 22760 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:42.816781 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:42.782248 22758 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:42.817799 22759 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:42.735317 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:42.750048 27953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.016s	sys 0.056s
I20260709 13:13:42.784034 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:42.784605 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.743161 22763 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:42.689435 27873 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:42.819336 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 84136.3us
I20260709 13:13:42.825960 22766 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:42.826363 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:42.747062 28038 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:42.774148 22834 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:42.827796 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1837) took 1034 ms. Trace:
I20260709 13:13:42.828631 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:42.824283 27700 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:42.778105 27929 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:42.750459 28079 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.828433 22834 rpcz_store.cc:276] 0709 13:13:41.793281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.793457 (+   176us) service_pool.cc:224] Handling call
0709 13:13:42.638624 (+845167us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cceee 0000000000375939 00007f68a6a401e0 00007f68a6b090c4 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:13:42.827717 (+189093us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48496256}
W20260709 13:13:42.833640 27953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.017s	sys 0.057s
I20260709 13:13:42.740010 28036 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.734289 27993 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.822252 28060 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.861562 28079 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.825995 28001 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:42.836771 28067 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:42.786981 28033 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.924562 23460 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:13:42.924708 22765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.039s	sys 0.011s
W20260709 13:13:42.925539 22765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.039s	sys 0.012s
I20260709 13:13:42.918934 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2174) took 1013 ms. Trace:
I20260709 13:13:42.926376 22761 rpcz_store.cc:276] 0709 13:13:41.906119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.471569 (+565450us) service_pool.cc:224] Handling call
0709 13:13:42.472055 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:42.627239 (+155184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.918567 (+291328us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400369054. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:42.918847 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354405632}
I20260709 13:13:42.859001 28068 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.822156 28054 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:13:42.858474 27692 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:42.936035 22912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.024s	sys 0.055s
I20260709 13:13:42.823565 27912 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:42.951758 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.013s	sys 0.015s
W20260709 13:13:42.953110 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.014s	sys 0.016s
W20260709 13:13:42.947429 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 498490us
W20260709 13:13:42.920665 27805 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.545s	user 0.003s	sys 0.009s
W20260709 13:13:42.948387 22912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.024s	sys 0.055s
W20260709 13:13:42.954896 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 566842us
I20260709 13:13:42.969293 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:42.970481 23465 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:42.973724 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:42.854594 27929 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:42.994503 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:42.997148 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:42.999154 22759 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:43.002816 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 169574us
I20260709 13:13:43.004894 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:43.006399 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:43.007251 22764 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:42.919232 27982 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:13:43.009145 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.015362 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.016963 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.018755 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:42.914235 28060 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:42.953538 27692 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:42.986035 27687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.043s
I20260709 13:13:42.951954 27899 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:43.022533 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:42.913935 27972 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.024026 28079 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:43.061492 28053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.013s	sys 0.018s
I20260709 13:13:43.062022 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:43.118468 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:43.118865 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58405.5us
I20260709 13:13:43.119275 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:42.986871 28054 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:13:43.009774 27942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.010s	sys 0.036s
I20260709 13:13:43.060163 23465 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.002801 27953 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:13:42.990556 27805 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.069715 22759 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
W20260709 13:13:43.080283 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 444795us
I20260709 13:13:43.008055 27977 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:43.012877 28017 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:43.128078 23457 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:43.013688 28068 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:43.108515 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89864.3us
I20260709 13:13:43.157482 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.108096 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:43.066079 28045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.008s	sys 0.023s
I20260709 13:13:43.124336 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.125075 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:43.128914 22764 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:43.161052 22766 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:43.102716 27807 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:13:43.129581 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.130928 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.173457 22759 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:43.122197 28053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.015s	sys 0.018s
I20260709 13:13:43.134107 28005 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:43.134500 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.215271 22759 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:43.136189 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.077488 28036 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:43.154533 28017 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.125099 27977 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:43.112854 27687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.004s	sys 0.043s
I20260709 13:13:43.113935 27692 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:43.159716 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2943) took 1104 ms. Trace:
W20260709 13:13:43.123795 27942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.013s	sys 0.038s
I20260709 13:13:43.132622 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:43.226904 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 448337us
I20260709 13:13:43.210237 22763 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:43.158496 27993 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:43.121711 27899 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.216167 23458 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 12: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:43.136456 28058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.015s	sys 0.019s
W20260709 13:13:43.217831 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60508.2us
I20260709 13:13:43.278029 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:43.170501 28045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.008s	sys 0.023s
I20260709 13:13:43.128545 28079 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:43.228470 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 92404.7us
I20260709 13:13:43.170333 28068 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.812538 28080 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 13:13:43.225713 22912 rpcz_store.cc:276] 0709 13:13:42.055297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.318047 (+262750us) service_pool.cc:224] Handling call
0709 13:13:42.331399 (+ 13352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:42.331862 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:42.331897 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:42.338668 (+  6771us) log.cc:844] Serialized 26 byte log entry
0709 13:13:42.956199 (+617531us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:42.956263 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.956369 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.159479 (+203110us) raft_consensus.cc:1685] finished
0709 13:13:43.159502 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.159649 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":843,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6232,"thread_start_us":608055,"threads_started":1,"wal-append.queue_time_us":773828}
I20260709 13:13:43.217413 27751 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:43.222549 28110 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:43.215399 28005 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:43.170053 28112 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:43.226197 27942 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:43.277158 28058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.015s	sys 0.019s
I20260709 13:13:43.234324 27792 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:43.273176 27529 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:43.297948 28068 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:43.300652 27751 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.320127 28110 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:43.309226 28112 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:43.355778 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 360595us
I20260709 13:13:43.313048 28118 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:43.296620 28079 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:43.368659 28119 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:43.369947 28079 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:43.336359 27807 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:43.370898 28116 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:13:43.372509 28108 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:13:43.336112 27751 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.331418 27942 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:43.381088 23465 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:43.383486 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2168) took 1523 ms. Trace:
I20260709 13:13:43.354660 28033 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:43.406792 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49194.3us
I20260709 13:13:43.363204 28112 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:43.393414 27959 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.981s	user 0.011s	sys 0.000s
I20260709 13:13:43.378346 28110 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:43.396013 28107 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
I20260709 13:13:43.397549 27805 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:43.400547 28011 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.877s	user 0.014s	sys 0.000s
I20260709 13:13:43.401000 27873 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:43.397799 27912 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:43.440515 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 314282us
I20260709 13:13:43.411324 28085 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:43.385337 28022 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:13:43.455569 22835 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:13:43.454654 23464 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:43.456343 22835 raft_consensus.cc:606] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:43.459110 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.431155 27959 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:43.459640 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.460038 23466 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:43.453143 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 307663us
I20260709 13:13:43.463613 23460 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
W20260709 13:13:43.465485 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 187668us
I20260709 13:13:43.466586 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.467142 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.468500 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.404042 28124 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 13:13:43.471174 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 176841us
I20260709 13:13:43.474226 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:43.477105 23463 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:43.396723 28005 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.470405 28054 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:43.479943 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.462423 28058 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.491344 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.456489 28121 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:13:43.493435 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 195153us
I20260709 13:13:43.489485 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.452198 28011 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.498314 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.475091 27929 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:43.482656 28033 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:43.508066 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.487913 27959 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.504463 27792 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:13:43.476564 27529 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:13:43.524039 27529 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:43.493549 28054 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.527339 28054 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:43.510479 27953 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:43.509449 28110 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.531821 27899 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:43.504595 27912 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:43.533937 27912 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:43.535004 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.511162 28005 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.535792 22764 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
W20260709 13:13:43.542172 28125 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:13:43.545853 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2171) took 1665 ms. Trace:
W20260709 13:13:43.547156 28129 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:13:43.546289 22765 rpcz_store.cc:276] 0709 13:13:41.881006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.238120 (+357114us) service_pool.cc:224] Handling call
0709 13:13:42.278661 (+ 40541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:42.926020 (+647359us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:42.926056 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:42.985822 (+ 59766us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.402799 (+416977us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:43.402863 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.402969 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.545564 (+142595us) raft_consensus.cc:1685] finished
0709 13:13:43.545592 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.545777 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22156,"raft.queue_time_us":40632,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5921,"thread_start_us":993035,"threads_started":2,"wal-append.queue_time_us":544692}
I20260709 13:13:43.506085 28130 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:43.549984 28130 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.552343 28130 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:43.553010 28010 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.976s	user 0.013s	sys 0.000s
I20260709 13:13:43.498080 28123 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:13:43.554188 28123 consensus_queue.cc:1243] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:13:43.555958 27753 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.188s	user 0.012s	sys 0.001s
I20260709 13:13:43.556217 28067 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:43.556726 28067 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:43.556535 27753 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.494416 28001 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:43.557825 28001 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:43.557636 28067 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.473412 28128 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:13:43.558907 28011 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:43.559612 28011 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:43.560060 28011 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:43.567483 28122 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:43.567963 27807 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:43.499675 28058 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.572036 28058 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:43.572602 28138 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:43.384306 22757 rpcz_store.cc:276] 0709 13:13:41.860105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.860236 (+   131us) service_pool.cc:224] Handling call
0709 13:13:41.878373 (+ 18137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.878693 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:42.045638 (+166945us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.382752 (+1337114us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.383035 (+   283us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:43.383106 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:43.383168 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.383250 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.383403 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68778,"prepare.queue_time_us":411929,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":17712,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":7789,"thread_start_us":1319424,"threads_started":4,"wal-append.queue_time_us":788725}
I20260709 13:13:43.486210 27942 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.491441 27977 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:43.581328 22766 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
W20260709 13:13:43.512317 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 245831us
W20260709 13:13:43.587481 22761 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.000s	sys 0.021s
W20260709 13:13:43.588965 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81022us
W20260709 13:13:43.589017 22761 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.021s
I20260709 13:13:43.590399 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.560904 28011 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.617180 27959 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:43.627306 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.598778 28053 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:43.599452 28054 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:43.602056 28085 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:43.631500 28085 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:43.602800 28140 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:43.605441 28060 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:43.609352 27805 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:43.568028 28122 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:43.610924 27692 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.595782 27529 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:43.641435 28122 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:43.644409 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 229702us
I20260709 13:13:43.595909 28010 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:43.645762 28010 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:43.646207 28010 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:43.646253 27692 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:43.615566 28068 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:43.648509 28068 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:43.649924 22835 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:43.649825 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.618225 28113 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:13:43.652766 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:43.653209 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.656294 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.658084 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.644579 28122 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:43.628788 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:43.660614 28053 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:43.670944 28053 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:43.686075 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57402us
W20260709 13:13:43.665288 28038 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.808s	user 0.005s	sys 0.007s
I20260709 13:13:43.595993 28072 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:13:43.650650 28113 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:43.703585 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.702392 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.661796 28079 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:43.716101 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 222274us
I20260709 13:13:43.535379 27912 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.737397 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:43.558836 28001 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:43.714042 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.718724 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.739897 22832 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:13:43.740085 22908 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:43.720158 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.741166 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:43.721583 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.742592 22836 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:43.721841 22834 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:13:43.743278 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.724841 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.743716 22834 pending_rounds.cc:85] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 13
I20260709 13:13:43.721876 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.725601 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.744105 22834 pending_rounds.cc:107] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:13:43.744300 22832 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:43.652081 28005 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:13:43.744689 22834 raft_consensus.cc:2889] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:43.744925 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.752048 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:43.746078 22832 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:43.752157 22835 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:13:43.752836 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.753933 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.754763 22837 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:43.695013 28053 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.701682 27792 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.760434 23458 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:43.761703 23430 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.751s (SENT)
I20260709 13:13:43.735903 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.767855 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:43.775955 22841 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15. Re-sending same reply.
I20260709 13:13:43.723842 28079 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.782281 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:43.699601 28060 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:13:43.747061 28080 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:43.808995 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.820417 22837 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:43.817759 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.821338 22835 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:43.820648 22908 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:43.780815 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.824293 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:43.824749 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 333475us
I20260709 13:13:43.826412 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:43.826468 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.744508 27959 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:13:43.725189 28038 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.818607 28144 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:13:43.839179 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.840572 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:43.841825 28023 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.842170 22757 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:43.843570 22760 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:43.843966 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69635.4us
I20260709 13:13:43.863548 22762 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:13:43.801344 28141 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:43.809310 28079 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:43.869556 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:43.871553 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:43.876480 22837 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.871680 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:43.820338 28080 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:43.883855 28080 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.887871 28080 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.888756 22762 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:43.890918 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.892601 28058 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:43.893134 28058 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:43.893676 28058 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.902980 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.905750 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.908002 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:43.781636 27792 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:43.911259 22910 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:43.918193 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 418151us
I20260709 13:13:43.932332 22838 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:43.933230 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.953794 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.948213 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:43.954759 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46113.1us
I20260709 13:13:43.956485 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:43.953127 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.958652 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.959604 22759 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:43.954293 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.960497 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.961455 22763 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:43.955358 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:43.962836 22758 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:43.957072 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:43.972209 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:43.956981 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:43.813982 28143 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260709 13:13:44.009022 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:43.914829 28118 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:43.941906 27753 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:43.830163 28113 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.017624 22837 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:43.912765 27959 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:13:43.875648 28023 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.062671 27929 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:44.063822 28094 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:44.064255 28094 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:44.064520 28023 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:43.966519 28036 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:13:44.065142 28094 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.066676 28094 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:43.974480 28141 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:44.043793 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:44.044879 28011 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:44.046527 28151 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 13:13:44.048018 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 226361us
I20260709 13:13:44.025244 28072 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:44.051079 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:44.051471 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.060281 28067 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.062288 27959 consensus_queue.cc:1243] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:44.080623 22758 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:44.016949 28149 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.057878 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:43.974418 28033 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:13:43.974287 27807 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:13:44.092276 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.113085 22910 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:44.118153 22760 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:44.042418 27529 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:44.134543 22912 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:44.160175 22759 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:44.151647 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.164127 22762 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:44.165362 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.165730 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.166826 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.167975 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.168704 22837 raft_consensus.cc:2376] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14. Re-sending same reply.
W20260709 13:13:44.170154 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 213166us
I20260709 13:13:44.175925 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:44.128140 28150 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:44.178144 28141 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.178821 22912 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:44.113536 28072 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:44.138058 28148 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:13:44.179325 28022 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:13:44.199996 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.202379 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:44.205101 27751 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.803s	user 0.011s	sys 0.000s
W20260709 13:13:44.207053 28112 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.004s
I20260709 13:13:44.210335 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:44.213161 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.069888 28085 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.213953 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.179388 27792 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:44.112126 28161 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:44.219256 28138 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.646s	user 0.006s	sys 0.005s
I20260709 13:13:44.138974 28149 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.222298 28038 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:44.222668 28162 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:13:44.224573 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.254410 28005 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.232224 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.284788 22838 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:44.235666 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 278759us
I20260709 13:13:44.250308 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.286499 22762 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:44.287493 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:44.293099 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.294261 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.255678 22910 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:44.233146 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.257141 22915 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:13:44.297477 22915 pending_rounds.cc:85] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 12
I20260709 13:13:44.297784 22915 pending_rounds.cc:107] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:13:44.298032 22915 raft_consensus.cc:2889] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:44.264899 22837 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:44.266916 22833 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:44.263077 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 703) took 1270 ms. Trace:
I20260709 13:13:44.299059 22761 rpcz_store.cc:276] 0709 13:13:42.992400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.992602 (+   202us) service_pool.cc:224] Handling call
0709 13:13:42.993072 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:42.993385 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:43.590629 (+597244us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.000654 (+410025us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.262710 (+262056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.262772 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:44.262825 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.262897 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.263013 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43360,"prepare.queue_time_us":500267,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":2608,"raft.queue_time_us":42702,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":1141233,"threads_started":3,"wal-append.queue_time_us":449773}
I20260709 13:13:44.300674 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:44.299116 22839 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:44.269217 22757 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 13. Ops: []
W20260709 13:13:44.271502 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61277.5us
I20260709 13:13:44.303052 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:44.304705 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:44.272667 23430 leader_election.cc:336] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.670s (SENT)
I20260709 13:13:44.305934 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.234836 27972 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.234318 27959 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:44.308873 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.280202 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:44.231962 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.262164 28072 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:44.306459 23430 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.959s (SENT)
I20260709 13:13:44.311298 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:44.268187 28010 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.000s
I20260709 13:13:44.269249 28152 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:13:44.303190 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:44.306857 22761 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:44.296422 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:44.275844 28118 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.314229 22834 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:44.266215 28144 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:44.274231 28161 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.321571 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:44.316198 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:44.273859 28011 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.263183 27953 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:44.281840 27792 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.315826 27982 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:44.278355 28138 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:44.323617 28152 consensus_queue.cc:1243] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:44.308697 27899 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:44.308979 28023 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:13:44.324038 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.327292 28118 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.267122 27751 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.338274 27977 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:44.332299 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.333573 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:44.320509 28140 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:13:44.345582 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.319861 27873 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:13:44.274310 28080 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:44.323323 28038 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.330194 27700 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:44.676776 27700 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.267367 28112 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.678323 22766 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:44.673111 28080 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.261984 28150 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.679708 28112 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:44.679713 28080 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:44.680333 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.314673 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.682410 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.682542 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.683768 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.683866 28150 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:44.679693 22732 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items
I20260709 13:13:44.333873 28054 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:44.356218 27986 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:44.684792 28054 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.684620 28038 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.332041 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.686492 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.687122 28038 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:44.371331 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64928.4us
I20260709 13:13:44.687616 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.688679 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.686995 22732 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2789)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2781)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2790)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2780)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2799)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2796)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2786)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2209)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2211)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2794)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2212)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2806)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2214)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2216)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2822)
I20260709 13:13:44.690069 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:44.690507 23430 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 2.103s (SENT)
I20260709 13:13:44.690788 22838 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:13:44.333869 28161 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:44.373221 22833 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:44.319219 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.696125 22761 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:44.349671 27792 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.380321 28154 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:44.722612 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.381045 28118 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:44.381119 28153 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:13:44.382201 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.701668 28122 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.984s	user 0.007s	sys 0.008s
W20260709 13:13:44.702834 28094 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.636s	user 0.016s	sys 0.004s
I20260709 13:13:44.387291 28079 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:44.390172 28164 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:13:44.390940 28167 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:13:44.398434 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.755470 22840 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:44.755817 22757 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:44.318914 27959 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.622577 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:44.586014 28138 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.594053 28149 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:44.393326 28113 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:44.353205 27982 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:44.508535 28157 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
I20260709 13:13:44.346853 27953 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:44.536427 22658 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 135) took 35131 ms (client timeout 15000 ms). Trace:
W20260709 13:13:44.492516 28058 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.001s
I20260709 13:13:44.533396 22906 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:44.520313 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.778498 22839 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:13:44.313364 27972 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.781955 27972 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:44.530404 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.720s (SENT)
I20260709 13:13:44.487365 28085 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:44.782788 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:44.783835 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 253460us
W20260709 13:13:44.784489 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.734s (SENT)
I20260709 13:13:44.785004 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:44.522426 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.785861 22760 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:44.787155 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.787734 22760 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:44.789641 27805 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.356879 27942 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:44.793984 27942 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:44.794935 27942 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.505189 28110 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:44.797739 28110 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:44.798691 28110 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.800028 28110 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.537441 22661 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.802578 22661 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:44.804224 22661 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 9 to 12, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 12 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.359218 27899 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:44.530076 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.806157 22758 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:44.806496 27899 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.807243 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.807816 27899 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:44.507263 28053 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.003s
I20260709 13:13:44.343130 28011 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.808715 22914 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:44.324399 28010 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.538626 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:44.809718 22765 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:44.503041 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.810451 22908 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:44.448203 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.656636 22835 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:44.811931 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.812050 28011 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:44.496414 22805 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:44.531087 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.399927 22913 pending_rounds.cc:85] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:13:44.813952 22913 pending_rounds.cc:107] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 11.7
W20260709 13:13:44.813871 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 317985us
I20260709 13:13:44.814291 22913 raft_consensus.cc:2889] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:44.498934 27751 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:44.814723 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:44.492825 28130 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:44.816488 28038 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:44.816987 28130 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:44.598402 27993 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:44.817736 28130 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:44.818269 28130 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:44.817797 27993 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.498423 22763 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:44.819206 28130 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.820600 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.334048 28144 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.684176 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.825376 22836 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:44.359390 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.685261 28054 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:44.817273 28038 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:13:44.703367 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.032s	sys 0.101s
W20260709 13:13:44.834699 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50252.2us
W20260709 13:13:44.834968 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.033s	sys 0.101s
W20260709 13:13:44.835594 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 863429us
I20260709 13:13:44.692175 22732 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2789)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2781)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2824)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2823)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2790)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2812)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2780)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2813)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2799)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2796)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2786)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2209)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2211)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2794)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2212)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2806)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2214)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2216)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2822)
W20260709 13:13:44.836212 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.385102 22911 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:44.385783 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:44.758108 22841 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:13:44.758167 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 455628us
I20260709 13:13:44.763358 22759 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:13:44.719851 28154 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:44.744154 28153 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:44.784122 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2963) took 1130 ms. Trace:
I20260709 13:13:44.807969 22758 raft_consensus.cc:2368] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
I20260709 13:13:44.810547 22834 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:44.844741 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:44.463439 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.845767 22764 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:13:44.846035 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:44.846890 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.847465 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:44.848002 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.848960 22764 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:13:44.677359 27700 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:44.840660 28155 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:44.850766 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:44.851442 22841 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:44.851964 28185 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:44.300520 28005 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.860543 28154 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:44.862463 28154 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.863356 28005 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:44.868607 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 959) took 1404 ms. Trace:
I20260709 13:13:44.869112 23462 rpcz_store.cc:276] 0709 13:13:43.464641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.464791 (+   150us) service_pool.cc:224] Handling call
0709 13:13:43.465246 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.473886 (+  8640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.473935 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.487664 (+ 13729us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.873966 (+386302us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:43.874007 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.874094 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.868341 (+994247us) raft_consensus.cc:1685] finished
0709 13:13:44.868369 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.868536 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11379,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5532,"thread_start_us":364821,"threads_started":1,"wal-append.queue_time_us":603073}
I20260709 13:13:44.877324 27792 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:44.825644 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:44.829566 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.379036 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.827006 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:44.719252 22766 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:44.881940 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.765632 27982 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:44.841244 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 455571us
I20260709 13:13:44.842001 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2962) took 1188 ms. Trace:
I20260709 13:13:44.837925 22762 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:44.892280 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.889889 22907 rpcz_store.cc:276] 0709 13:13:43.653123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.664862 (+ 11739us) service_pool.cc:224] Handling call
0709 13:13:43.665268 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.690780 (+ 25512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.690833 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.703148 (+ 12315us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.081384 (+378236us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:44.081434 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.081529 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.841769 (+760240us) raft_consensus.cc:1685] finished
0709 13:13:44.841795 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.841939 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1936,"raft.queue_time_us":20326,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":15358,"thread_start_us":309175,"threads_started":1,"wal-append.queue_time_us":438509}
I20260709 13:13:44.844183 22909 rpcz_store.cc:276] 0709 13:13:43.653433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.664222 (+ 10789us) service_pool.cc:224] Handling call
0709 13:13:43.664681 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.664990 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:43.695241 (+ 30251us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.511792 (+816551us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b18000937c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:13:44.777179 (+265387us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.780324 (+  3145us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.782167 (+  1843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.782770 (+   603us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:44.782960 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.783185 (+   225us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.783986 (+   801us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":105201,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":1392,"raft.queue_time_us":1972,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":17240,"spinlock_wait_cycles":49795328,"thread_start_us":658776,"threads_started":2,"wal-append.queue_time_us":583106}
I20260709 13:13:44.854179 22761 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.820601 28130 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:44.854595 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.762465 28138 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.852994 22841 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:44.852418 28185 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:44.859232 28153 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.844518 27986 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:44.883484 22836 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:44.924290 22832 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:44.885138 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:44.834648 28038 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.826434 28058 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:44.893041 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 66138.9us
I20260709 13:13:44.882283 28174 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:44.890367 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:44.895761 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.854858 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.908416 22758 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:44.928042 28094 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:13:44.909667 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:44.916762 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.941941 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2957) took 1290 ms. Trace:
I20260709 13:13:44.908604 27912 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:44.930721 23430 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 2.028s (SENT)
I20260709 13:13:44.930891 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2964) took 1066 ms. Trace:
I20260709 13:13:44.894776 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:44.837814 28122 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 13:13:44.935396 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.916086 28169 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:13:44.918756 28072 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.593s	user 0.014s	sys 0.000s
W20260709 13:13:44.855404 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.962841 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:44.937696 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47449.5us
I20260709 13:13:44.936509 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:44.964705 22909 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:44.908457 28155 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.910168 22661 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 12, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 12 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.917562 22835 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:44.973143 22764 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:44.973536 22839 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:44.952301 22760 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:44.975106 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.975709 22760 raft_consensus.cc:2376] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13. Re-sending same reply.
I20260709 13:13:44.954252 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.942482 22906 rpcz_store.cc:276] 0709 13:13:43.651519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.654067 (+  2548us) service_pool.cc:224] Handling call
0709 13:13:43.654443 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.775864 (+1121421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.941635 (+165771us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400041854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:44.941865 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2020631424}
I20260709 13:13:44.955121 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.978590 22758 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:44.979314 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:44.957228 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:44.958676 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.924369 28185 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.984164 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:44.934391 28038 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:44.957278 22908 rpcz_store.cc:276] 0709 13:13:43.864936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.865082 (+   146us) service_pool.cc:224] Handling call
0709 13:13:43.865388 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.810906 (+945518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.930642 (+119736us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:44.930831 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173039360}
I20260709 13:13:44.968148 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:44.982766 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52078us
I20260709 13:13:44.997556 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:44.985073 22834 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:44.999370 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.959250 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.989357 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.001329 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.007045 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.010635 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.015726 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:45.036252 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.048817 22832 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:44.921645 28036 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:13:44.958863 28010 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.062673 28138 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:45.062923 28190 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:13:44.984752 28094 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:45.064378 28187 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:13:45.065814 28200 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:13:45.065645 28193 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:45.068343 22833 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:45.035292 27912 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.035326 22661 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 9, leader changed from <none> to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:45.043510 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.072108 22836 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:45.038030 28122 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:45.047353 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:44.958976 28153 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.079489 22766 raft_consensus.cc:2376] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13. Re-sending same reply.
I20260709 13:13:44.942458 28140 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.016062 28011 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:45.055801 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.057706 28079 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:45.059831 28080 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:44.990118 28155 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.061179 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:45.041361 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:45.071146 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:45.093099 22833 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:13:45.095877 22841 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:45.069365 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:45.096932 22837 raft_consensus.cc:2376] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14. Re-sending same reply.
I20260709 13:13:45.088699 22832 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:44.989832 28072 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.068825 27700 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.076438 22732 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2814)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2815)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2816)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2817)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2818)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2825)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2819)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2820)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2828)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 720)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2833)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2799)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2831)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2830)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2796)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2786)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2222)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2829)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2832)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2782)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2204)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2205)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2209)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2210)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2211)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2794)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2212)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2806)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2214)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2215)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2216)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2217)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2218)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 722)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 723)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 724)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 725)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2220)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 726)
I20260709 13:13:45.082548 22661 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.106415 22761 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:45.107196 22661 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:13:45.085398 28067 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.954s	user 0.011s	sys 0.004s
I20260709 13:13:45.075737 27986 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:45.102495 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.114854 22836 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:45.105137 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:44.995450 28127 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:45.108644 22661 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 12, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 12 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:45.095335 28011 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.076339 27959 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:45.113977 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 103413us
I20260709 13:13:45.078105 28122 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:45.152983 23458 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:45.107241 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:13:45.138064 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.124171 28011 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:45.154600 28196 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:13:45.153364 22833 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:45.153458 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:45.160065 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.161443 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:45.162052 22758 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:45.164083 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 696) took 1360 ms. Trace:
I20260709 13:13:45.164685 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.165797 22758 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:13:45.164498 22911 rpcz_store.cc:276] 0709 13:13:43.803799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.808530 (+  4731us) service_pool.cc:224] Handling call
0709 13:13:43.808952 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.839008 (+1030056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.163840 (+324832us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54000ea954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:45.164019 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324572544}
W20260709 13:13:45.172549 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72833.2us
I20260709 13:13:45.153102 22661 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 12 to 13, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 13 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.173738 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.175518 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.184758 22757 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:45.177222 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:45.189014 23461 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:45.189896 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.124579 28174 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.198318 23463 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.193840 23465 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:45.198675 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.195407 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:45.151386 28079 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.201052 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.202646 22757 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:45.197746 28138 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.162536 27953 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:45.215335 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.171898 28127 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:45.189364 28140 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:45.189169 27873 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:45.223955 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.225421 28058 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.188387 27986 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:45.228556 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.241906 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.230257 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 692) took 1506 ms. Trace:
W20260709 13:13:45.231983 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.245105 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.236877 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:45.214995 28192 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:13:45.246774 22758 raft_consensus.cc:2368] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
W20260709 13:13:45.248323 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 285063us
I20260709 13:13:45.212044 28138 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.243305 22914 rpcz_store.cc:276] 0709 13:13:43.724335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.725333 (+   998us) service_pool.cc:224] Handling call
0709 13:13:45.229995 (+1504662us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540020bcd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.230173 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447111552}
I20260709 13:13:45.256054 22661 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:45.226866 28144 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:45.266415 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.239903 27873 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.249205 27953 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.242802 27986 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.276883 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.273017 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2781) took 1065 ms. Trace:
I20260709 13:13:45.212761 28079 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.281957 22764 rpcz_store.cc:276] 0709 13:13:44.207911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.854593 (+646682us) service_pool.cc:224] Handling call
0709 13:13:45.272812 (+418219us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b5400312254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.272949 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77136640}
I20260709 13:13:45.278198 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.279524 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.286597 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.287002 22757 raft_consensus.cc:2376] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14. Re-sending same reply.
I20260709 13:13:45.258162 28138 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.231444 28010 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:13:45.259845 28174 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:45.288041 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.273073 28140 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:45.318444 28140 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:45.318912 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.319721 28140 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.320400 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.322971 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.324501 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.324901 28038 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.289132 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.297171 22764 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:45.285430 27986 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:45.237139 28122 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.291203 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.276302 27753 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:45.303320 28203 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:13:45.305009 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1855) took 1025 ms. Trace:
I20260709 13:13:45.305197 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.308743 27993 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:45.309444 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2805) took 1037 ms. Trace:
I20260709 13:13:45.311467 28079 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:45.292855 27953 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.278825 27873 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:45.313556 28150 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:45.302520 28206 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:13:45.296866 28174 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:45.325932 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2120) took 1604 ms (client timeout 2108 ms). Trace:
W20260709 13:13:45.314064 28204 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:13:45.326588 28207 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:45.328958 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:45.331763 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2779) took 1162 ms. Trace:
I20260709 13:13:45.346416 28131 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:13:45.384871 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 630) took 1082 ms. Trace:
I20260709 13:13:45.422653 23458 rpcz_store.cc:276] 0709 13:13:44.302663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.302801 (+   138us) service_pool.cc:224] Handling call
0709 13:13:45.384595 (+1081794us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400740254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.384800 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952282496}
I20260709 13:13:45.415894 27982 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:45.351397 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.344519 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:45.424474 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.425007 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.424803 22908 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
W20260709 13:13:45.308759 28205 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:13:45.417580 28174 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:45.389335 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:45.426707 22840 rpcz_store.cc:269] 0709 13:13:43.721526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.721659 (+   133us) service_pool.cc:224] Handling call
0709 13:13:45.325773 (+1604114us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001bc954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.325872 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2325270912}
I20260709 13:13:45.467422 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.467758 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.276082 27529 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.342555 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.436092 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.473757 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.346765 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.475493 22760 raft_consensus.cc:2410] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:13:45.476493 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.473838 22764 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.336483 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:45.477522 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 141096us
W20260709 13:13:45.333022 28161 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.002s
I20260709 13:13:45.487716 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.342057 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.489012 22836 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:45.490411 22836 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:45.345754 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.491516 22758 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.399350 22661 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:13:45.328397 27805 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 13:13:45.420886 22838 rpcz_store.cc:276] 0709 13:13:44.279401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.692310 (+412909us) service_pool.cc:224] Handling call
0709 13:13:44.692640 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.695338 (+  2698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.695370 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.751286 (+ 55916us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.110997 (+359711us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:45.111064 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.111170 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.304767 (+193597us) raft_consensus.cc:1685] finished
0709 13:13:45.304798 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.304949 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2359,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":39773,"thread_start_us":335715,"threads_started":1,"wal-append.queue_time_us":461964}
W20260709 13:13:45.423460 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 118382us
I20260709 13:13:45.342643 27700 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:45.311481 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.343237 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2198) took 1259 ms. Trace:
W20260709 13:13:45.332957 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.436332 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.436996 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.415777 22765 rpcz_store.cc:276] 0709 13:13:44.169611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.181729 (+ 12118us) service_pool.cc:224] Handling call
0709 13:13:44.339528 (+157799us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:13:44.339707 (+   179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.810137 (+470430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.331511 (+521374us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400393354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:45.331689 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154478720}
W20260709 13:13:45.500538 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.500193 22759 rpcz_store.cc:276] 0709 13:13:44.083753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.161330 (+ 77577us) service_pool.cc:224] Handling call
0709 13:13:44.162457 (+  1127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.837450 (+674993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.342854 (+505404us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:45.343156 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1381099392}
I20260709 13:13:45.427572 22766 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:45.501526 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2799) took 1272 ms. Trace:
I20260709 13:13:45.501868 22766 rpcz_store.cc:276] 0709 13:13:44.229296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.319935 (+1090639us) service_pool.cc:224] Handling call
0709 13:13:45.501438 (+181503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.464699 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.502986 22908 raft_consensus.cc:2368] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
I20260709 13:13:45.504099 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.504477 27805 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:45.504918 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2796) took 1276 ms. Trace:
I20260709 13:13:45.505268 22765 rpcz_store.cc:276] 0709 13:13:44.228019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.503804 (+1275785us) service_pool.cc:224] Handling call
0709 13:13:45.504863 (+  1059us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.506182 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:45.465636 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 305618us
W20260709 13:13:44.822913 22658 rpcz_store.cc:269] 0709 13:13:09.419847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.487623 (+ 67776us) service_pool.cc:224] Handling call
0709 13:13:10.121961 (+634338us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5b090 00007f68a1af9f3a 00007f68a78b01cc 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc 00007f68a5e6e392 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:44.536148 (+34414187us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:35.197051 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:35.197808 (+   757us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:35.197865 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:13:35.197901 (+    36us) tablet.cc:662] Decoding operations
0709 13:13:35.199140 (+  1239us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:35.199261 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:35.199292 (+    31us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:13:35.446097 (+246805us) tablet.cc:701] Row locks acquired
0709 13:13:35.446161 (+    64us) write_op.cc:260] PREPARE: finished
0709 13:13:35.446418 (+   257us) write_op.cc:270] Start()
0709 13:13:35.446620 (+   202us) write_op.cc:276] Timestamp: P: 1783602815446338 usec, L: 0
0709 13:13:35.446672 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:13:35.447607 (+   935us) log.cc:844] Serialized 4362 byte log entry
0709 13:13:35.804031 (+356424us) op_driver.cc:464] REPLICATION: finished
0709 13:13:36.415222 (+611191us) write_op.cc:301] APPLY: starting
0709 13:13:36.415328 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:36.753962 (+338634us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:36.834192 (+ 80230us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:37.202176 (+367984us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:37.202559 (+   383us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:13:37.202633 (+    74us) write_op.cc:312] APPLY: finished
0709 13:13:37.203135 (+   502us) log.cc:844] Serialized 105 byte log entry
0709 13:13:38.069960 (+866825us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b
0709 13:13:38.333455 (+263495us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:38.352013 (+ 18558us) write_op.cc:454] Released schema lock
0709 13:13:38.352158 (+   145us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:38.352991 (+   833us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:38.353038 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2506632,"reported_tablets":28,"spinlock_wait_cycles":18977536,"child_traces":[["op",{"apply.queue_time_us":608554,"apply.run_cpu_time_us":51707,"apply.run_wall_time_us":1937632,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50129,"num_ops":11,"prepare.queue_time_us":140457,"prepare.run_cpu_time_us":8569,"prepare.run_wall_time_us":280357,"raft.queue_time_us":158259,"raft.run_cpu_time_us":105661,"raft.run_wall_time_us":719890,"replication_time_us":326558,"spinlock_wait_cycles":61308288,"thread_start_us":1971022,"threads_started":5,"wal-append.queue_time_us":1063066,"wal-append.run_cpu_time_us":14560,"wal-append.run_wall_time_us":2225665}]]}
I20260709 13:13:45.507570 23430 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.343250 28155 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.512071 28207 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:45.514341 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:45.514729 28130 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.001s
I20260709 13:13:45.515259 27953 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.515769 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.477649 27529 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.516595 27529 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.516906 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.468226 28154 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.509327 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.519726 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.519099 22661 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 3 to 13, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 13 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.521590 28161 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.523929 28161 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:45.418061 22761 rpcz_store.cc:276] 0709 13:13:44.271869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.722226 (+450357us) service_pool.cc:224] Handling call
0709 13:13:45.309271 (+587045us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54000816d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.309383 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299573632}
I20260709 13:13:45.525976 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.526010 22915 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:45.487844 28122 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:45.527316 28161 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:45.527750 23457 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:13:45.528061 28161 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:45.528652 28161 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:45.528980 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 635) took 1173 ms. Trace:
I20260709 13:13:45.529327 23457 rpcz_store.cc:276] 0709 13:13:44.355991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.356206 (+   215us) service_pool.cc:224] Handling call
0709 13:13:44.356588 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.528113 (+1171525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.528700 (+   587us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54005140d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:45.528907 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2692761856}
I20260709 13:13:45.529556 28161 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.429246 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:45.463060 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.531816 28038 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.358666 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:45.355568 28054 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.000s
I20260709 13:13:45.382951 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.532625 28038 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:45.478621 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:45.380237 28189 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.678s	user 0.004s	sys 0.000s
I20260709 13:13:45.487129 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:45.493619 22832 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:13:45.380899 28080 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:45.540851 22909 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.557364 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.559316 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:45.560081 22758 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:45.561378 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82885.9us
I20260709 13:13:45.535524 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2973) took 1139 ms. Trace:
I20260709 13:13:45.533347 22757 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:45.538956 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:45.532663 28131 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:45.526449 22766 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:45.562525 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:45.542181 23465 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:13:45.563212 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2786) took 1353 ms. Trace:
I20260709 13:13:45.527376 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.563977 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.564482 22761 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:45.562493 22913 rpcz_store.cc:276] 0709 13:13:44.396359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.399192 (+  2833us) service_pool.cc:224] Handling call
0709 13:13:44.399615 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:44.815002 (+415387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.815038 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:44.815638 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 13:13:45.082823 (+267185us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:45.093841 (+ 11018us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.094479 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.094555 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.535286 (+440731us) raft_consensus.cc:1685] finished
0709 13:13:45.535312 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.535461 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10431,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":195,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":294,"thread_start_us":266779,"threads_started":1,"wal-append.queue_time_us":362023}
I20260709 13:13:45.566236 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.567818 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.532748 22839 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:45.357344 28072 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:45.561831 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.604660 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
W20260709 13:13:45.604872 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65996.2us
W20260709 13:13:45.488294 27899 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.005s
I20260709 13:13:45.563771 22766 rpcz_store.cc:276] 0709 13:13:44.209435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.516654 (+1307219us) service_pool.cc:224] Handling call
0709 13:13:45.563142 (+ 46488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.550978 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.521198 22661 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 10 to 11, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 11 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:45.572446 27912 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:45.343361 28127 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.573535 28131 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:45.575593 27959 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:45.579628 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.580901 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:45.537009 28185 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.501s	user 0.003s	sys 0.007s
I20260709 13:13:45.582783 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:45.536473 28110 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.002s
I20260709 13:13:45.585050 27751 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.533298 28038 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:45.561890 28054 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.499159 27792 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.464097 27753 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.590751 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 2971) took 1218 ms. Trace:
I20260709 13:13:45.662438 22762 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:45.662672 22910 rpcz_store.cc:276] 0709 13:13:44.372038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.372247 (+   209us) service_pool.cc:224] Handling call
0709 13:13:44.372702 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.383300 (+ 10598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.383338 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.383896 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.598108 (+214212us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:44.598197 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.617681 (+ 19484us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.379515 (+761834us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:45.590407 (+210892us) raft_consensus.cc:1685] finished
0709 13:13:45.590433 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.590686 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11369,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":90862336,"thread_start_us":213675,"threads_started":1,"wal-append.queue_time_us":270338}
I20260709 13:13:45.592135 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.613744 28122 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:45.577453 27982 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.592883 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.593744 28208 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.598528 27972 raft_consensus.cc:993] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:45.512986 28011 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.602891 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.603818 22761 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:45.606027 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.607549 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:45.668879 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61423.1us
I20260709 13:13:45.611002 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.613730 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2800) took 1384 ms. Trace:
I20260709 13:13:45.670823 22759 rpcz_store.cc:276] 0709 13:13:44.229636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.550582 (+1320946us) service_pool.cc:224] Handling call
0709 13:13:45.613653 (+ 63071us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:45.550202 28113 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.002s
I20260709 13:13:45.613554 27953 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:13:45.617568 28072 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:45.618378 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.615912 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:45.614298 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.675760 22766 raft_consensus.cc:2410] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:13:45.675817 23465 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:45.614997 27912 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:45.625908 27899 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:45.584384 27805 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.599500 28005 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:45.513000 28207 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 13:13:45.635169 22913 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:45.636690 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:45.640863 22835 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 9.
I20260709 13:13:45.640396 22661 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 10, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 10 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:45.667752 28122 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:45.688848 22661 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 10 to 15, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.691051 22661 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 9 to 11, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 11 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:45.641599 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.003s	sys 0.039s
W20260709 13:13:45.693142 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.003s	sys 0.039s
I20260709 13:13:45.644105 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.644441 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.611078 28154 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:45.646216 22764 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:45.676764 27972 raft_consensus.cc:1081] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:45.676275 27959 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:45.671473 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2815) took 1155 ms. Trace:
I20260709 13:13:45.714563 22762 rpcz_store.cc:276] 0709 13:13:44.516462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.104886 (+588424us) service_pool.cc:224] Handling call
0709 13:13:45.671286 (+566400us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540035dc54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.671417 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255726336}
I20260709 13:13:45.716015 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.715571 22661 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.716828 22762 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.672094 22910 raft_consensus.cc:1217] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[12.7-14.8]   Dedup: 12.7->[14.8-14.8]
W20260709 13:13:45.668928 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66137.9us
W20260709 13:13:45.673794 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67851.1us
I20260709 13:13:45.613384 28058 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:13:45.723798 22910 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:13:45.672387 28110 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.665045 27792 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.682587 22765 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:45.683238 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2782) took 1475 ms. Trace:
I20260709 13:13:45.743657 22766 rpcz_store.cc:276] 0709 13:13:44.208196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.618105 (+1409909us) service_pool.cc:224] Handling call
0709 13:13:45.683159 (+ 65054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.744582 22761 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:45.669207 27986 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.743436 22661 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:13:45.700018 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63454.4us
I20260709 13:13:45.748870 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:45.700614 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2208) took 1494 ms. Trace:
I20260709 13:13:45.749555 22764 rpcz_store.cc:276] 0709 13:13:44.206145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.699797 (+1493652us) service_pool.cc:224] Handling call
0709 13:13:45.700104 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.700298 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.700323 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:45.700367 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:45.700398 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.700450 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.700551 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.750800 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.751333 22764 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.752537 22913 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
W20260709 13:13:45.659246 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.754603 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.639523 28185 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.639010 27751 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.755050 22913 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:45.755450 27751 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.756713 28185 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:45.674736 27529 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.759529 27529 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:45.690027 27805 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.677008 22760 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:45.761791 22907 raft_consensus.cc:2368] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
I20260709 13:13:45.762112 27805 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.762867 28113 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:45.763289 28113 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:45.763631 22912 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:45.724752 27899 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.764183 28113 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.764762 27899 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:45.765302 27899 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.766350 28011 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.672922 28208 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.769871 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:45.770306 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:45.720897 28005 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:45.771302 28208 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:45.771770 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.771929 28208 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:45.772379 28208 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:45.772739 28208 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:45.742609 28072 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:45.773865 28193 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.774137 28150 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:45.684415 28131 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.774438 28193 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.775707 28131 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:45.776672 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:45.777451 27986 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:45.778175 27986 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:45.778646 27986 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:45.703263 28122 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:45.746414 27792 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.720224 28001 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:45.697129 22914 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:45.799582 22762 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:45.800689 22906 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:45.784039 22908 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.784320 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2821) took 1189 ms. Trace:
I20260709 13:13:45.805967 22765 rpcz_store.cc:276] 0709 13:13:44.594866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.505935 (+911069us) service_pool.cc:224] Handling call
0709 13:13:45.784110 (+278175us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:45.784256 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403865856}
I20260709 13:13:45.689108 28038 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.786500 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2207) took 1580 ms. Trace:
I20260709 13:13:45.773698 27972 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:45.739672 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.814188 22757 rpcz_store.cc:276] 0709 13:13:44.205902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.676352 (+1470450us) service_pool.cc:224] Handling call
0709 13:13:45.698559 (+ 22207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.775510 (+ 76951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.775546 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:45.775597 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:45.775636 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.786255 (+ 10619us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b54002233d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:45.786304 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.786433 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176428800}
I20260709 13:13:45.788417 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.815192 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.816975 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:45.793066 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2204) took 1588 ms. Trace:
I20260709 13:13:45.817778 22761 rpcz_store.cc:276] 0709 13:13:44.205081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.675107 (+1470026us) service_pool.cc:224] Handling call
0709 13:13:45.675403 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.745106 (+ 69703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.792808 (+ 47702us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b54002dbed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:45.793003 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158660864}
I20260709 13:13:45.795197 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:45.800810 22839 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:45.785976 27753 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:45.807214 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.792925 22837 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.007s	sys 0.016s
W20260709 13:13:45.821103 22837 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.007s	sys 0.016s
I20260709 13:13:45.823674 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:45.824483 22760 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:45.815232 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75558.7us
I20260709 13:13:45.828460 22661 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.786854 28154 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:13:45.831012 23460 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.043s	sys 0.028s
I20260709 13:13:45.789921 28113 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:45.831507 23460 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.043s	sys 0.028s
I20260709 13:13:45.831226 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.833755 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:45.835824 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 634) took 1503 ms. Trace:
I20260709 13:13:45.836222 23460 rpcz_store.cc:276] 0709 13:13:44.332823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.332971 (+   148us) service_pool.cc:224] Handling call
0709 13:13:44.333274 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.333519 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:13:44.762926 (+429407us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.833841 (+1070915us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.834056 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:45.834102 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:45.834148 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.834426 (+   278us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.835734 (+  1308us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":644620,"prepare.run_cpu_time_us":3378,"prepare.run_wall_time_us":33759,"raft.queue_time_us":342314,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":114,"thread_start_us":1474088,"threads_started":3,"wal-append.queue_time_us":529236}
I20260709 13:13:45.783104 27986 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:45.851537 22911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.014s	sys 0.018s
W20260709 13:13:45.852147 22911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.014s	sys 0.018s
W20260709 13:13:45.860996 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.020s
W20260709 13:13:45.926059 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.012s	sys 0.023s
I20260709 13:13:45.774266 28072 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.821772 27972 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:45.926517 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 578987us
I20260709 13:13:45.797956 28001 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:45.927899 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:45.828274 28155 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:45.848999 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:45.942247 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:45.950903 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 465066us
I20260709 13:13:45.952446 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:45.955646 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 124493us
I20260709 13:13:45.957538 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.825210 28011 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:45.931751 22762 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:13:45.937791 27972 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:45.974644 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2210) took 1767 ms. Trace:
I20260709 13:13:45.932870 22913 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.933786 22912 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:45.934725 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:45.975126 22762 rpcz_store.cc:276] 0709 13:13:44.206760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.930034 (+1723274us) service_pool.cc:224] Handling call
0709 13:13:45.930939 (+   905us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:45.974249 (+ 43310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.974554 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.829768 28036 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:45.857967 28094 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:45.772006 28005 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.967489 22760 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:45.968793 22907 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:45.997051 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:45.997277 22760 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.998149 22907 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:13:45.928212 28058 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.998274 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2794) took 1770 ms. Trace:
W20260709 13:13:45.931131 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.861246 27873 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:45.965173 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:45.998829 22760 rpcz_store.cc:276] 0709 13:13:44.227465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.996695 (+1769230us) service_pool.cc:224] Handling call
0709 13:13:45.997080 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.997970 (+   890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.998208 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:45.832283 28154 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:45.980932 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:46.006062 22833 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.981225 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.007057 22839 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:45.786580 27912 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:45.990365 22762 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:46.020889 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2791) took 1794 ms. Trace:
I20260709 13:13:46.021342 22762 rpcz_store.cc:276] 0709 13:13:44.226665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.989794 (+1763129us) service_pool.cc:224] Handling call
0709 13:13:45.990153 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.020551 (+ 30398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.020810 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:46.000689 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:45.975111 28155 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:45.963963 27700 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:45.999552 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.038636 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:45.943713 28110 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.958750 27953 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:46.002426 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:46.003036 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2212) took 1749 ms. Trace:
I20260709 13:13:46.042558 22765 rpcz_store.cc:276] 0709 13:13:44.253450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.997434 (+1743984us) service_pool.cc:224] Handling call
0709 13:13:46.002519 (+  5085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.002722 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.002748 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.002793 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:46.002822 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.002878 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.002980 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:46.044735 22765 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:46.050113 22915 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:46.051339 28218 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:13:46.004638 28054 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:46.062384 27953 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:46.063825 27953 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.031512 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:46.066567 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64256.2us
I20260709 13:13:45.832757 28038 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.000417 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:46.008064 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:45.982501 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.008285 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.019866 22838 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:45.950312 27959 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:13:45.835018 28150 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.952024 28174 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.025743 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:46.001585 28153 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.910s	user 0.004s	sys 0.008s
I20260709 13:13:45.832167 28138 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.031013 28127 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:13:46.045462 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2806) took 1768 ms. Trace:
W20260709 13:13:46.073376 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 125987us
I20260709 13:13:46.078056 22839 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.078433 22835 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:46.108646 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 145902us
I20260709 13:13:46.008605 28053 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:13:46.123972 22765 rpcz_store.cc:276] 0709 13:13:44.277422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.043927 (+1766505us) service_pool.cc:224] Handling call
0709 13:13:46.044333 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.045167 (+   834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.045396 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:46.125337 22763 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:46.126921 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2213) took 1784 ms. Trace:
I20260709 13:13:46.111681 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:45.990025 28011 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.071154 28054 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:46.131330 22806 leader_election.cc:400] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
I20260709 13:13:46.136042 27805 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:46.147377 22806 leader_election.cc:403] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:13:46.147739 27805 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:46.119673 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 81149.7us
I20260709 13:13:46.008806 28058 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:46.150127 28221 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:46.127736 22838 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:46.152071 28058 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.152670 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:46.153088 22760 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.158748 22835 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:13:46.160219 22840 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:46.160796 28153 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:46.148849 27805 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.161466 28153 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.162663 27805 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.163058 28174 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.165073 28153 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:46.166074 28153 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:46.166797 27873 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:46.153319 22838 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:46.107033 28150 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:46.167654 28150 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.167078 28153 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.167578 28154 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.042574 28219 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:46.136061 28217 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:13:46.170212 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.181515 22805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.011s	sys 0.035s
I20260709 13:13:46.170387 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:46.174225 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:46.183624 22805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.011s	sys 0.035s
I20260709 13:13:46.183707 23461 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:46.105862 27912 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:46.087657 22833 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:46.073123 28161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.029s
I20260709 13:13:46.128295 22765 rpcz_store.cc:276] 0709 13:13:44.342096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.125788 (+1783692us) service_pool.cc:224] Handling call
0709 13:13:46.126316 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.126575 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.126602 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.126646 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:46.126676 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.126734 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.126853 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:46.179173 22909 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:46.112974 28143 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:46.186674 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.191068 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:46.146359 28216 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:13:46.185101 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:46.137820 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:46.192479 28161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.010s	sys 0.029s
I20260709 13:13:46.196415 27899 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.222414 27899 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:46.203485 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.205156 22840 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:46.227339 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.192165 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:46.135557 28011 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.228317 22833 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:46.205546 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:46.233716 22835 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.240289 22833 raft_consensus.cc:2410] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:46.240424 22840 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:46.071611 28001 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.249599 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2142) took 1283 ms. Trace:
I20260709 13:13:46.249997 22834 rpcz_store.cc:276] 0709 13:13:44.966051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.151207 (+185156us) service_pool.cc:224] Handling call
0709 13:13:45.178223 (+ 27016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.178470 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.178494 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.204682 (+ 26188us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.688564 (+483882us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:45.688606 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.688688 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.249335 (+560647us) raft_consensus.cc:1685] finished
0709 13:13:46.249359 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.249531 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":418580,"threads_started":1,"wal-append.queue_time_us":715531}
I20260709 13:13:46.226755 28130 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:46.182176 28219 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.213361 22839 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:46.185366 28217 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.244307 28011 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:46.193729 28122 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.269024 28217 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:46.243073 28072 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.269533 28011 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:46.248059 28224 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:46.280090 22841 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:46.253857 28130 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:46.286001 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.212359 22760 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:46.287807 22835 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:46.297379 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.280670 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 328338us
I20260709 13:13:46.349426 22832 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:46.352736 27751 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.002s
I20260709 13:13:46.275383 28001 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:46.358707 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.359236 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.278832 28011 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.371491 22732 leader_election.cc:400] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
I20260709 13:13:46.374629 22732 leader_election.cc:403] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:13:46.291735 28131 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:46.292522 28143 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.383920 22841 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:46.386287 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 383131us
I20260709 13:13:46.356606 28224 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:46.392431 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:46.398948 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1909) took 1107 ms. Trace:
I20260709 13:13:46.399556 22906 rpcz_store.cc:276] 0709 13:13:45.291313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.473517 (+182204us) service_pool.cc:224] Handling call
0709 13:13:46.398691 (+925174us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540010efd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:46.398866 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447779584}
I20260709 13:13:46.372586 28231 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.367951 28155 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:46.406522 22910 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:46.406196 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:46.353462 27751 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.364663 28144 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:46.420126 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:46.406221 28143 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:46.557826 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.473795 22761 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:13:46.542425 22838 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:13:46.565104 22761 pending_rounds.cc:85] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 7
I20260709 13:13:46.415630 28234 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.487815 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:46.565603 22761 pending_rounds.cc:107] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:13:46.565990 22761 raft_consensus.cc:2889] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:46.566108 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.479218 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2214) took 2134 ms. Trace:
W20260709 13:13:46.566125 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78438.2us
I20260709 13:13:46.413795 28153 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.471899 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:46.462708 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 292412us
I20260709 13:13:46.475514 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1912) took 1182 ms. Trace:
I20260709 13:13:46.485843 22758 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:13:46.420631 27792 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.003s
I20260709 13:13:46.467955 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1916) took 1171 ms. Trace:
I20260709 13:13:46.641817 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2146) took 1674 ms. Trace:
W20260709 13:13:46.542236 22759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.015s	sys 0.040s
W20260709 13:13:46.611374 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.004s	sys 0.009s
W20260709 13:13:46.610488 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.566979 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2215) took 2221 ms. Trace:
I20260709 13:13:46.414870 28080 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:46.580456 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55637.4us
W20260709 13:13:46.687235 22759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.024s	sys 0.043s
I20260709 13:13:46.684357 22910 rpcz_store.cc:276] 0709 13:13:45.296549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.742645 (+446096us) service_pool.cc:224] Handling call
0709 13:13:46.400769 (+658124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.413694 (+ 12925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.467712 (+ 54018us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b540010efd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:46.467878 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1512589184}
I20260709 13:13:46.685855 22760 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:46.687479 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.006s	sys 0.011s
W20260709 13:13:46.676573 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 527207us
I20260709 13:13:46.471244 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:46.685786 22837 rpcz_store.cc:276] 0709 13:13:44.967272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.230408 (+263136us) service_pool.cc:224] Handling call
0709 13:13:45.275176 (+ 44768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.821645 (+546469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.821677 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.832716 (+ 11039us) log.cc:844] Serialized 26 byte log entry
0709 13:13:46.366144 (+533428us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:46.366208 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.366306 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.641494 (+275188us) raft_consensus.cc:1685] finished
0709 13:13:46.641541 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.641732 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41893,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":949884,"threads_started":2,"wal-append.queue_time_us":696280}
I20260709 13:13:46.701891 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:46.478157 28228 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:13:46.712214 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:46.690253 22761 rpcz_store.cc:276] 0709 13:13:44.345766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.473120 (+2127354us) service_pool.cc:224] Handling call
0709 13:13:46.473397 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.566693 (+ 93296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.566925 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:46.685323 22915 rpcz_store.cc:276] 0709 13:13:45.292676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.672506 (+379830us) service_pool.cc:224] Handling call
0709 13:13:45.672777 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.050644 (+377867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.475273 (+424629us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400125cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:46.475448 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854196480}
W20260709 13:13:46.703029 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 629338us
I20260709 13:13:46.708544 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:46.567265 22765 rpcz_store.cc:276] 0709 13:13:44.345128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.197043 (+1851915us) service_pool.cc:224] Handling call
0709 13:13:46.197391 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.477952 (+280561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.477994 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.478037 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:46.478069 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.478947 (+   878us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400311354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:46.479006 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.479147 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644726016}
W20260709 13:13:46.639803 28208 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.867s	user 0.012s	sys 0.000s
I20260709 13:13:46.476581 27529 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:46.718881 28208 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.719117 27529 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.720008 28153 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:46.720405 28153 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.720719 28153 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:46.643858 28122 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.721117 27529 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:46.443316 28144 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.723364 28144 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.696050 27792 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:46.636641 28237 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:13:46.727205 22766 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:46.422108 27953 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.728183 27953 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.729076 27792 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.027482 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:46.703322 27986 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:46.709890 28209 raft_consensus.cc:2764] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:13:46.582005 28054 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:46.470088 28094 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:46.472373 28053 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:46.487030 27751 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.468147 28005 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.569033 28058 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:46.509815 28232 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:46.698323 28038 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:13:46.568167 28233 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
W20260709 13:13:46.692461 28113 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.858s	user 0.010s	sys 0.004s
I20260709 13:13:46.580781 28110 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:46.750622 28215 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.529s	user 0.003s	sys 0.001s
I20260709 13:13:46.697892 28241 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.730403 27953 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:46.414697 28130 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.093633 28130 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:46.933099 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:46.900056 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.099488 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.732371 28079 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:46.637593 28193 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.734725 28214 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:47.102077 28214 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:47.103127 28214 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.946380 22841 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:47.106166 27753 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:47.106729 27753 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.107029 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:47.105772 28155 consensus_queue.cc:1175] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" responder_term: 14 status { last_received { term: 13 index: 9 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:47.107724 27753 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.108740 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.908643 28140 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:47.109356 27753 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:47.054265 27986 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.721123 28208 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.979594 22834 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:47.112946 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.902634 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2209) took 2696 ms. Trace:
I20260709 13:13:47.113659 22766 rpcz_store.cc:276] 0709 13:13:44.206438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.757202 (+1550764us) service_pool.cc:224] Handling call
0709 13:13:45.757528 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.727612 (+970084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.902317 (+174705us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b540024b8d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:46.902557 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2229582592}
I20260709 13:13:46.742689 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2201) took 2538 ms. Trace:
I20260709 13:13:47.059123 28054 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:46.957984 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:46.659260 28219 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:46.736083 27912 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:46.740284 28154 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:47.004580 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2205) took 2799 ms. Trace:
I20260709 13:13:46.732149 28036 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:46.733464 22763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2824) took 2072 ms (client timeout 1686 ms). Trace:
W20260709 13:13:47.123749 22763 rpcz_store.cc:269] 0709 13:13:44.661690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.916470 (+254780us) service_pool.cc:224] Handling call
0709 13:13:46.733197 (+1816727us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400002dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:46.733393 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697371392}
I20260709 13:13:47.125265 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:46.875587 27529 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.731735 28213 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:47.005074 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2216) took 2658 ms. Trace:
I20260709 13:13:46.543497 28224 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:46.634382 28138 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:46.753579 28150 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.006s
W20260709 13:13:46.981110 28235 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:13:46.752949 28217 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.634943 28236 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:46.917224 22836 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:46.735688 27972 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.731s	user 0.007s	sys 0.003s
I20260709 13:13:47.138849 28213 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:47.139161 28054 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.139896 28213 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.140901 28036 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.705385 28131 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:46.836644 27805 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.000s
I20260709 13:13:47.133147 28150 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.145769 28131 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.147025 28217 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:46.734164 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 733837us
I20260709 13:13:47.147487 28150 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:46.962149 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.127707 27912 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.611153 28149 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
I20260709 13:13:47.148574 22833 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:46.731525 28246 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:47.127686 27986 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.150517 28246 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.129613 27529 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.699131 28234 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.153249 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.154220 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.154821 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.155289 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:46.908180 27899 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.416374 28231 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:46.715603 28080 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:13:46.780735 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72273.2us
W20260709 13:13:46.818833 28244 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.128s	user 0.001s	sys 0.001s
I20260709 13:13:46.811055 28248 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:46.935256 28011 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.003s
W20260709 13:13:46.877349 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 165224us
I20260709 13:13:47.190424 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:47.190426 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:46.736824 28143 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:46.955025 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:46.735033 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2211) took 2528 ms. Trace:
I20260709 13:13:46.850143 22761 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:47.195815 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2822) took 2600 ms. Trace:
I20260709 13:13:47.196040 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:47.196177 22761 rpcz_store.cc:276] 0709 13:13:44.596241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.849150 (+2252909us) service_pool.cc:224] Handling call
0709 13:13:46.849642 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.195506 (+345864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.195749 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.016372 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 640) took 1188 ms. Trace:
I20260709 13:13:47.025025 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.193795 22757 rpcz_store.cc:276] 0709 13:13:44.207025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.935798 (+1728773us) service_pool.cc:224] Handling call
0709 13:13:45.964868 (+ 29070us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:46.241795 (+276927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.241830 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:46.254398 (+ 12568us) log.cc:844] Serialized 52 byte log entry
0709 13:13:46.621239 (+366841us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:46.622184 (+   945us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.634085 (+ 11901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.634199 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.734784 (+100585us) raft_consensus.cc:1685] finished
0709 13:13:46.734809 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.734971 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11545,"prepare.queue_time_us":311,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":187639,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":243,"thread_start_us":603630,"threads_started":2,"wal-append.queue_time_us":456277}
W20260709 13:13:47.036798 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.030493 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.039186 22906 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:47.057890 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:47.200392 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 170148us
I20260709 13:13:47.201467 28150 raft_consensus.cc:2764] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:13:47.201905 22879 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.052474 28247 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.203392 28247 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.204586 28080 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.054358 28053 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.088768 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.223407 22839 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:46.974493 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.225451 22840 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:47.114884 22758 rpcz_store.cc:276] 0709 13:13:44.203943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.620920 (+1416977us) service_pool.cc:224] Handling call
0709 13:13:45.628706 (+  7786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.685226 (+1056520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.742349 (+ 57123us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b5400393354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:46.742604 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970601344}
I20260709 13:13:46.931164 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2217) took 2102 ms. Trace:
I20260709 13:13:47.226382 22764 rpcz_store.cc:276] 0709 13:13:44.828905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.930237 (+2101332us) service_pool.cc:224] Handling call
0709 13:13:46.930525 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.930722 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.930759 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.930817 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:46.930854 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.930927 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.931083 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.116777 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 704) took 1937 ms. Trace:
I20260709 13:13:47.118916 22838 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.120150 22834 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:47.120744 22837 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:47.121937 22759 rpcz_store.cc:276] 0709 13:13:44.205345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.676902 (+1471557us) service_pool.cc:224] Handling call
0709 13:13:45.677219 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.677435 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:13:45.986577 (+309142us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.705451 (+718874us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.004167 (+298716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.004225 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:47.004275 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.004354 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.004506 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17678,"prepare.queue_time_us":576957,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":88,"raft.queue_time_us":14637,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":3534,"spinlock_wait_cycles":691456,"thread_start_us":1316792,"threads_started":3,"wal-append.queue_time_us":739505}
I20260709 13:13:47.128091 22760 rpcz_store.cc:276] 0709 13:13:44.346366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.724518 (+2378152us) service_pool.cc:224] Handling call
0709 13:13:46.724928 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.725168 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.725199 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:46.725746 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:13:46.894483 (+168737us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:46.894536 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.894651 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:47.004838 (+110187us) raft_consensus.cc:1685] finished
0709 13:13:47.004862 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.005000 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":168304,"threads_started":1,"wal-append.queue_time_us":275839}
W20260709 13:13:47.131307 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.137784 28236 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.138183 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.067515 28005 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.119817 28233 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.138868 27972 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.147827 28131 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:46.731055 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 882105us
I20260709 13:13:47.189595 28208 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:47.159659 22766 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:47.228586 22911 rpcz_store.cc:276] 0709 13:13:45.179316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.179925 (+   609us) service_pool.cc:224] Handling call
0709 13:13:45.180239 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.184518 (+  4279us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:13:45.218644 (+ 34126us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.392884 (+174240us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b180014a760. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:45.863202 (+470318us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.863491 (+   289us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.925157 (+ 61666us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.925354 (+   197us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:45.925396 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.925487 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.030539 (+105052us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180014a760. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:46.486999 (+456460us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b180014a760. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:46.737320 (+250321us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:47.116488 (+379168us) raft_consensus.cc:1685] finished
0709 13:13:47.116526 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.116709 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30126,"prepare.queue_time_us":92291,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":122,"raft.queue_time_us":14296,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":26789,"spinlock_wait_cycles":85382656,"thread_start_us":633571,"threads_started":1,"wal-append.queue_time_us":961528}
I20260709 13:13:47.249897 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 722) took 2345 ms. Trace:
I20260709 13:13:47.161116 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.250645 22834 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:47.251014 23457 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:47.251834 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:47.252444 28268 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:13:47.253031 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.253175 28005 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.092239 28241 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.255307 28241 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:47.119656 28122 consensus_queue.cc:1175] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6545453e74344bb2957dd3729c5b3870" responder_term: 14 status { last_received { term: 13 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:47.259477 22762 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:13:47.259881 22762 pending_rounds.cc:85] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 7
I20260709 13:13:47.259917 28236 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:47.260864 27912 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.260986 22838 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:47.166826 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:47.261752 23460 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:47.262228 28153 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.541s	user 0.014s	sys 0.000s
I20260709 13:13:47.262679 28153 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.191612 23458 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:47.199944 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:46.870577 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 450562us
I20260709 13:13:47.055584 28094 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.266038 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:47.165794 28246 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.169147 27899 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.267047 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:47.268906 27899 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.206794 27805 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:47.159919 28185 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:47.270361 28153 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.207265 27986 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:47.270267 28185 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.270959 27986 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.271435 27986 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.163550 28219 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:47.273615 28208 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.273885 28219 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.275797 28208 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.241595 28217 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:47.209558 28231 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.209965 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.245494 28233 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:47.278530 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68685.8us
I20260709 13:13:47.279903 28233 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.280717 22760 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:47.213522 23463 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:47.281409 23463 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:47.282311 23463 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.283082 27972 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.284088 22832 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:47.200554 28011 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.285259 22759 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:47.239627 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.078017 28232 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.286134 22758 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:47.286585 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.287361 22758 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:13:47.287767 28232 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.287914 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2226) took 1749 ms. Trace:
I20260709 13:13:47.144793 28250 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:13:47.288335 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:47.242574 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46675.2us
I20260709 13:13:47.289034 22907 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:47.289600 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:47.244899 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.290910 22836 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 9.
W20260709 13:13:47.246830 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.291530 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2218) took 2429 ms. Trace:
I20260709 13:13:47.291915 22765 rpcz_store.cc:276] 0709 13:13:44.862628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.929199 (+2066571us) service_pool.cc:224] Handling call
0709 13:13:46.929751 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.930083 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:46.930781 (+   698us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.124564 (+193783us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.291058 (+166494us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b180028e460. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:13:47.291224 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.291275 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:47.291317 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.291381 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.291478 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5789,"prepare.queue_time_us":2540,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":23836,"raft.queue_time_us":3180,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":3487,"spinlock_wait_cycles":69173760,"thread_start_us":189935,"threads_started":1,"wal-append.queue_time_us":281175}
I20260709 13:13:47.294023 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:47.297251 28266 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 13:13:47.247875 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 724) took 2342 ms. Trace:
I20260709 13:13:47.260259 22762 pending_rounds.cc:107] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:13:47.305433 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 728) took 1609 ms. Trace:
I20260709 13:13:47.305734 22762 raft_consensus.cc:2889] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:47.205832 27873 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:47.319613 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.269974 28053 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.208401 28247 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.280579 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:47.211697 28234 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.283309 23463 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.198251 28193 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.197162 23465 rpcz_store.cc:276] 0709 13:13:45.828012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.828165 (+   153us) service_pool.cc:224] Handling call
0709 13:13:45.828512 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.967327 (+138815us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:13:45.971683 (+  4356us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.014569 (+1042886us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.015139 (+   570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.015227 (+    88us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:47.015295 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.016017 (+   722us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400587fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:47.016095 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.016284 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":311790,"prepare.queue_time_us":286122,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":87,"raft.queue_time_us":312848,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":209,"spinlock_wait_cycles":318576640,"thread_start_us":730269,"threads_started":2,"wal-append.queue_time_us":898212}
W20260709 13:13:47.330379 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49904.2us
I20260709 13:13:47.331629 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.332391 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.332940 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:47.333989 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44506.3us
I20260709 13:13:47.335914 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.287801 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.289059 28250 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:47.298475 22839 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:47.298488 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.299954 22765 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:47.300611 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.301005 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:47.250303 22766 rpcz_store.cc:276] 0709 13:13:44.904451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.130315 (+2225864us) service_pool.cc:224] Handling call
0709 13:13:47.159299 (+ 28984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:47.249625 (+ 90326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.249841 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.302680 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.288342 22758 rpcz_store.cc:276] 0709 13:13:45.539036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.286792 (+1747756us) service_pool.cc:224] Handling call
0709 13:13:47.287090 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.287695 (+   605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.287864 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.305876 22763 rpcz_store.cc:276] 0709 13:13:45.696288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.304636 (+1608348us) service_pool.cc:224] Handling call
0709 13:13:47.304936 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.305132 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.305156 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.305200 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:47.305231 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.305278 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.305377 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.308689 23460 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:13:47.305189 22757 rpcz_store.cc:276] 0709 13:13:44.905058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.199305 (+2294247us) service_pool.cc:224] Handling call
0709 13:13:47.245394 (+ 46089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.247542 (+  2148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.247572 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.247615 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:47.247649 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.247713 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.247821 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.315902 22907 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:13:47.273144 28185 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.321208 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 726) took 2372 ms. Trace:
I20260709 13:13:47.322134 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.360632 22911 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:47.360893 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 727) took 1665 ms. Trace:
I20260709 13:13:47.277483 28219 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.360409 22760 rpcz_store.cc:276] 0709 13:13:44.949294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.233764 (+2284470us) service_pool.cc:224] Handling call
0709 13:13:47.246778 (+ 13014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.281141 (+ 34363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.320963 (+ 39822us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:47.321146 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77162880}
I20260709 13:13:47.402110 28001 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.402693 28001 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.403779 22760 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:47.159219 23461 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:47.405371 28001 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.339139 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2220) took 2430 ms. Trace:
I20260709 13:13:47.405952 28001 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:47.331629 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.333066 22910 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:47.345016 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.309624 27899 raft_consensus.cc:2764] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:47.324287 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.361364 22765 rpcz_store.cc:276] 0709 13:13:45.695883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.299455 (+1603572us) service_pool.cc:224] Handling call
0709 13:13:47.299764 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.360589 (+ 60825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.360831 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.319432 28005 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:47.345942 28236 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:47.304351 27912 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:47.367983 22763 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:13:47.358201 28150 raft_consensus.cc:2764] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:47.375948 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.377066 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.379382 22758 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:47.371134 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:47.386132 22766 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:47.392427 27986 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.357889 28053 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:47.394716 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.376698 28155 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:47.378630 28224 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.401261 27753 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:47.374846 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:47.384928 28094 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.422444 28079 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.339743 28140 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:47.368779 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68096.9us
I20260709 13:13:47.445384 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:47.446022 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.447644 28247 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:47.410351 28219 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.451179 27986 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:47.431502 27912 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.318910 27805 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:13:47.350664 28250 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.454954 28250 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.455349 27912 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.441883 28224 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.457150 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.426463 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2232) took 1010 ms. Trace:
I20260709 13:13:47.458207 22763 rpcz_store.cc:276] 0709 13:13:46.416383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.367459 (+951076us) service_pool.cc:224] Handling call
0709 13:13:47.367721 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.422491 (+ 54770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.426392 (+  3901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:47.458986 28224 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.429158 22912 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:47.430469 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59437.8us
I20260709 13:13:47.430768 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 731) took 1712 ms. Trace:
I20260709 13:13:47.461148 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.461344 22766 rpcz_store.cc:276] 0709 13:13:45.718248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.351407 (+1633159us) service_pool.cc:224] Handling call
0709 13:13:47.385943 (+ 34536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.430518 (+ 44575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.430713 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27392}
I20260709 13:13:47.462000 22909 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:47.410195 28011 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.461148 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:47.463451 27873 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:47.371497 28185 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.436075 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.406221 22759 rpcz_store.cc:276] 0709 13:13:44.908734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.232793 (+2324059us) service_pool.cc:224] Handling call
0709 13:13:47.233193 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.285656 (+ 52463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.338902 (+ 53246us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b5400368b54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:47.339079 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119318272}
I20260709 13:13:47.435241 28155 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.467813 22908 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:47.371757 28138 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:13:47.427313 28236 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.422117 28005 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.346118 28193 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.464136 27873 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.442631 27899 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.434123 28130 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:47.459406 22913 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:47.470176 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.470377 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:47.406430 28001 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.473958 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.474195 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.507570 22912 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:47.507663 23460 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:47.509465 28001 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.509351 28250 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.477072 22914 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:13:47.400482 23462 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:47.439877 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62849.4us
I20260709 13:13:47.520426 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.521710 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.470690 28150 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:47.486943 23463 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:47.489184 28005 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.531355 22806 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:47.531749 22806 leader_election.cc:403] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:13:47.499670 28130 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:47.532550 22910 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:47.477196 28236 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.501969 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 725) took 2596 ms. Trace:
I20260709 13:13:47.502362 27805 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:47.502483 27873 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.474692 28185 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:47.530210 28271 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.316s	user 0.000s	sys 0.003s
W20260709 13:13:47.504289 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 238327us
I20260709 13:13:47.552153 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:47.504747 28259 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.505265 28233 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:47.530401 28150 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.506111 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.556591 22766 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:47.475497 28155 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:47.505175 28140 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:47.512996 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:47.566167 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53294.7us
I20260709 13:13:47.567691 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:47.515326 28153 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:47.474931 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:47.538954 28217 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:47.569248 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94412.8us
I20260709 13:13:47.530929 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.546012 27805 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.521937 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:47.575042 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53184.8us
I20260709 13:13:47.544196 22764 rpcz_store.cc:276] 0709 13:13:44.905342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.227759 (+2322417us) service_pool.cc:224] Handling call
0709 13:13:47.228181 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.501614 (+273433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.501645 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.501686 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:47.501717 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.501785 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.501905 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":272943,"raft.queue_time_us":275409,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107}
I20260709 13:13:47.484993 28094 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.504166 28258 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:47.554112 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.532919 27899 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.504310 22757 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:47.561734 28150 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.563287 28110 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:47.516893 27986 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:13:47.537233 28149 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:47.571730 22763 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.572557 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:47.565364 23461 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:47.495357 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2809) took 3172 ms. Trace:
I20260709 13:13:47.605834 22762 rpcz_store.cc:276] 0709 13:13:44.323479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.066437 (+1742958us) service_pool.cc:224] Handling call
0709 13:13:46.734328 (+667891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.324262 (+589934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.493994 (+169732us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b5400002dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:47.495110 (+  1116us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b5400002dd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:47.495282 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2741871104}
I20260709 13:13:47.583328 27972 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:47.561262 22806 connection.cc:256] RPC call timeout handler was delayed by -1.9257597319999604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 13:13:47.581862 28036 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:47.582293 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 207546us
I20260709 13:13:47.552433 28219 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:47.558871 22732 leader_election.cc:400] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
I20260709 13:13:47.561302 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:47.550972 28224 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:13:47.572371 28154 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:13:47.598351 28213 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.552294 28122 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:13:47.565441 28140 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.619537 28140 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.620798 28140 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:47.621805 27805 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.623018 27529 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.624272 28250 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:47.625705 28185 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.613396 28217 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:47.626646 28217 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.627000 28217 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.627113 27805 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.581413 28138 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.628234 28138 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.630424 28138 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:47.568565 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:47.630600 28131 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:47.631201 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62721.9us
I20260709 13:13:47.562971 27912 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:13:47.574024 28143 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:47.614001 22732 leader_election.cc:403] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:47.648012 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89279.9us
I20260709 13:13:47.605532 28193 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:47.571408 28233 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.598557 28258 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.616976 28213 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.640347 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:47.642365 22910 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:13:47.614902 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53801us
I20260709 13:13:47.672390 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:47.653230 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.616343 28154 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:47.611565 28036 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.612525 27972 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:47.639518 28001 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:47.631093 28131 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:47.691495 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175003us
W20260709 13:13:47.692049 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58952.2us
I20260709 13:13:47.646802 28259 raft_consensus.cc:2764] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:47.648919 28054 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:47.646948 28247 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.700028 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.706854 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.708467 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.700958 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.710079 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.711062 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.701030 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.653110 28232 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
I20260709 13:13:47.701925 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.714570 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.720412 22906 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:47.702181 22913 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:47.720605 22839 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.702649 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.722081 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.722539 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.723156 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.714253 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.723794 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.724407 23461 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:47.714282 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.655531 28143 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:47.735628 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.736428 23463 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:47.735630 22912 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:47.736816 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.737404 22832 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.717327 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.717762 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.739068 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.739646 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.739938 22762 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:47.718346 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:47.741375 22909 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.719110 22766 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:13:47.759272 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.703795 28054 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:47.725863 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.724678 22841 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:47.767334 22839 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:47.729039 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.768749 22906 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:47.711684 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:47.699378 22910 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:13:47.695796 27972 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.611174 28130 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.743122 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:47.800601 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.801443 22835 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:47.801601 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 129352us
I20260709 13:13:47.781785 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.727860 28058 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.809715 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:47.747088 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.747205 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.748863 22838 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:47.811250 22840 raft_consensus.cc:2368] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13.
I20260709 13:13:47.741473 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.751009 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.752266 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.812178 22757 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:47.654630 28110 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:13:47.697588 28193 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:47.736050 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:47.758423 22836 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:13:47.759505 22766 pending_rounds.cc:85] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:13:47.816345 22766 pending_rounds.cc:107] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:13:47.816653 22766 raft_consensus.cc:2889] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:47.817824 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.818931 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.715076 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:47.819962 23462 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:47.824363 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69498.3us
I20260709 13:13:47.723156 28153 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.713419 27899 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:13:47.745733 28259 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:47.842193 23462 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:47.800647 23458 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.803310 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.843801 22834 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:47.808673 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92158.7us
I20260709 13:13:47.745869 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:47.770020 22910 pending_rounds.cc:85] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:13:47.826843 22832 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:47.830421 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.845052 22910 pending_rounds.cc:107] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:13:47.845479 22910 raft_consensus.cc:2889] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:47.845546 22765 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:47.781877 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.721935 28241 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:13:47.838860 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:47.838956 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.839079 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.762920 27912 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:47.720266 27529 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:13:47.805374 28232 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.846769 22760 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:47.875865 22757 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:47.847697 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.877859 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.878607 22832 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.836848 22909 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:47.772778 22758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46950
I20260709 13:13:47.879945 22758 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:47.880470 22758 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:47.721985 28248 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:13:47.727540 28138 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:13:47.856920 22835 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:47.729357 28247 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:47.863587 22766 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:47.691968 28036 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.833047 28193 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:47.870184 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.871891 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.872684 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.908233 22838 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:47.760337 28208 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
W20260709 13:13:47.883419 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44651.8us
I20260709 13:13:47.885375 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:47.890041 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55965.2us
I20260709 13:13:47.831802 28246 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.936355 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.936794 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:47.866782 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.838277 27751 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:47.773669 28094 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:13:47.937407 22766 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:47.929584 22758 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.942342 22832 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:47.848946 27986 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:47.931572 28127 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:47.940986 28113 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:13:47.949296 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.981863 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:47.984131 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:47.985093 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.838141 28079 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:13:47.987489 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:47.950428 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:47.998971 22834 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:48.001570 23458 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:48.004622 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.005493 22838 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:48.006603 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:48.007296 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.856401 27982 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:13:47.828589 28058 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.850584 28153 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:47.933095 28193 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:48.015801 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:47.858834 28259 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.019934 23460 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:47.946677 28250 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.026100 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:47.995702 28127 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:48.016393 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:48.028716 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.512s (SENT) [suppressed 38 similar messages]
I20260709 13:13:48.044838 22832 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:48.032186 22763 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:48.032960 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.014400 28001 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:48.035216 22835 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:47.979835 27953 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.036839 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:48.038657 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:48.050854 22764 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:48.007371 28234 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.950439 28143 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.053319 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:47.905709 28232 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.053380 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.056041 22762 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:48.060573 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 128572us
I20260709 13:13:48.053153 28011 raft_consensus.cc:2764] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:13:48.065737 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.070926 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.071641 23460 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:48.071731 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.009927 27986 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.043578 28127 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.996569 28246 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.038520 27873 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:13:48.041457 28250 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:48.094703 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57988.2us
I20260709 13:13:48.099536 28233 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.101895 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.103920 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.115821 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.118391 23463 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:48.059114 27953 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:48.117398 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 136721us
I20260709 13:13:48.125627 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:48.126392 28293 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:48.127377 28283 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:13:48.127878 28280 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.393s	user 0.001s	sys 0.001s
I20260709 13:13:48.129917 28259 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.130384 28286 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.247s
I20260709 13:13:48.131095 28258 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.142280 28213 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:48.143698 28288 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:13:48.115063 27982 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.166215 28153 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.166683 27982 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:48.167320 28291 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:13:48.168335 27805 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
W20260709 13:13:48.149281 28290 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:48.123857 28131 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.123286 28278 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.298s
I20260709 13:13:48.123420 28248 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:48.147729 28293 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:48.143899 28298 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
W20260709 13:13:48.162909 28287 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:48.164069 28058 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.154794 28304 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.117331 28306 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:48.145079 28289 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:13:48.169898 28285 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
I20260709 13:13:48.170078 28302 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:48.167353 28232 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:48.177733 28236 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.005s
I20260709 13:13:48.173441 28293 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.183269 28247 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.181238 28302 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.185402 28250 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:48.186081 28236 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.185348 28302 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:48.188676 28236 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.191902 28293 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.194861 28293 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.195833 28302 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.227246 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:48.242774 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:48.243700 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:48.245054 22764 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:13:48.247394 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175572us
I20260709 13:13:48.248198 27805 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:48.248916 27805 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:48.250756 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1878) took 1204 ms. Trace:
I20260709 13:13:48.250962 27805 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.254307 27805 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.145674 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:48.257469 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 112427us
W20260709 13:13:48.259475 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 113727us
I20260709 13:13:48.260715 27953 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.164672 27986 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.244714 22763 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:48.142854 28001 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.264955 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:48.266269 22763 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:48.268596 22763 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:48.242816 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 723) took 3337 ms. Trace:
W20260709 13:13:48.172840 28150 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.000s
I20260709 13:13:48.285017 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:48.286149 28185 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:48.286790 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:48.289561 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:48.292855 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.295320 27912 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:48.297055 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.251338 22833 rpcz_store.cc:276] 0709 13:13:47.046329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.220751 (+174422us) service_pool.cc:224] Handling call
0709 13:13:47.221067 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:47.237526 (+ 16459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.237562 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:47.241274 (+  3712us) log.cc:844] Serialized 52 byte log entry
0709 13:13:47.508888 (+267614us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b18001fe460. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:13:47.636750 (+127862us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:47.636810 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.636907 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.249861 (+612954us) raft_consensus.cc:1685] finished
0709 13:13:48.249901 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.250444 (+   543us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26796,"raft.queue_time_us":20317,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5163,"spinlock_wait_cycles":40526592,"thread_start_us":127591,"threads_started":1,"wal-append.queue_time_us":479326}
I20260709 13:13:48.306413 27982 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.308439 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.309903 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:48.339078 28185 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:48.339078 22910 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:48.339648 28185 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:48.339790 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.340710 22833 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:48.341888 22834 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:48.342279 28150 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.343072 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.353924 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:48.357465 28278 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:13:48.361318 28310 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:13:48.363252 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53878.5us
I20260709 13:13:48.364300 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.370376 28309 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.317981 22761 rpcz_store.cc:276] 0709 13:13:44.904755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.197193 (+2292438us) service_pool.cc:224] Handling call
0709 13:13:47.197630 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:47.499658 (+302028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.499710 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:47.500410 (+   700us) log.cc:844] Serialized 26 byte log entry
0709 13:13:47.649363 (+148953us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:47.649768 (+   405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.649844 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.241948 (+592104us) raft_consensus.cc:1685] finished
0709 13:13:48.242004 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.242475 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":295722,"raft.queue_time_us":303149,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":120,"thread_start_us":145240,"threads_started":1,"wal-append.queue_time_us":255407}
I20260709 13:13:48.371474 28309 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.277170 28054 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.384785 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:48.386044 28005 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.386830 28214 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:48.387229 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:48.388278 22906 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:13:48.385031 28122 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:48.389654 28122 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:48.391162 28122 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.395025 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:48.406462 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.375572 28247 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:48.408313 28247 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.410292 22907 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:48.410540 28150 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.412671 28278 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:48.415441 28155 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.841s	user 0.006s	sys 0.012s
I20260709 13:13:48.415961 28155 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.412909 22834 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:48.424142 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71955.6us
W20260709 13:13:48.421916 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 160797us
I20260709 13:13:48.432014 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.434697 28213 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:48.434865 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.435474 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.436518 22839 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:48.402480 28214 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:48.425799 28278 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:48.450100 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.443602 28213 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:48.437047 28247 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.463461 22840 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:48.471362 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.472545 22840 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:48.474073 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.475395 22840 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:48.477368 22840 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:48.478358 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.514405 22762 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:48.516727 28214 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.522176 22913 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:48.518697 28278 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.536582 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:48.543098 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:48.539913 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:48.545205 22913 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:48.546016 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:48.546465 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 417074us
I20260709 13:13:48.547636 28001 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:48.548058 28001 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.548439 28001 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.550195 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.551542 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:48.552649 28079 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.553650 28079 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:48.554169 28079 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.554672 28079 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:48.555210 28140 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.014s
I20260709 13:13:48.553903 28213 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.558203 28213 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:48.560325 28001 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:48.562158 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.563059 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43676.5us
I20260709 13:13:48.563642 28001 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:48.567019 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 202880us
I20260709 13:13:48.571153 28302 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.577785 28054 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.582206 28127 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:48.587208 28217 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.821s	user 0.016s	sys 0.010s
I20260709 13:13:48.588081 28217 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:48.592188 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.596761 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 44183.4us
W20260709 13:13:48.602028 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53575.6us
I20260709 13:13:48.603168 22833 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:48.597101 28127 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:48.607471 28079 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.620786 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:48.637843 22841 raft_consensus.cc:2368] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:48.628183 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:48.600584 28150 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:48.637358 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:48.658342 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:48.660409 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 324528us
I20260709 13:13:48.666976 22906 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:48.663460 22834 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:13:48.638994 28155 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.634418 28127 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.674218 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61839.9us
I20260709 13:13:48.676064 28080 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:48.676568 28080 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.676782 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:48.677002 28080 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:48.625331 28246 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:48.685933 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 313435us
W20260709 13:13:48.691594 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73644.3us
I20260709 13:13:48.692596 28231 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:48.692773 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:48.693239 28231 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.693624 28248 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:48.694772 28143 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:48.697613 22911 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:48.694610 28248 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.693873 28231 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.693794 22914 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:48.698750 22913 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.678630 28306 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:48.706144 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50385.6us
I20260709 13:13:48.708527 22765 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:48.711164 28185 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.720084 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:48.733659 22833 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [15.8-16.9]
I20260709 13:13:48.735031 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:48.735713 22833 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.733008 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.681677 28127 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.740345 22834 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:48.718902 27912 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:48.743321 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 56787.3us
I20260709 13:13:48.744995 28110 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.744455 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:48.718796 28296 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:48.751070 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:48.756709 23461 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:48.761821 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:48.763441 28005 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:48.764017 28005 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.764412 28005 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:48.758939 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:48.765040 28314 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:13:48.773046 23465 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:48.773976 22758 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.774510 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46950 (request call id 2842) took 1056 ms. Trace:
I20260709 13:13:48.774763 22758 rpcz_store.cc:276] 0709 13:13:47.737883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.750620 (+ 12737us) service_pool.cc:224] Handling call
0709 13:13:48.773200 (+1022580us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b4c0034b9c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689fdbda20 00007f689fdcea29 00007f689fdcf13f 00007f689fd932ab 00007f689fdc0f3c 00007f689fdc13fe 00007f68a004e88b 00007f68a1a4d495 00007f68a1a9e30f 00007f68a1ae8a55 00007f68a6ab8cd0
0709 13:13:48.774428 (+  1228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84023552}
I20260709 13:13:48.779477 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:48.744408 28306 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.785917 28292 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:48.786767 28292 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:48.786149 28316 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:13:48.785939 28143 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.788374 28143 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.767831 28309 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.758193 28155 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.794458 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:48.816497 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84007.8us
I20260709 13:13:48.823279 22911 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:48.784927 28246 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.832445 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 75722us
I20260709 13:13:48.832221 22834 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:48.833952 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:48.825232 28247 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:48.784287 28110 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:13:48.799917 27912 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.860256 27912 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:48.869840 28185 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.871991 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:48.874014 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.876677 27899 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:48.877735 28113 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:48.878131 28122 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:48.882468 28036 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.915s	user 0.013s	sys 0.007s
I20260709 13:13:48.883003 28036 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.884482 23462 raft_consensus.cc:2368] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:48.886189 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:48.886897 28193 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.714s	user 0.016s	sys 0.006s
I20260709 13:13:48.887431 28193 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:48.890533 28324 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:13:48.893810 22835 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:48.897326 28276 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:48.897830 28276 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:48.898651 28319 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:13:48.901904 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:48.903369 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2177) took 1140 ms. Trace:
I20260709 13:13:48.903779 22841 rpcz_store.cc:276] 0709 13:13:47.763267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.778951 (+ 15684us) service_pool.cc:224] Handling call
0709 13:13:48.903108 (+1124157us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54003223d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:48.903281 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704390528}
I20260709 13:13:48.908355 28217 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.909456 28282 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:48.909902 28282 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:48.910928 28282 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.913975 28296 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.915221 28306 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:48.916759 27972 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.085s	user 0.009s	sys 0.012s
I20260709 13:13:48.917172 22837 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:48.917654 22837 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:48.917682 28143 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.918601 28248 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:48.918548 22837 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.832896 28278 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:13:48.920017 22837 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:48.920424 27805 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.006s
I20260709 13:13:48.922230 28127 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:48.922922 28247 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.832825 28292 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:48.927486 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2179) took 1155 ms. Trace:
I20260709 13:13:48.927861 22839 rpcz_store.cc:276] 0709 13:13:47.772675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.866359 (+ 93684us) service_pool.cc:224] Handling call
0709 13:13:48.927258 (+1060899us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:48.927419 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1114328832}
I20260709 13:13:48.932556 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1938) took 1231 ms. Trace:
I20260709 13:13:48.932996 22907 rpcz_store.cc:276] 0709 13:13:47.704376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.722917 (+ 18541us) service_pool.cc:224] Handling call
0709 13:13:48.932266 (+1209349us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54003381d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:48.932476 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577605504}
I20260709 13:13:48.917244 27972 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.936412 27972 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.937821 22840 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:48.938002 28110 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:48.938532 28110 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:48.939579 28110 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.940342 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1894) took 1169 ms. Trace:
I20260709 13:13:48.941015 28110 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:48.942377 28321 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.237s	user 0.003s	sys 0.000s
I20260709 13:13:48.853271 28246 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:48.944701 22841 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:48.940755 22840 rpcz_store.cc:276] 0709 13:13:47.771442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.479149 (+707707us) service_pool.cc:224] Handling call
0709 13:13:48.479705 (+   556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.938314 (+458609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.940076 (+  1762us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b5400039dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:48.940259 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053045632}
I20260709 13:13:48.955677 23465 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:48.957122 22835 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:48.962131 22757 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:13:48.965569 22833 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:48.912045 28282 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.921099 28278 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:13:48.925380 28247 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:48.977898 28281 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:13:48.978423 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 159888us
I20260709 13:13:48.978751 22909 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:13:48.998139 22835 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:48.982151 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:48.982483 22761 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:48.959296 22913 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:48.994710 22759 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:13:49.001585 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:49.002369 22765 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
W20260709 13:13:49.002803 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.003551 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.001902 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:49.004020 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:48.997918 22910 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:49.009106 22906 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:48.998684 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:49.010957 22907 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:49.000454 22759 pending_rounds.cc:85] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 5
I20260709 13:13:49.013046 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:49.016943 28005 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.017750 28143 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:49.019127 22912 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:49.019152 28143 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:49.020490 28143 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.020941 28246 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.021711 28079 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:49.023054 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:49.023802 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:49.009677 28185 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.024652 22763 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:49.024881 28185 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.026716 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:48.969697 27982 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.015249 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:49.011646 22759 pending_rounds.cc:107] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:13:49.043421 22759 raft_consensus.cc:2889] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:48.985263 28036 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.049702 22834 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:49.007388 28193 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:49.064239 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49147.5us
W20260709 13:13:49.073666 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.073380 28305 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:13:49.074249 28080 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.075938 22915 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:49.076108 28005 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:49.077560 22911 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:49.080893 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 303520us
I20260709 13:13:49.113184 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:49.115334 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49934.5us
I20260709 13:13:49.115449 22837 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:49.117015 22833 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:49.021332 28231 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.121526 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:49.123405 28036 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:49.125756 27912 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:49.065402 28185 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:49.137318 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46684.6us
W20260709 13:13:49.136788 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.145203 28248 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:49.145906 28248 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.148916 28248 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.130028 28246 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:49.171569 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:13:49.171895 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 145300us
I20260709 13:13:49.172473 23457 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:49.173122 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:49.173964 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:49.174687 22766 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:49.175181 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.176097 28231 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:49.176802 28231 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:49.177636 28241 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.200s	user 0.015s	sys 0.011s
W20260709 13:13:49.178048 28241 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.303s	user 0.015s	sys 0.011s
I20260709 13:13:49.151011 28079 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:49.179032 28079 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.179479 28079 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:49.181829 27751 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:49.182771 27751 consensus_queue.cc:1243] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:49.183521 27805 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:49.186131 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:49.186986 22763 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:49.187507 28214 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:49.187958 28214 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:49.189849 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:49.195742 28155 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.195925 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:49.196499 28155 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:49.197671 28318 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:13:49.200291 23461 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:49.173396 22910 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:13:49.203809 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.210991 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.216455 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:49.140363 28080 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:49.209760 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:49.215061 28306 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 13:13:49.225847 28241 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:49.153474 28113 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:49.228658 28155 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.230037 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:49.266670 28281 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:49.267534 22915 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:49.230796 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:49.270148 23462 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:49.231683 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:49.232070 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.271586 22909 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:49.271485 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:49.226310 28296 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:13:49.161314 28193 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:49.275244 28213 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:49.129894 27982 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.276139 28213 consensus_queue.cc:1243] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:49.276734 28213 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:49.277696 28278 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:49.278044 27982 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:49.278123 28278 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:49.255816 28333 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:49.279522 28278 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.280939 28278 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:49.256259 27700 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:49.281620 28079 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.261656 28113 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.284454 28079 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:49.265781 28305 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:49.275048 28193 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.253259 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:49.265183 22879 leader_election.cc:400] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13.
I20260709 13:13:49.288437 28219 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:49.290055 22908 raft_consensus.cc:2368] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:49.220785 28246 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:49.292582 22910 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:13:49.252748 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:49.299118 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:49.332451 22763 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:49.299506 28154 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:49.300127 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:49.301358 22834 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:49.302945 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:49.304546 28213 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.307145 28276 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.312937 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:49.288023 22766 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:49.189007 28214 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.327817 22907 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
W20260709 13:13:49.331060 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 83396.8us
I20260709 13:13:49.333530 27972 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:49.334827 22915 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:49.349146 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:49.351869 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:49.357250 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:49.361984 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:49.362648 22913 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:49.358172 22835 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:49.331753 28219 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:49.334255 28154 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:49.405691 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:49.406533 22763 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:49.409830 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 236850us
I20260709 13:13:49.410368 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:49.418244 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66461.3us
I20260709 13:13:49.423067 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:49.366446 28276 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:49.409055 28219 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:49.403476 28278 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:49.422407 22766 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
W20260709 13:13:49.414788 28122 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.004s
I20260709 13:13:49.346029 28213 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.440451 28246 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:49.443109 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 178168us
I20260709 13:13:49.417727 28292 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:13:49.419489 28079 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.473462 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:49.453852 28150 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:49.480991 22834 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:49.486150 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:49.487813 22763 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
W20260709 13:13:49.490917 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.461412 28278 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:49.469622 27972 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:49.479058 28155 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.502144 22763 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:49.509253 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 178237us
I20260709 13:13:49.511245 27972 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.513646 28351 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:49.506609 28150 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:49.514619 28328 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.515457 28328 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.515872 28150 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.517347 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2184) took 1661 ms. Trace:
I20260709 13:13:49.518893 28213 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.517838 22832 rpcz_store.cc:276] 0709 13:13:47.855441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.806915 (+951474us) service_pool.cc:224] Handling call
0709 13:13:48.834967 (+ 28052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.916013 (+ 81046us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:49.164149 (+248136us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.516717 (+352568us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.516777 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:49.516828 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.516899 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.517055 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63211,"prepare.queue_time_us":356828,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":28,"thread_start_us":437474,"threads_started":2,"wal-append.queue_time_us":255321}
I20260709 13:13:49.520349 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:49.521262 22832 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:49.523168 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:49.523327 22832 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:49.524024 28281 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:49.524657 28330 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.525699 28330 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.524641 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:49.528693 28281 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:49.529070 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:49.494897 28305 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.525780 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:49.535610 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 853) took 1239 ms. Trace:
I20260709 13:13:49.536108 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1901) took 1265 ms. Trace:
I20260709 13:13:49.547482 22832 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:13:49.529330 28110 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:13:49.536778 22840 rpcz_store.cc:276] 0709 13:13:48.296569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.954665 (+658096us) service_pool.cc:224] Handling call
0709 13:13:48.962373 (+  7708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.962650 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:49.163610 (+200960us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.535279 (+371669us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.535331 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:49.535374 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.535446 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.535549 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62273,"prepare.queue_time_us":310394,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":37,"thread_start_us":500730,"threads_started":2,"wal-append.queue_time_us":346556}
I20260709 13:13:49.544335 22839 rpcz_store.cc:276] 0709 13:13:48.270514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.928748 (+658234us) service_pool.cc:224] Handling call
0709 13:13:48.929212 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.970341 (+ 41129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.970392 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.030641 (+ 60249us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.475860 (+445219us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:49.475909 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.475991 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.535905 (+ 59914us) raft_consensus.cc:1685] finished
0709 13:13:49.535929 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.536055 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75067,"raft.queue_time_us":40766,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":11697,"thread_start_us":359980,"threads_started":1,"wal-append.queue_time_us":442112}
I20260709 13:13:49.529270 28281 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:49.495467 28079 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.542299 27899 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.529858 28122 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:49.563426 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2183) took 1686 ms. Trace:
W20260709 13:13:49.568627 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 114404us
I20260709 13:13:49.563483 28122 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.551992 28193 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.564443 28328 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.571035 28278 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.563938 22838 rpcz_store.cc:276] 0709 13:13:47.848300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.788605 (+940305us) service_pool.cc:224] Handling call
0709 13:13:48.789354 (+   749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.789932 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.789973 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.790620 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.140527 (+349907us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:49.140584 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.140703 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.534219 (+393516us) raft_consensus.cc:1685] finished
0709 13:13:49.534247 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.534410 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77559,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"thread_start_us":349399,"threads_started":1,"wal-append.queue_time_us":734305}
I20260709 13:13:49.595468 22833 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:49.597676 22840 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:49.603305 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.605370 22834 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:49.594663 28282 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:49.611294 28282 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:49.612509 28282 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.569430 27899 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.614820 27700 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:49.615553 27700 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:49.563916 28281 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.550019 28305 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:49.630916 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 101964us
I20260709 13:13:49.617332 28342 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:49.624075 28079 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:49.625695 28328 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.628504 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:49.650403 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.561442 28155 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.714493 28306 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.714725 28342 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.715261 28306 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.715960 28155 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.717175 28342 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:49.657388 28305 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:49.658258 28355 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:13:49.659243 28345 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:13:49.667622 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:49.701866 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:49.717648 28130 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.717433 28306 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:49.724316 27700 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:49.655416 28338 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:13:49.801461 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133935us
I20260709 13:13:49.735291 28122 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:49.736969 28350 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
I20260709 13:13:49.740097 28330 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:49.742492 28352 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:13:49.748271 28328 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:49.759470 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 163109us
W20260709 13:13:49.761181 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.038s	sys 0.031s
W20260709 13:13:49.815163 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.040s	sys 0.032s
I20260709 13:13:49.766000 22834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:13:49.815892 22834 raft_consensus.cc:606] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:49.816288 22834 raft_consensus.cc:614] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:49.767417 28348 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.241s	user 0.001s	sys 0.002s
I20260709 13:13:49.777606 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:49.817993 22765 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:49.802767 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:49.822127 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:49.810693 28305 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:49.831120 27982 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:49.837772 28358 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:49.846989 28080 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:49.850214 28219 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:49.851133 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 965015us
I20260709 13:13:49.830070 28079 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:49.813592 28130 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:49.854007 22837 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:49.854522 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 77123.8us
I20260709 13:13:49.829809 28127 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:49.834350 27982 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.866575 28298 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.867331 28298 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:49.870939 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.871796 28298 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:49.873903 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48186.5us
I20260709 13:13:49.878458 22763 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: [12.11-15.12]
I20260709 13:13:49.881156 28127 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:49.881703 28127 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:49.882731 28127 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.884426 28154 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:49.886188 27982 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:49.892690 22760 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:49.897758 22757 pending_rounds.cc:85] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:13:49.898191 22757 pending_rounds.cc:107] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:13:49.898496 22757 raft_consensus.cc:2889] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:49.911048 28328 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:49.923588 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.925371 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.924306 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.982486 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:49.991111 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:49.991477 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 847) took 2089 ms. Trace:
I20260709 13:13:49.991905 22760 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:49.992133 22836 rpcz_store.cc:276] 0709 13:13:47.824075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.714078 (+890003us) service_pool.cc:224] Handling call
0709 13:13:48.714480 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.846356 (+131876us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:13:49.139440 (+293084us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.446243 (+306803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.446311 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:49.446382 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.766380 (+319998us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:49.913053 (+146673us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000c67d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:49.913119 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.913310 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53960,"prepare.queue_time_us":426177,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":60314,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":6540,"spinlock_wait_cycles":267574912,"thread_start_us":510448,"threads_started":2,"wal-append.queue_time_us":386139}
W20260709 13:13:49.995718 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 118428us
I20260709 13:13:49.999245 22914 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:50.001832 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.020607 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:50.024472 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:49.983817 28154 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:49.957655 28278 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:50.025261 22836 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:50.037973 22835 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:13:50.044087 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 180551us
I20260709 13:13:50.047987 22760 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:50.051164 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:50.052801 22763 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:50.057137 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:50.058001 22835 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:50.062458 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:50.054908 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:50.068768 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:50.059897 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:50.073088 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:50.074314 22840 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:50.073792 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 246685us
I20260709 13:13:50.077370 22834 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 15. Ops: []
I20260709 13:13:50.079735 23457 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:13:50.086261 23457 pending_rounds.cc:85] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 8
I20260709 13:13:50.086548 23457 pending_rounds.cc:107] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:13:50.086760 23457 raft_consensus.cc:2889] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:50.090265 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69779.1us
I20260709 13:13:50.093335 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:50.074203 28358 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:50.075955 28079 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:50.044479 28278 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:50.098356 28372 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:13:50.107059 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:50.110689 28079 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.118086 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:50.119334 22763 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:13:50.120478 27899 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.129341 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.130713 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.131696 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.132491 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3027) took 1115 ms. Trace:
I20260709 13:13:50.132752 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.149677 22908 rpcz_store.cc:276] 0709 13:13:49.017030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.045858 (+ 28828us) service_pool.cc:224] Handling call
0709 13:13:50.132196 (+1086338us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54003381d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:50.132416 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228541696}
I20260709 13:13:50.134339 28298 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.134531 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.137565 28331 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:50.141026 22836 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:50.157380 22764 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:50.158070 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:50.159075 22834 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:50.160723 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:50.162617 28333 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:50.169242 22841 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:50.170377 22758 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:50.171604 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.151034 28358 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:50.173550 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:50.175387 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:50.186869 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.199903 22906 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:50.158689 28298 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.210173 28333 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:50.230005 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56609.5us
W20260709 13:13:50.311877 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 80790.4us
I20260709 13:13:50.312975 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:50.200918 28358 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:50.205895 28305 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:50.346014 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:50.359838 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46978.4us
W20260709 13:13:50.366691 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 248786us
I20260709 13:13:50.368697 28344 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:50.350090 28122 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.380807 28122 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:50.382490 28374 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:13:50.385807 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:50.362499 28080 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:13:50.388008 22840 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: [13.11-14.12]
I20260709 13:13:50.391531 28219 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.395582 28298 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.396940 28305 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.351003 28036 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:50.209352 28331 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.358295 28358 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:50.400871 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:50.395624 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.420070 22841 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:13:50.421308 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:50.427472 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65519.5us
I20260709 13:13:50.414517 28344 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:50.430063 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 294957us
W20260709 13:13:50.441039 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72357.6us
I20260709 13:13:50.402318 28214 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:50.438927 22839 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:13:50.439853 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 332327us
I20260709 13:13:50.406044 28110 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:50.437808 28298 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.448604 28344 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.465204 28298 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.467504 28344 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.437947 28113 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:50.468571 28113 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:50.467687 27912 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:50.469165 28113 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:50.468101 28298 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:50.439378 28219 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:50.471836 28110 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.472260 28110 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:50.483193 28127 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:50.413568 28375 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:13:50.399652 28130 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:50.486943 28381 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:50.487534 28381 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:50.414256 28122 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.505214 22807 leader_election.cc:400] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:50.505784 22807 leader_election.cc:403] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:50.507300 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:50.514683 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:50.516125 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:50.353338 27982 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.516956 23465 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:50.526710 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.537669 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.400543 28036 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.564190 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.562513 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:50.561257 23457 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:13:50.562513 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:50.572022 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.565189 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:50.572221 22906 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:13:50.573169 23462 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:50.502909 28219 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:50.565729 28365 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:13:50.527366 28130 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:13:50.583413 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59681.1us
I20260709 13:13:50.585417 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2188) took 1337 ms. Trace:
I20260709 13:13:50.580896 27700 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.583102 27899 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.501928 28247 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:13:50.587327 22839 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55836
I20260709 13:13:50.607470 22839 raft_consensus.cc:606] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Received request to transfer leadership
I20260709 13:13:50.607849 22839 raft_consensus.cc:614] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:50.588196 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:50.591897 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:50.608403 22841 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:50.592417 22912 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:50.538906 28213 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:50.599869 22837 rpcz_store.cc:276] 0709 13:13:49.248372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.266254 (+ 17882us) service_pool.cc:224] Handling call
0709 13:13:49.266645 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.854465 (+587820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.585096 (+730631us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400038ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:50.585339 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350317440}
I20260709 13:13:50.489128 28381 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.604876 22879 leader_election.cc:400] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:50.611722 22879 leader_election.cc:403] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:50.612890 28282 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:50.612989 28094 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:50.563210 28122 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:50.591588 28310 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:50.613910 28122 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.615110 28094 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:50.616539 28122 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.581102 28278 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:50.613422 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:50.617753 23461 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:13:50.605382 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.025s	sys 0.051s
I20260709 13:13:50.613873 28310 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:50.618286 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.025s	sys 0.051s
W20260709 13:13:50.618724 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 525597us
I20260709 13:13:50.627979 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:50.649927 28331 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.628552 27899 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.652869 27899 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.655551 28036 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.659075 28310 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.645776 27700 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:50.662297 27700 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.662845 27700 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:50.663254 28219 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:50.665637 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52294us
I20260709 13:13:50.666831 28248 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:50.667347 28248 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:50.668372 28248 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.665143 28113 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.672546 28113 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:50.674541 27982 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:50.676749 28381 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:50.690497 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.690474 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:50.708371 22908 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:50.673326 28113 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:50.710232 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:50.736383 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 424789us
W20260709 13:13:50.743306 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.735474 22759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.001s	sys 0.018s
W20260709 13:13:50.745824 22759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.001s	sys 0.018s
I20260709 13:13:50.713600 28375 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:50.709007 28383 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:50.708761 28276 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:50.755834 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 144650us
I20260709 13:13:50.757548 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:50.751266 28248 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:50.716477 27982 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:13:50.807510 28333 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:50.736924 28113 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.756541 28375 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:50.755357 28391 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:13:50.818892 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 237494us
I20260709 13:13:50.776657 28358 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.842051 27700 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.850244 28193 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:50.818248 28375 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:50.820698 28282 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:50.785985 28276 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.856338 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:50.871888 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75879.3us
I20260709 13:13:50.843775 28130 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
W20260709 13:13:50.885782 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 321005us
W20260709 13:13:50.888891 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.829813 28333 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:50.893895 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:50.844655 28110 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:50.887007 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:50.867117 28282 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.888464 27912 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:50.925940 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 334125us
I20260709 13:13:50.927624 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:50.914717 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:50.891673 28246 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:50.894589 27899 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:50.936169 27912 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.936911 27899 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.935443 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:50.935551 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.935942 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:50.941273 27912 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:50.989717 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 62486.1us
I20260709 13:13:50.990989 22805 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:50.925056 28282 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:50.932754 28298 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:51.012822 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 260183us
I20260709 13:13:50.996454 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.004608 22912 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:50.941989 28036 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:50.994326 28154 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.968s	user 0.017s	sys 0.000s
I20260709 13:13:50.940757 27899 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:51.032501 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:51.035229 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 267583us
I20260709 13:13:51.037284 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.007411 28331 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:51.004868 28407 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:51.025108 28036 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.978785 28246 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:50.996440 28333 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.068769 27700 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:51.027582 28330 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:51.032848 28305 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:13:51.064970 28080 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:13:51.129390 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.025633 28373 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:50.942152 28278 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:13:51.024331 27912 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.013216 28310 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.148186 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 137812us
W20260709 13:13:51.153723 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 297921us
I20260709 13:13:51.203292 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:51.141880 28113 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.169955 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:51.200831 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:51.151917 28036 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:51.135210 28384 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:51.237145 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.158327 28330 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.157697 28407 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:51.164772 28193 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:51.236266 28403 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:51.248484 27899 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:51.249645 28423 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:51.249922 28246 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:51.250877 28110 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:51.252928 28214 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.750s	user 0.007s	sys 0.005s
I20260709 13:13:51.256340 28113 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:51.217790 28409 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.257900 28150 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:51.258494 28330 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:51.258665 28399 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:13:51.216765 28421 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.260813 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 864) took 1093 ms. Trace:
I20260709 13:13:51.400317 22836 rpcz_store.cc:276] 0709 13:13:50.167104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.231464 (+ 64360us) service_pool.cc:224] Handling call
0709 13:13:50.354679 (+123215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.406647 (+ 51968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.406697 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:50.432349 (+ 25652us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.873437 (+441088us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:50.873541 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.873655 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.260498 (+386843us) raft_consensus.cc:1685] finished
0709 13:13:51.260577 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.260747 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6825,"raft.queue_time_us":57581,"raft.run_cpu_time_us":708,"raft.run_wall_time_us":9318,"thread_start_us":394101,"threads_started":1,"wal-append.queue_time_us":669250}
W20260709 13:13:51.309603 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 105876us
I20260709 13:13:51.237603 28358 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:51.414461 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 196843us
I20260709 13:13:51.342816 22761 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:13:51.344458 23463 raft_consensus.cc:2410] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:13:51.263266 28416 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:51.370428 22763 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
W20260709 13:13:51.311321 28395 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.386s	user 0.001s	sys 0.001s
I20260709 13:13:51.220373 28310 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.264123 28193 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:51.391021 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:51.220095 27912 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.220507 28373 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:51.421878 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.260490 28219 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.143s
I20260709 13:13:51.416855 22761 pending_rounds.cc:85] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:13:51.442587 22761 pending_rounds.cc:107] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:13:51.442917 22761 raft_consensus.cc:2889] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:51.429859 22760 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:51.448143 22765 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:51.448462 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:51.417646 22763 pending_rounds.cc:85] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:13:51.451318 22763 pending_rounds.cc:107] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:13:51.451683 22763 raft_consensus.cc:2889] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:51.389518 28246 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:51.438761 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:51.433751 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:51.403877 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:51.458643 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:51.431041 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.462389 22759 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:51.311465 28384 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:51.441919 28397 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.417s	user 0.001s	sys 0.001s
I20260709 13:13:51.425678 28400 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.259721 28407 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.462869 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:51.514545 23461 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:51.511394 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 107632us
I20260709 13:13:51.345387 28403 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.447852 28358 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:51.519181 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:51.462931 28214 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:51.455714 28246 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:51.531942 28422 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.534379 28410 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
W20260709 13:13:51.535012 28402 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.385s	user 0.000s	sys 0.002s
I20260709 13:13:51.532021 28429 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:51.524192 28424 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.537832 28426 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:51.540596 28403 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.541887 28333 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:51.515146 28400 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.544658 28113 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:51.544853 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 861) took 1398 ms. Trace:
I20260709 13:13:51.515496 28150 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.545976 28358 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.547629 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.630806 22763 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:51.553733 28276 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.624486 28429 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:51.637815 28150 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.638903 28429 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.640331 28150 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.625584 28426 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:51.642035 28127 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.158s	user 0.022s	sys 0.000s
I20260709 13:13:51.642840 28425 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.643100 28426 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:51.562588 28310 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.564203 28373 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:51.463364 28416 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:13:51.565915 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46843us
W20260709 13:13:51.577099 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.414775 28423 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.620486 28113 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:51.570678 28214 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.421007 28421 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.572365 28246 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:51.440024 28409 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.639575 28358 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:51.557905 28248 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.760s	user 0.014s	sys 0.007s
I20260709 13:13:51.663729 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 811) took 1595 ms. Trace:
W20260709 13:13:51.668651 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 654) took 2476 ms (client timeout 2162 ms). Trace:
I20260709 13:13:51.680444 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.800228 22761 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:13:51.682416 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.686709 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:51.700590 28418 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 13:13:51.708232 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.724196 22906 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:51.724773 22915 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:51.725909 22914 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:51.726549 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:51.730784 28386 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.917s	user 0.000s	sys 0.016s
I20260709 13:13:51.734727 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:51.735821 28408 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.182s	user 0.005s	sys 0.000s
I20260709 13:13:51.737901 28079 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:51.742743 28396 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.678s	user 0.002s	sys 0.003s
I20260709 13:13:51.746557 28193 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.785929 28310 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.751991 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.643620 28425 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.759918 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.832834 22763 raft_consensus.cc:2368] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
W20260709 13:13:51.760335 28036 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.523s	user 0.014s	sys 0.000s
I20260709 13:13:51.620558 22835 rpcz_store.cc:276] 0709 13:13:50.146816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.146939 (+   123us) service_pool.cc:224] Handling call
0709 13:13:50.151073 (+  4134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.176698 (+ 25625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.176756 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:50.228541 (+ 51785us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.723742 (+495201us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:50.723973 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.724192 (+   219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.544484 (+820292us) raft_consensus.cc:1685] finished
0709 13:13:51.544522 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.544766 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12122,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":25136,"thread_start_us":337757,"threads_started":1,"wal-append.queue_time_us":517146}
I20260709 13:13:51.762149 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.565627 28414 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:51.642859 28127 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.762548 28402 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:13:51.781698 22833 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:51.784282 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:51.845803 22832 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:51.801162 22765 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:51.847118 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:51.796418 23463 rpcz_store.cc:269] 0709 13:13:49.192225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.195458 (+  3233us) service_pool.cc:224] Handling call
0709 13:13:51.668252 (+2472794us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:51.668517 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4938515840}
W20260709 13:13:51.804697 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.042s
W20260709 13:13:51.848168 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.000s	sys 0.043s
W20260709 13:13:51.848567 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 678738us
I20260709 13:13:51.620103 28424 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.779143 28113 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:51.811213 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:51.873688 22841 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:51.777791 28214 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:51.804991 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:51.794431 23460 rpcz_store.cc:276] 0709 13:13:50.068443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.068601 (+   158us) service_pool.cc:224] Handling call
0709 13:13:50.085716 (+ 17115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:50.156824 (+ 71108us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:50.162970 (+  6146us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.870126 (+707156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:50.870224 (+    98us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:50.870301 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.663111 (+792810us) spinlock_profiling.cc:243] Waited 70.8 ms on lock 0x7b5400375354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:51.663204 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.663433 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34131,"prepare.queue_time_us":6474,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":85935,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":98,"spinlock_wait_cycles":162860416,"thread_start_us":670750,"threads_started":2,"wal-append.queue_time_us":588950}
W20260709 13:13:51.831447 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.833429 23461 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:51.594072 28422 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.837231 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:51.883234 22760 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:51.843196 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:51.885272 22761 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:51.848740 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:51.820240 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:51.825193 28417 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.886685 22839 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:13:51.869724 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.886014 22764 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12. Re-sending same reply.
I20260709 13:13:51.862922 22765 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:51.860246 28282 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.832s	user 0.017s	sys 0.000s
I20260709 13:13:51.900614 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.901096 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:51.901782 22834 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:51.893787 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.888964 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.894827 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.899173 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:51.905694 22759 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:13:51.842865 27912 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.903823 22758 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:51.904533 22914 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:51.866737 28414 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:51.908948 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.920552 22915 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:51.875922 28193 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:51.843775 28281 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:51.908069 28375 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.032s	user 0.012s	sys 0.006s
I20260709 13:13:51.912676 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.936787 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.889534 28213 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.840348 28310 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.951959 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.831645 28403 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:13:51.941289 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:51.941915 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:51.845803 28036 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.834637 28079 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:51.946636 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:51.972534 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.973300 22834 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:51.953217 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.849856 28127 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.957840 28414 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:51.958879 22661 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 136) took 21689 ms (client timeout 15000 ms). Trace:
I20260709 13:13:51.959609 28422 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.907285 28416 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.951432 28213 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:51.960440 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:51.940933 28248 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:51.957216 28413 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.427s	user 0.003s	sys 0.001s
I20260709 13:13:51.965951 22760 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:51.886895 28417 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.979869 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:51.981658 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:51.984282 22839 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:51.984549 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.008s	sys 0.076s
I20260709 13:13:51.989637 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:51.989895 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.008s	sys 0.076s
W20260709 13:13:51.984316 22757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.060s	sys 0.054s
I20260709 13:13:51.905920 28282 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:51.995621 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.996392 22761 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:51.951604 28375 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.002761 22914 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:52.003393 28079 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.003834 28375 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.004935 28278 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:52.007819 28417 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.009996 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.997440 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.011462 22761 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:52.011360 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:52.012462 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.012665 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.013561 22761 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:52.014595 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.015902 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.020841 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.026882 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 749) took 2018 ms. Trace:
I20260709 13:13:52.028005 22911 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:52.027374 22762 rpcz_store.cc:276] 0709 13:13:50.008487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.013394 (+  4907us) service_pool.cc:224] Handling call
0709 13:13:50.036470 (+ 23076us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:50.074464 (+ 37994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.074501 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:13:50.100441 (+ 25940us) log.cc:844] Serialized 78 byte log entry
0709 13:13:50.490720 (+390279us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:50.565100 (+ 74380us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.565345 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.565420 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.729933 (+1164513us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:52.026312 (+296379us) raft_consensus.cc:1685] finished
0709 13:13:52.026632 (+   320us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.026787 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14429,"prepare.queue_time_us":41334,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":26858,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":11515,"spinlock_wait_cycles":6597376,"thread_start_us":354369,"threads_started":1,"wal-append.queue_time_us":605435}
I20260709 13:13:52.030368 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.031067 28310 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.022737 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 870) took 1443 ms. Trace:
I20260709 13:13:52.031667 22840 rpcz_store.cc:276] 0709 13:13:50.579798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.588726 (+  8928us) service_pool.cc:224] Handling call
0709 13:13:50.589140 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.606532 (+ 17392us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:50.606569 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:50.630538 (+ 23969us) log.cc:844] Serialized 26 byte log entry
0709 13:13:51.044338 (+413800us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:51.044429 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.044538 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.787647 (+743109us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:52.022382 (+234735us) raft_consensus.cc:1685] finished
0709 13:13:52.022418 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.022666 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3360,"raft.queue_time_us":15574,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":22222,"spinlock_wait_cycles":105792896,"thread_start_us":351816,"threads_started":1,"wal-append.queue_time_us":693242}
I20260709 13:13:52.033847 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:52.035708 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:52.037122 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.038627 28421 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:51.990424 22760 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:52.040472 28422 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.041062 22834 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:52.041290 28248 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:52.042407 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.044539 22839 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:52.046305 28409 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:52.048209 22879 connection.cc:256] RPC call timeout handler was delayed by -2.4240183699998852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 13:13:52.048730 28079 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:52.118153 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.119205 22834 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:13:52.121786 22765 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:52.051397 28036 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.053145 28247 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.053735 28424 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:52.058928 28381 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.381s	user 0.015s	sys 0.003s
I20260709 13:13:52.062064 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:52.063562 28330 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.001s
I20260709 13:13:51.958884 28423 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.063964 28431 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.031195 22759 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:51.960973 28281 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.027194 28094 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:52.039342 28384 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.104423 22832 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
W20260709 13:13:51.990926 22757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.060s	sys 0.055s
W20260709 13:13:51.984390 28412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.040s	sys 0.077s
W20260709 13:13:52.113282 22879 connection.cc:256] RPC call timeout handler was delayed by -2.3817183700000442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 13:13:52.123306 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45449.5us
I20260709 13:13:52.126339 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:51.991969 28434 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:51.982261 22661 rpcz_store.cc:269] 0709 13:13:30.198336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.198654 (+   318us) service_pool.cc:224] Handling call
0709 13:13:51.871508 (+21672854us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:46.963647 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:46.964480 (+   833us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:46.964547 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:13:46.964588 (+    41us) tablet.cc:662] Decoding operations
0709 13:13:46.999608 (+ 35020us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:46.999743 (+   135us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:46.999773 (+    30us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:13:47.003525 (+  3752us) tablet.cc:701] Row locks acquired
0709 13:13:47.003588 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:13:47.003912 (+   324us) write_op.cc:270] Start()
0709 13:13:47.004105 (+   193us) write_op.cc:276] Timestamp: P: 1783602827003791 usec, L: 0
0709 13:13:47.004140 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:13:47.005009 (+   869us) log.cc:844] Serialized 7378 byte log entry
0709 13:13:47.808505 (+803496us) op_driver.cc:464] REPLICATION: finished
0709 13:13:48.173235 (+364730us) write_op.cc:301] APPLY: starting
0709 13:13:48.176742 (+  3507us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:48.193243 (+ 16501us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:48.193348 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:49.243032 (+1049684us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:49.254362 (+ 11330us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:13:49.254468 (+   106us) write_op.cc:312] APPLY: finished
0709 13:13:49.255352 (+   884us) log.cc:844] Serialized 172 byte log entry
0709 13:13:49.546873 (+291521us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:49.635261 (+ 88388us) write_op.cc:454] Released schema lock
0709 13:13:49.635365 (+   104us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:49.636044 (+   679us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:49.636087 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2313488,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":317672,"apply.run_cpu_time_us":59632,"apply.run_wall_time_us":1474048,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":512917,"prepare.run_cpu_time_us":17512,"prepare.run_wall_time_us":148070,"raft.queue_time_us":389255,"raft.run_cpu_time_us":8525,"raft.run_wall_time_us":339730,"replication_time_us":740173,"thread_start_us":1018736,"threads_started":5,"wal-append.queue_time_us":393906,"wal-append.run_cpu_time_us":28858,"wal-append.run_wall_time_us":2505048}]]}
I20260709 13:13:52.160635 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:52.127465 28435 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.179s	user 0.001s	sys 0.001s
I20260709 13:13:52.122459 28333 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.168952 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:52.169562 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.006s	sys 0.013s
W20260709 13:13:52.170847 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.006s	sys 0.013s
I20260709 13:13:52.154306 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.162788 22761 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:52.173408 22765 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:52.126238 28383 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.175518 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.179675 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:52.164912 28412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.040s	sys 0.077s
I20260709 13:13:52.178083 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.157362 28094 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.182915 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.115358 28422 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.148964 28246 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.180151 22763 raft_consensus.cc:2368] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
I20260709 13:13:52.149870 28079 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.149700 28213 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.183755 22765 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:13:52.195305 22759 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:52.190303 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:52.195387 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:52.235577 22840 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:52.188205 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 158043us
W20260709 13:13:52.197127 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8049329330001456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:13:52.237043 22879 connection.cc:256] RPC call timeout handler was delayed by -1.4323329329999979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
I20260709 13:13:52.192113 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:52.237479 22879 connection.cc:256] RPC call timeout handler was delayed by -1.233432933000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
I20260709 13:13:52.162423 28330 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:52.235764 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52953us
I20260709 13:13:52.267287 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:52.270115 22765 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:13:52.270872 22765 pending_rounds.cc:85] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:13:52.271173 22765 pending_rounds.cc:107] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:13:52.271461 22765 raft_consensus.cc:2889] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:52.273223 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:52.275303 23463 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:13:52.276916 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.277608 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:52.277843 22841 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:52.260772 22832 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:52.280504 22832 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:52.261909 28298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.031s	sys 0.035s
I20260709 13:13:52.161975 28431 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.178157 28333 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.286268 23460 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:52.302083 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:52.302850 22879 leader_election.cc:400] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14.
I20260709 13:13:52.283106 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:52.284654 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:52.303216 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.303682 22840 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:52.190577 28080 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:52.307662 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 142529us
W20260709 13:13:52.310693 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.288159 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:52.288656 22836 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:52.292860 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:52.292721 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:52.267800 27912 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.241377 28094 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:52.298765 23466 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 16. Ops: []
I20260709 13:13:52.176407 28434 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.306049 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:52.303910 22841 raft_consensus.cc:2376] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16. Re-sending same reply.
I20260709 13:13:52.313383 22759 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:52.313820 22911 raft_consensus.cc:606] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.195794 28247 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.322945 23464 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:52.329357 23461 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:52.186980 28383 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.282472 28442 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.295861 28384 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.350198 22761 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:13:52.361608 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.375983 22732 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.512s (SENT)
I20260709 13:13:52.378142 22763 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:52.267664 28248 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.303123 28278 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:13:52.338778 28403 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:52.394572 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.402506 22762 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:52.398696 22758 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:52.321887 28441 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:13:52.377359 28421 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:52.328910 28080 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.342777 28442 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.333863 27912 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.344640 28384 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:52.424103 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 75008.4us
I20260709 13:13:52.370671 28424 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:13:52.379956 28214 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:52.429136 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53227.2us
W20260709 13:13:52.453456 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 150413us
W20260709 13:13:52.512019 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 223968us
I20260709 13:13:52.512807 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.512987 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.432809 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:52.427018 28246 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.525717 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.407337 28403 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:52.533771 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 101091us
I20260709 13:13:52.427115 28443 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:52.532855 22835 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:52.513912 28441 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.524343 28424 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:52.427556 28438 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:52.540729 28458 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:13:52.546816 28193 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:52.592300 28193 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.596634 28193 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.428146 28421 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:52.598493 28333 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.599491 28246 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:52.598673 28421 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.601832 28214 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.602753 28214 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:52.607717 28247 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.533922 28391 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:52.614888 28449 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
W20260709 13:13:52.616201 28454 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 13:13:52.617039 28403 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.533320 28407 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:52.587790 28080 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.589980 27912 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.563325 28441 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.525576 28429 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:13:52.556659 28443 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.571905 22835 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: [14.7-16.8]
I20260709 13:13:52.626114 28442 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.545091 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:52.635762 22759 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
W20260709 13:13:52.624112 28451 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:13:52.662304 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.668339 28384 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.673789 28281 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.678707 28443 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.627945 28330 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.679262 28434 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.680044 28422 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:52.680209 28431 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:52.640313 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87609.2us
I20260709 13:13:52.652299 28441 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.573865 28079 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.659273 28429 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:52.683192 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.698089 22759 raft_consensus.cc:2368] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:52.693933 22840 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
W20260709 13:13:52.709077 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 173913us
I20260709 13:13:52.713940 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:52.683382 28113 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.786s	user 0.014s	sys 0.000s
W20260709 13:13:52.719944 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.720880 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2292) took 1434 ms. Trace:
I20260709 13:13:52.724553 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.726218 22759 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:13:52.721443 22766 rpcz_store.cc:276] 0709 13:13:51.286822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.288051 (+  1229us) service_pool.cc:224] Handling call
0709 13:13:51.326643 (+ 38592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.802434 (+475791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:51.802491 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:51.802788 (+   297us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:51.802848 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.720496 (+917648us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b540035d754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:52.720574 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.720778 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093522688}
I20260709 13:13:52.708642 28442 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:52.708168 28443 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:52.697017 28429 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:52.732963 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.735421 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.736070 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.742326 22908 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:52.742707 22911 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:13:52.743340 22766 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:52.744176 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.744779 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:52.715572 28383 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:52.732661 28414 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.070s	sys 0.084s
I20260709 13:13:52.745990 22766 raft_consensus.cc:2376] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15. Re-sending same reply.
I20260709 13:13:52.736286 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.728808 28330 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:52.784258 22732 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.040s (SENT)
I20260709 13:13:52.754146 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.791540 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1919) took 1012 ms. Trace:
I20260709 13:13:52.794441 22835 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:52.783663 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2884) took 1022 ms. Trace:
I20260709 13:13:52.795854 22839 rpcz_store.cc:276] 0709 13:13:51.779456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.905416 (+125960us) service_pool.cc:224] Handling call
0709 13:13:52.791292 (+885876us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b54000ec754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:52.791448 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48598272}
I20260709 13:13:52.769233 22832 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:52.796618 22761 rpcz_store.cc:276] 0709 13:13:51.760929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.178081 (+417152us) service_pool.cc:224] Handling call
0709 13:13:52.783405 (+605324us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400393354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:52.783592 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242350848}
W20260709 13:13:52.798635 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 273016us
I20260709 13:13:52.718679 28113 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.795816 22759 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 9.
I20260709 13:13:52.796927 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.751533 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.805533 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.806324 22766 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:52.807754 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.809024 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.805160 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.801096 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.817116 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:52.802244 28441 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.807227 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:52.822275 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:52.823536 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:52.824430 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2865) took 1264 ms. Trace:
W20260709 13:13:52.768254 28463 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:13:52.830030 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.830018 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.825115 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:52.830907 22766 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:52.750841 28414 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.071s	sys 0.084s
I20260709 13:13:52.831014 22911 raft_consensus.cc:2376] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13. Re-sending same reply.
I20260709 13:13:52.831790 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.832175 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.832504 22766 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:52.800386 28433 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.254s	user 0.010s	sys 0.015s
I20260709 13:13:52.828763 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.828989 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:52.833971 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.834802 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.838274 22910 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:52.825017 22763 rpcz_store.cc:276] 0709 13:13:51.559862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.236262 (+676400us) service_pool.cc:224] Handling call
0709 13:13:52.285270 (+ 49008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.394749 (+109479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.824094 (+429345us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x7b5400311854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:52.824347 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190722816}
I20260709 13:13:52.841444 22915 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:52.843521 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1914) took 1070 ms. Trace:
I20260709 13:13:52.844040 22836 rpcz_store.cc:276] 0709 13:13:51.773272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.785727 (+ 12455us) service_pool.cc:224] Handling call
0709 13:13:51.811642 (+ 25915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.313221 (+501579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.843244 (+530023us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54001bc954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:52.843448 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096506240}
W20260709 13:13:52.853062 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68846.4us
I20260709 13:13:52.853996 22764 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:52.785262 28127 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.853120 22906 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:52.842762 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:52.856971 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:52.759063 28333 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.802533 28330 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.833868 28441 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.745134 28443 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:52.860149 28425 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:52.853708 28417 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:52.861820 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.940877 22832 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:52.862924 22906 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:13:52.863461 28113 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.864183 28403 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:52.865554 28219 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:52.846506 28193 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:52.753780 28429 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:52.838222 28344 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:13:52.846932 28433 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.010s	sys 0.015s
I20260709 13:13:52.867020 22759 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:52.966815 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 756) took 1146 ms. Trace:
I20260709 13:13:52.867961 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2858) took 1452 ms. Trace:
W20260709 13:13:52.868870 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67888.7us
I20260709 13:13:52.970413 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:52.871950 28094 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:13:52.872498 28465 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:52.877978 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:52.980365 23462 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:52.980511 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:52.879176 23457 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:52.879769 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:52.881759 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.982026 22911 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:52.889213 28401 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:52.892685 28400 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.894649 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2880) took 1135 ms. Trace:
I20260709 13:13:52.991430 22758 rpcz_store.cc:276] 0709 13:13:51.759699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.935604 (+175905us) service_pool.cc:224] Handling call
0709 13:13:52.894355 (+958751us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b5400221ad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:52.894554 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025521408}
I20260709 13:13:52.895609 28426 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:52.897365 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:52.902335 28466 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.236s	user 0.003s	sys 0.000s
I20260709 13:13:52.910597 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.910893 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:52.913288 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.010474 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.914494 22762 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:52.915607 28276 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:52.916564 28416 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:13:52.917739 22838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.018s	sys 0.026s
W20260709 13:13:52.927883 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.929862 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2869) took 1368 ms. Trace:
W20260709 13:13:53.024973 22838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.019s	sys 0.027s
I20260709 13:13:52.931213 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:52.932240 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73618.3us
I20260709 13:13:52.965873 28403 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:52.934700 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.944387 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:52.944577 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:52.946750 22766 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:13:52.933568 28448 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:52.946955 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:52.980221 28465 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:52.967444 22759 rpcz_store.cc:276] 0709 13:13:51.820555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.866304 (+1045749us) service_pool.cc:224] Handling call
0709 13:13:52.866650 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.966414 (+ 99764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.966730 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:52.978575 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:52.978953 22732 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:52.979708 23465 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:13:52.878203 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:52.985292 22761 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 13: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:52.946502 28417 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:53.038494 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:53.039556 28417 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.039915 28473 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:53.041785 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:53.024338 28429 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.042513 28441 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:52.965214 28344 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.983124 28094 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.045744 28344 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:53.046203 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67360.6us
I20260709 13:13:53.047983 28094 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.012315 22911 raft_consensus.cc:2368] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:13:53.050379 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:52.946153 28193 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:53.020167 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.022682 22913 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:53.021176 28276 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:52.934376 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.061599 22908 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:53.025944 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1921) took 1245 ms. Trace:
I20260709 13:13:52.943220 28333 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.026789 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.025163 22765 rpcz_store.cc:276] 0709 13:13:51.561379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.281664 (+720285us) service_pool.cc:224] Handling call
0709 13:13:52.286014 (+  4350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.286491 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.286523 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.304652 (+ 18129us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.454325 (+149673us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:52.929139 (+474814us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.929438 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.929556 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.929602 (+    46us) raft_consensus.cc:1685] finished
0709 13:13:52.929614 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.929790 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84930,"prepare.queue_time_us":436180,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":2136,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":7863,"spinlock_wait_cycles":426752,"thread_start_us":470628,"threads_started":2,"wal-append.queue_time_us":271526}
I20260709 13:13:53.031298 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.070142 22914 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:13:53.032444 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:52.983966 28219 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:53.086169 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2862) took 1540 ms. Trace:
I20260709 13:13:53.086849 28344 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.087639 28282 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:53.086740 22760 rpcz_store.cc:276] 0709 13:13:51.545678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.191208 (+645530us) service_pool.cc:224] Handling call
0709 13:13:52.226105 (+ 34897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.226639 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.226678 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.235634 (+  8956us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.443200 (+207566us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:52.734809 (+291609us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.736938 (+  2129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.737052 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.901644 (+164592us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:53.085732 (+184088us) raft_consensus.cc:1685] finished
0709 13:13:53.085876 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.086085 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":447091,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":5961,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":48129,"spinlock_wait_cycles":36289920,"thread_start_us":337399,"threads_started":2,"wal-append.queue_time_us":383884}
I20260709 13:13:53.089802 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:53.077127 23462 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:52.972370 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:53.091377 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.037197 23465 pending_rounds.cc:85] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 11
I20260709 13:13:53.091769 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 755) took 1271 ms. Trace:
I20260709 13:13:53.092060 23465 pending_rounds.cc:107] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:13:53.092434 23465 raft_consensus.cc:2889] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:53.092257 22763 rpcz_store.cc:276] 0709 13:13:51.820219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.856732 (+1036513us) service_pool.cc:224] Handling call
0709 13:13:52.857144 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.857408 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:13:52.891511 (+ 34103us) log.cc:844] Serialized 14 byte log entry
0709 13:13:53.091387 (+199876us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.091451 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:53.091516 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.091590 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.091709 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33503,"prepare.queue_time_us":34465,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":199357,"threads_started":1}
I20260709 13:13:53.049055 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.094664 22833 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 11: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:52.968400 22757 rpcz_store.cc:276] 0709 13:13:51.415516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.415773 (+   257us) service_pool.cc:224] Handling call
0709 13:13:51.416237 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.168051 (+751814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.168096 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.189985 (+ 21889us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.384228 (+194243us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:52.681013 (+296785us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.695280 (+ 14267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.695407 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.867706 (+172299us) raft_consensus.cc:1685] finished
0709 13:13:52.867732 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.867888 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26786,"prepare.queue_time_us":505665,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":139152,"raft.run_cpu_time_us":513,"raft.run_wall_time_us":10347,"thread_start_us":1164769,"threads_started":3,"wal-append.queue_time_us":224620}
I20260709 13:13:53.054889 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.103889 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.104723 22836 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
W20260709 13:13:53.025987 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45565.1us
I20260709 13:13:53.057771 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.058583 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 757) took 1237 ms. Trace:
I20260709 13:13:53.027630 28416 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:53.026007 22839 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:53.071343 22907 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.034543 28427 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.025817 28426 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.079403 28330 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:53.057246 28193 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.033923 28472 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:53.085134 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.127858 22908 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:53.030123 23460 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:13:53.128237 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.051924 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.095850 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2301) took 1111 ms. Trace:
I20260709 13:13:53.096200 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.067180 28414 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:53.129276 22911 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:13:53.143503 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:53.146636 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 321616us
I20260709 13:13:53.148511 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.106334 22758 rpcz_store.cc:276] 0709 13:13:51.820841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.035097 (+1214256us) service_pool.cc:224] Handling call
0709 13:13:53.057795 (+ 22698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.058053 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.058095 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.058159 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:53.058209 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.058293 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.058508 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:53.107250 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.150161 22836 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:53.151051 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.151273 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2206) took 1170 ms. Trace:
I20260709 13:13:53.116825 22914 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:53.083786 28470 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:53.120393 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.052567 28465 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.162173 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.161921 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.115619 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.089202 28344 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.129843 22832 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:53.179838 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.106117 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.129390 22763 rpcz_store.cc:276] 0709 13:13:51.983598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.094217 (+1110619us) service_pool.cc:224] Handling call
0709 13:13:53.094600 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.094792 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.094819 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.094935 (+   116us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:53.094973 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.095059 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.095174 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:53.130803 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:53.134141 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.181448 22914 raft_consensus.cc:2368] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13.
I20260709 13:13:53.151636 22835 rpcz_store.cc:276] 0709 13:13:51.980719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.646530 (+665811us) service_pool.cc:224] Handling call
0709 13:13:52.794135 (+147605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.799096 (+  4961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.151056 (+351960us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000ec750. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:53.151217 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323717376}
I20260709 13:13:53.116432 28213 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.190593 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.171517 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.149145 22911 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:53.117563 28441 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:53.189031 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.192133 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:53.169302 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53796.2us
I20260709 13:13:53.178069 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:53.189865 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83866.1us
I20260709 13:13:53.195868 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:53.199553 22907 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:53.199647 22758 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:53.123389 28427 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.198858 28219 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:53.216289 28424 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:53.216405 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.216570 28427 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.217684 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.197292 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:53.200682 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2895) took 1203 ms. Trace:
I20260709 13:13:53.218497 22758 rpcz_store.cc:276] 0709 13:13:51.996950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.149982 (+1153032us) service_pool.cc:224] Handling call
0709 13:13:53.191466 (+ 41484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.200428 (+  8962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.200624 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:53.219125 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.219950 22914 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:53.225777 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2885) took 1464 ms. Trace:
I20260709 13:13:53.226394 22762 rpcz_store.cc:276] 0709 13:13:51.761229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.407902 (+646673us) service_pool.cc:224] Handling call
0709 13:13:52.914060 (+506158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.014276 (+100216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.225474 (+211198us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400221ad0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:53.225694 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131054848}
I20260709 13:13:53.220127 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.203488 22908 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:13:53.205549 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.234481 22839 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 10.
I20260709 13:13:53.206722 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:53.207257 23460 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:53.211781 28443 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.178501 28431 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:13:53.189471 28094 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.164469 28276 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.247033 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.214061 28472 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.232842 23466 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:53.207880 28426 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.221117 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:53.252813 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.237416 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.238513 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.253981 22911 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.255296 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.247669 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:53.255919 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.213752 28465 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.242033 23460 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:13:53.257755 23464 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:53.257826 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.258009 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:53.258432 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.245899 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.256582 22758 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15. Re-sending same reply.
I20260709 13:13:53.270622 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.268942 22907 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:53.268457 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.274837 22910 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:13:53.275589 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:53.276244 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:53.269481 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.278656 22835 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:13:53.279707 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2214) took 1269 ms. Trace:
I20260709 13:13:53.279986 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.193249 28376 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 13:13:53.289147 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 91957.3us
I20260709 13:13:53.290136 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:53.252856 28330 raft_consensus.cc:2764] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:13:53.233276 28414 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:53.274144 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.262918 28404 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:53.280031 22835 rpcz_store.cc:276] 0709 13:13:52.010017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.253548 (+1243531us) service_pool.cc:224] Handling call
0709 13:13:53.278373 (+ 24825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:53.279450 (+  1077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.279652 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:53.292405 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.250191 28094 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.229444 28424 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:53.208923 28416 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.254513 28472 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:53.168275 28403 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.301893 28401 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:53.303357 28036 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:53.303841 28213 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:53.339982 22835 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:53.340658 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2216) took 1330 ms. Trace:
I20260709 13:13:53.340979 22835 rpcz_store.cc:276] 0709 13:13:52.010611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.313547 (+1302936us) service_pool.cc:224] Handling call
0709 13:13:53.339787 (+ 26240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.340418 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.340600 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:53.305042 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2215) took 1294 ms. Trace:
I20260709 13:13:53.342345 22836 rpcz_store.cc:276] 0709 13:13:52.010325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.254038 (+1243713us) service_pool.cc:224] Handling call
0709 13:13:53.279360 (+ 25322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.304581 (+ 25221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.304612 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.304664 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:53.304697 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.304768 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.304878 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:53.305543 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2875) took 1718 ms (client timeout 2079 ms). Trace:
I20260709 13:13:53.304930 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1923) took 1511 ms. Trace:
I20260709 13:13:53.343952 22837 rpcz_store.cc:276] 0709 13:13:51.793226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.422209 (+628983us) service_pool.cc:224] Handling call
0709 13:13:52.527271 (+105062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.714319 (+187048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.714353 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.740366 (+ 26013us) log.cc:844] Serialized 26 byte log entry
0709 13:13:53.143387 (+403021us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:53.143440 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.143513 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:53.304621 (+161108us) raft_consensus.cc:1685] finished
0709 13:13:53.304655 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.304856 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3477,"raft.queue_time_us":14988,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":9560,"thread_start_us":562007,"threads_started":2,"wal-append.queue_time_us":505479}
I20260709 13:13:53.309923 28248 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:53.343806 22764 rpcz_store.cc:269] 0709 13:13:51.564251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.900290 (+336039us) service_pool.cc:224] Handling call
0709 13:13:53.282102 (+1381812us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540035e654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:53.282242 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2172576640}
I20260709 13:13:53.347059 22837 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:53.306689 22806 leader_election.cc:400] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
I20260709 13:13:53.306852 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2209) took 1305 ms. Trace:
I20260709 13:13:53.246677 28441 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:53.282517 22759 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:53.311378 23460 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:53.312806 22910 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:13:53.293467 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.324148 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.329589 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.359309 22913 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:53.321705 28376 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:53.331367 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.365882 22758 raft_consensus.cc:2376] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13. Re-sending same reply.
I20260709 13:13:53.273108 28448 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:53.336254 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2212) took 1333 ms. Trace:
I20260709 13:13:53.227588 28425 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:13:53.330637 22908 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:53.347661 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.336387 28094 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.339395 28213 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.366897 22841 rpcz_store.cc:276] 0709 13:13:52.002679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.772848 (+770169us) service_pool.cc:224] Handling call
0709 13:13:52.773324 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.773658 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:53.191293 (+417635us) log.cc:844] Serialized 14 byte log entry
0709 13:13:53.335883 (+144590us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.335941 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:53.335997 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.336068 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.336190 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":495588,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":1256,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":12006,"thread_start_us":561391,"threads_started":2}
I20260709 13:13:53.348285 22806 leader_election.cc:403] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:53.328150 28344 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:53.378362 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.379799 22915 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:13:53.357649 28405 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.806s	user 0.024s	sys 0.024s
I20260709 13:13:53.310509 28426 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:13:53.332628 28472 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.356070 22911 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:53.321365 28443 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:13:53.374835 22837 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:13:53.337199 28036 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:53.377907 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 879) took 1048 ms. Trace:
I20260709 13:13:53.392290 22835 rpcz_store.cc:276] 0709 13:13:52.329567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.358354 (+1028787us) service_pool.cc:224] Handling call
0709 13:13:53.377160 (+ 18806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.377412 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.377449 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.377541 (+    92us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:53.377588 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.377660 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.377807 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:53.380038 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.394037 22910 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.362690 22840 rpcz_store.cc:276] 0709 13:13:52.001535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.702271 (+700736us) service_pool.cc:224] Handling call
0709 13:13:52.702835 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.716922 (+ 14087us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:52.740070 (+ 23148us) log.cc:844] Serialized 14 byte log entry
0709 13:13:53.171499 (+431429us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:53.190827 (+ 19328us) log.cc:844] Serialized 26 byte log entry
0709 13:13:53.190990 (+   163us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:53.191034 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.191105 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:53.306625 (+115520us) raft_consensus.cc:1685] finished
0709 13:13:53.306648 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.306793 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18679,"raft.queue_time_us":774,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":34644,"spinlock_wait_cycles":10003200,"thread_start_us":414996,"threads_started":1,"wal-append.queue_time_us":544811}
I20260709 13:13:53.347618 28442 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.321602 28414 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:53.385530 28405 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.831s	user 0.024s	sys 0.025s
I20260709 13:13:53.383636 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.358898 28248 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.349676 22906 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 10.
I20260709 13:13:53.407573 22913 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:53.408414 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:53.327913 28431 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:13:53.386672 22907 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:53.365520 28376 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:53.419107 28376 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.420938 28465 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:53.382793 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.406394 28442 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.376343 28213 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:53.310343 28484 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:13:53.411936 28414 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.413789 28424 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.414705 28441 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.415874 28401 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:53.417326 22913 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:13:53.422889 22908 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:53.406065 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.450670 22915 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:53.410399 28248 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:53.434813 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 141889us
I20260709 13:13:53.396755 28036 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:53.388186 28330 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14.
I20260709 13:13:53.385331 28344 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.434569 27899 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.465574 22914 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:13:53.466586 22765 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:53.472484 22910 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:53.475697 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2299) took 1497 ms. Trace:
W20260709 13:13:53.498696 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 207138us
I20260709 13:13:53.500113 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.444948 28421 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:13:53.482324 22761 rpcz_store.cc:276] 0709 13:13:51.977940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.037978 (+1060038us) service_pool.cc:224] Handling call
0709 13:13:53.038349 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.038600 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:53.121365 (+ 82765us) log.cc:844] Serialized 14 byte log entry
0709 13:13:53.475207 (+353842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.475288 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:53.475363 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.475458 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.475619 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42679,"prepare.queue_time_us":83078,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":3705,"raft.queue_time_us":141992,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":3736,"thread_start_us":324395,"threads_started":2,"wal-append.queue_time_us":272253}
I20260709 13:13:53.506820 22806 leader_election.cc:400] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:13:53.508663 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:53.512687 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 139442us
I20260709 13:13:53.516614 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:53.467774 27899 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.522780 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:53.522966 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99646.9us
I20260709 13:13:53.524431 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:53.495123 28330 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:13:53.545218 22806 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:53.522225 28298 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:53.521011 28473 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:53.553153 28488 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:13:53.555986 28487 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:13:53.557699 28490 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:13:53.513753 28094 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.544667 28330 consensus_queue.cc:1243] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:13:53.578686 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:53.597270 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.601855 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:53.602816 22763 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.524428 28493 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:13:53.561981 28494 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:53.610564 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.591336 28506 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:53.614904 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:53.625423 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.648841 28080 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:53.618930 28494 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:53.649614 28094 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.650065 28494 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.651386 27899 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.651998 27899 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:53.652336 27899 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.652676 27899 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:53.659173 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 114100us
I20260709 13:13:53.595804 28442 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:53.668452 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.662699 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:53.615334 28248 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.678644 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:53.680459 28496 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:13:53.630030 28278 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:53.667090 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.696074 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.627712 28509 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:53.574226 28504 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.108s
W20260709 13:13:53.707108 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44530.1us
I20260709 13:13:53.631186 28473 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:13:53.649628 28080 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:53.714478 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 178505us
I20260709 13:13:53.689393 28278 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:53.708503 28500 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.157s	user 0.004s	sys 0.001s
I20260709 13:13:53.705044 28442 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:53.723070 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.711578 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:53.708316 28421 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:53.723093 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.708127 28429 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:53.704342 28344 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:53.730788 22761 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:53.724534 28433 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:13:53.731741 28248 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.732973 28278 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.748076 22763 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:53.724615 28127 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:13:53.766912 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44046us
W20260709 13:13:53.775624 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64124.2us
I20260709 13:13:53.776783 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:53.777755 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47758.7us
I20260709 13:13:53.724705 28425 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:13:53.783154 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:53.750078 28193 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.606s	user 0.014s	sys 0.001s
I20260709 13:13:53.781170 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.802667 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2213) took 1799 ms. Trace:
I20260709 13:13:53.804348 28079 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:53.787159 28248 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.754758 28429 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:53.807611 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:53.824880 22839 rpcz_store.cc:276] 0709 13:13:52.002942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.250704 (+1247762us) service_pool.cc:224] Handling call
0709 13:13:53.270144 (+ 19440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:53.280697 (+ 10553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.280731 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:53.307880 (+ 27149us) log.cc:844] Serialized 52 byte log entry
0709 13:13:53.445454 (+137574us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:53.445516 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.445589 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:53.802358 (+356769us) raft_consensus.cc:1685] finished
0709 13:13:53.802384 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.802583 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5792,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":21414,"thread_start_us":97942,"threads_started":1,"wal-append.queue_time_us":177801}
I20260709 13:13:53.827975 22836 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.763103 28421 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:53.812412 28376 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.813275 28494 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.840140 27899 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.786394 28514 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:13:53.844689 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.835400 28442 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:13:53.845561 22764 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:53.846856 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.831875 28193 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.802930 28278 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.835597 28473 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:53.850600 28193 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:53.787323 28509 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:53.822956 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:53.852412 28509 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:53.852644 22834 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.852910 28509 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.834344 28429 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:53.853852 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.826464 22763 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:53.803790 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:53.863427 22763 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:53.864089 22761 raft_consensus.cc:2410] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:53.866165 22764 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:13:53.829219 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.830807 28080 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:53.869122 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:13:53.786592 28513 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.060s	user 0.000s	sys 0.003s
I20260709 13:13:53.836756 23459 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:53.827244 28079 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:53.823601 28495 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:53.833233 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.858362 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:53.785807 28433 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:53.828765 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.869716 28421 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:53.823851 28036 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:53.879359 22836 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:53.879518 22834 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
W20260709 13:13:53.880505 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47392.8us
I20260709 13:13:53.882557 22910 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:53.892138 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.893138 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.876861 28473 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.894403 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:53.800881 28213 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:53.898307 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 71070.3us
I20260709 13:13:53.881975 28080 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:53.881831 28504 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.877123 28495 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:53.905745 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.906200 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.881690 28433 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:53.908607 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:53.917819 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.930724 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.901856 28473 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.940742 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:53.904961 28510 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:53.913379 28504 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.928100 28080 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:13:53.906769 28405 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:53.936546 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52770.9us
I20260709 13:13:53.950608 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.907044 28425 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:53.907323 28431 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:53.931088 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:53.926801 28127 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.922147 28441 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:53.927434 28433 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.913595 28494 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:53.948666 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 177239us
I20260709 13:13:53.952054 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:55848 (request call id 1) took 1767 ms. Trace:
I20260709 13:13:53.945729 28510 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.958402 28080 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:53.944239 28079 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:53.944777 28473 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.935715 28330 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:53.972604 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2300) took 1991 ms. Trace:
I20260709 13:13:53.996791 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:13:53.996235 22837 rpcz_store.cc:276] 0709 13:13:52.176064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.912163 (+1736099us) service_pool.cc:224] Handling call
0709 13:13:53.933022 (+ 20859us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:53.977123 22730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.514s (SENT) [suppressed 34 similar messages]
I20260709 13:13:54.026180 22765 rpcz_store.cc:276] 0709 13:13:51.980878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.069550 (+1088672us) service_pool.cc:224] Handling call
0709 13:13:53.069939 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.467010 (+397071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.971951 (+504941us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b540037f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:53.972395 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911667456}
I20260709 13:13:53.968264 28431 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:53.969815 28405 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:53.974876 28441 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:53.965377 28333 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:53.968164 27899 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:54.006778 28080 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:54.045727 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.009095 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:54.047343 22915 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:54.011291 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:54.013566 28518 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:53.949165 28504 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.977331 28517 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:54.024983 22907 pending_rounds.cc:85] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:13:54.000905 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:53.952848 28383 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:54.047376 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:53.974495 28127 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.997943 28510 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:54.058204 22907 pending_rounds.cc:107] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 12.7
W20260709 13:13:54.060148 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64610.5us
W20260709 13:13:54.034583 28512 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.216s	user 0.000s	sys 0.002s
I20260709 13:13:54.064620 22907 raft_consensus.cc:2889] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:54.018472 28248 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:54.037025 28431 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:54.070827 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74159.9us
W20260709 13:13:54.063684 28472 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.660s	user 0.014s	sys 0.007s
I20260709 13:13:54.081024 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.058717 28518 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.085748 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.081554 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:54.046870 28080 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.060851 28079 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:54.088538 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.088147 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.061431 28213 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:54.065689 28127 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.082109 28425 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:54.059504 28383 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:54.105084 22836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
W20260709 13:13:54.098546 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.099205 22833 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:54.096503 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.138754 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:54.147718 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:54.148396 22837 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:54.149734 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 209108us
I20260709 13:13:54.102770 28080 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:54.151975 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:54.100353 28376 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:54.091162 28518 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:54.155251 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:54.175920 22765 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:13:54.176422 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:54.146613 28036 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:54.129869 22836 raft_consensus.cc:606] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
W20260709 13:13:54.156689 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.167310 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:54.182181 22757 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:54.184818 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:54.185776 22763 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:54.188743 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.189105 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:54.191371 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 887) took 1077 ms. Trace:
I20260709 13:13:54.191946 22835 rpcz_store.cc:276] 0709 13:13:53.113806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.402716 (+288910us) service_pool.cc:224] Handling call
0709 13:13:53.439133 (+ 36417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:53.547554 (+108421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.547610 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:53.550892 (+  3282us) log.cc:844] Serialized 26 byte log entry
0709 13:13:53.688394 (+137502us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:53.688434 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.688516 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.191051 (+502535us) raft_consensus.cc:1685] finished
0709 13:13:54.191091 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.191290 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61636,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8228,"thread_start_us":242506,"threads_started":2,"wal-append.queue_time_us":183647}
I20260709 13:13:54.200184 22834 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:54.201041 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:54.202539 22763 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
W20260709 13:13:54.203862 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.216377 22837 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:13:54.095170 28472 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:54.153067 28330 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:54.181427 28213 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:54.225004 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:54.190255 28494 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:54.189157 28401 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:53.062610 22838 rpcz_store.cc:276] 0709 13:13:51.780128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.914522 (+134394us) service_pool.cc:224] Handling call
0709 13:13:51.952381 (+ 37859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.952664 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:52.257004 (+304340us) log.cc:844] Serialized 14 byte log entry
0709 13:13:53.025537 (+768533us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.025602 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:53.025656 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.025729 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:53.025870 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":414882,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":2065,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5591,"spinlock_wait_cycles":39654400,"thread_start_us":1050910,"threads_started":2,"wal-append.queue_time_us":768039}
I20260709 13:13:54.221640 28219 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:54.238431 28219 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.238873 28219 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.244889 22841 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
W20260709 13:13:54.250263 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.222741 28376 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:54.259225 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 81543.1us
I20260709 13:13:54.094048 28441 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:54.244307 28401 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:54.194000 28405 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:54.280742 28494 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:54.282100 28376 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:54.326602 28472 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:54.310174 28401 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:54.346302 22712 auto_rebalancer.cc:893] Removing replace marker failed: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:13:53.998711 28495 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:54.279177 28515 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:54.374312 22712 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:13:54.375128 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 398223us
W20260709 13:13:54.394351 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.277729 28429 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:54.406369 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 249379us
I20260709 13:13:54.353271 28532 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:54.376415 28376 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:54.412273 28333 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:13:54.289389 28531 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:13:54.415553 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:54.383846 22712 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:13:54.378376 28530 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:54.430596 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:54.442363 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 363466us
I20260709 13:13:54.432057 28532 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:54.443291 28509 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.584s	user 0.016s	sys 0.000s
I20260709 13:13:54.427939 28495 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.486313 28509 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:54.488881 28509 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:54.428020 28519 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:54.440341 28520 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
I20260709 13:13:54.489651 28519 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:54.465359 28530 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:54.297224 28425 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:54.401937 28330 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:54.496161 28523 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
W20260709 13:13:54.526935 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111555us
W20260709 13:13:54.531149 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 116967us
W20260709 13:13:54.604406 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.579043 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 111233us
W20260709 13:13:54.580977 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.603729 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:13:54.614279 23431 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.003s	sys 0.011s
W20260709 13:13:54.616993 23431 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.003s	sys 0.011s
W20260709 13:13:54.618259 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 544118us
I20260709 13:13:54.519568 28494 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:54.638624 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2897) took 2090 ms. Trace:
I20260709 13:13:54.551826 28525 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:13:54.639691 22766 rpcz_store.cc:276] 0709 13:13:52.548268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.833615 (+285347us) service_pool.cc:224] Handling call
0709 13:13:54.634891 (+1801276us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54002224d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:54.638542 (+  3651us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242150144}
I20260709 13:13:54.601008 28537 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:54.629611 28522 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.347s	user 0.000s	sys 0.002s
I20260709 13:13:54.641816 28333 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:13:54.682605 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2221) took 1731 ms. Trace:
W20260709 13:13:54.679270 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.686415 22841 rpcz_store.cc:276] 0709 13:13:52.951288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.373342 (+422054us) service_pool.cc:224] Handling call
0709 13:13:53.389365 (+ 16023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.278647 (+889282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.682326 (+403679us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54000a5354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:54.682533 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966890880}
W20260709 13:13:54.664216 28533 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
W20260709 13:13:54.696774 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 92732.8us
I20260709 13:13:54.673440 28384 raft_consensus.cc:993] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:54.658123 28537 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:54.729816 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 105037us
W20260709 13:13:54.725773 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 122123us
I20260709 13:13:54.685549 28544 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:54.685776 28429 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:54.677822 28530 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:54.736776 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.673686 28524 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.318s	user 0.000s	sys 0.002s
I20260709 13:13:54.706561 28546 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:54.701701 28531 consensus_queue.cc:1243] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:54.676432 28525 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:13:54.752583 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:54.783370 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:54.783978 28248 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:54.784842 28472 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:54.657898 28543 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 13:13:54.782235 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6024768049999238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28730000000000006s for a response
I20260709 13:13:54.800740 28515 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:54.801436 28553 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.806512 28333 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:13:54.807412 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.863s (SENT)
W20260709 13:13:54.877912 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70535.9us
W20260709 13:13:54.779954 28421 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.878s	user 0.012s	sys 0.004s
W20260709 13:13:54.862730 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3937514920000922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 13:13:54.884114 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3901514920000864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:13:54.884802 22732 connection.cc:256] RPC call timeout handler was delayed by -1.3793514920000689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:13:54.885382 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2902514919999248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 13:13:54.744299 28384 raft_consensus.cc:1081] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:54.735678 28537 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.744817 28544 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:54.869050 28519 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:54.872541 28545 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:54.889142 28545 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:54.890450 28545 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:54.892396 28545 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:54.857471 28509 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:54.749742 28532 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:54.882651 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63772.8us
W20260709 13:13:54.860206 28127 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.708s	user 0.015s	sys 0.000s
W20260709 13:13:54.917572 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 165010us
W20260709 13:13:54.856891 28080 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.668s	user 0.003s	sys 0.011s
I20260709 13:13:54.778967 28546 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:54.906662 28544 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:54.934257 28433 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.958s	user 0.010s	sys 0.004s
W20260709 13:13:54.928308 28219 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.006s
I20260709 13:13:54.921478 28429 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:13:54.983282 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.860194 28401 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:54.981217 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 58939us
I20260709 13:13:54.979501 28425 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:13:54.921048 28421 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:54.931452 28127 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.002059 28554 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:55.002386 28518 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.005s
W20260709 13:13:55.004388 28504 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.923s	user 0.008s	sys 0.008s
W20260709 13:13:54.977421 28473 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.930s	user 0.013s	sys 0.000s
W20260709 13:13:55.005631 28495 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.519s	user 0.015s	sys 0.000s
W20260709 13:13:55.040685 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.064172 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.066444 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.196712 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.119884 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:55.000566 28510 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.919s	user 0.017s	sys 0.000s
I20260709 13:13:54.932511 28546 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:55.145689 28376 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
I20260709 13:13:54.882207 28553 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:54.931707 28080 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.179526 28544 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:55.203953 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 84195.3us
I20260709 13:13:55.206369 22913 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:55.222965 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.006346 28519 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.201s
W20260709 13:13:55.217850 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.011792 28219 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.224550 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:55.226327 28563 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:55.226748 28429 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:55.229519 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.231000 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:55.024020 28433 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.215849 28425 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:55.193748 28554 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.202397 28473 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.231424 28519 consensus_queue.cc:1243] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:13:55.247910 22906 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.245442 28567 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:55.191358 28504 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.228555 28405 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:55.233402 28384 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:55.220144 28376 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.193308 28518 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.235539 28543 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:55.210651 28510 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:55.277884 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.018s	sys 0.010s
W20260709 13:13:55.278286 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.019s	sys 0.010s
W20260709 13:13:55.278682 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 571500us
I20260709 13:13:55.247906 28429 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.246950 28333 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.286641 22840 raft_consensus.cc:2368] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
W20260709 13:13:55.277747 28431 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.193s	user 0.015s	sys 0.000s
I20260709 13:13:55.280496 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.299538 22833 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:55.310303 22906 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:13:55.312176 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 204838us
W20260709 13:13:55.315209 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2128514920000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 13:13:55.265161 28567 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.275681 28405 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.240962 28563 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.326050 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2298) took 3353 ms. Trace:
I20260709 13:13:55.326543 22759 rpcz_store.cc:276] 0709 13:13:51.971197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.052250 (+1081053us) service_pool.cc:224] Handling call
0709 13:13:53.282338 (+230088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.360140 (+ 77802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.324650 (+1964510us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540035e650. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:55.324799 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520960384}
I20260709 13:13:55.329205 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:55.334779 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:55.339459 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:55.344798 23431 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.221s (SENT)
W20260709 13:13:55.347826 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1932) took 1306 ms (client timeout 1569 ms). Trace:
W20260709 13:13:55.348448 22840 rpcz_store.cc:269] 0709 13:13:54.041581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.063607 (+ 22026us) service_pool.cc:224] Handling call
0709 13:13:55.347622 (+1284015us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540003a2d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:55.347762 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2763862784}
I20260709 13:13:55.311689 28431 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.300565 28333 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.358850 28421 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.357679 28401 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.378334 28553 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.378505 28080 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.328353 27912 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:55.379684 27912 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.380188 28405 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:55.380690 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.354162 28219 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:55.382650 28571 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 13:13:55.385299 28414 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:55.385828 28414 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:55.386755 28414 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.388741 28414 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:55.358734 28504 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.382666 28510 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.395310 28563 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.397282 28518 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.398782 28127 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.402171 28567 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:55.371798 28559 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:13:55.358592 28473 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:55.419396 28516 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.499s	user 0.001s	sys 0.002s
I20260709 13:13:55.420375 28554 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.414805 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:55.416606 27912 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.341693 28543 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:13:55.426638 28543 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:55.427167 28543 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.428256 28543 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.360831 28333 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:55.430652 28502 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.882s	user 0.000s	sys 0.011s
I20260709 13:13:55.413733 28549 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:55.432238 28549 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.433380 28549 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.435320 28549 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:55.425724 22909 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:55.439162 22909 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.440164 22909 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.441553 22909 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:55.442925 28491 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 2.023s	user 0.001s	sys 0.007s
I20260709 13:13:55.425724 28515 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.457069 28545 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.012s
W20260709 13:13:55.476534 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 245597us
I20260709 13:13:55.457857 28421 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.458261 28219 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.458472 28504 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.460156 28429 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:55.460660 28543 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:55.494251 28276 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.498485 28416 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.502401 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.458942 28515 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.459241 28546 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.510056 28403 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:55.521584 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.528175 22832 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:55.530272 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:55.530295 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.532869 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.530328 22837 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:13:55.534521 22838 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:55.534880 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.535596 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:55.535686 22879 connection.cc:256] RPC call timeout handler was delayed by -1.5269935440001063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:55.536754 22839 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:55.537637 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.484143 28219 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.484388 28421 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.539440 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.547655 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.539444 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:55.540328 23462 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:55.542328 23460 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:13:55.542215 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:55.516592 28424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.982s	user 0.019s	sys 0.049s
I20260709 13:13:55.503103 28504 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.543727 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.550285 22834 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:55.544461 22837 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:13:55.517745 28429 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:55.520473 28545 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:13:55.539114 28555 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.556013 28555 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.558583 28555 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.558979 28546 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.560751 28421 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.561980 28579 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:55.562582 28579 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.564782 28579 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.565804 22840 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:55.566776 28219 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:55.567597 28477 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:13:55.568177 22841 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:55.572642 22834 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:13:55.573500 28424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.039s	user 0.022s	sys 0.051s
I20260709 13:13:55.520709 28403 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:55.580930 22805 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:55.581697 28545 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.586174 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:55.583921 28504 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.585022 28509 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:55.585227 28510 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.585759 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:55.569710 28429 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:55.600986 22832 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:13:55.593983 28537 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.693s	user 0.015s	sys 0.000s
I20260709 13:13:55.606263 22834 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:55.649868 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.583973 28473 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:55.649070 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:55.652983 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.595371 28427 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:55.658668 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.663929 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.663857 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.667788 22834 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:55.667030 28080 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:55.673247 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.645470 28579 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.626319 28429 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:55.687429 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.688248 22907 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:55.691623 22841 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:55.600899 28510 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.664439 28546 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.594946 28403 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.695462 22837 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:55.695966 28403 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:55.659312 28537 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.693980 28579 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.695631 28546 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.596246 28545 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.650900 28531 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:13:55.716846 22909 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:13:55.720685 22909 pending_rounds.cc:85] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 11
I20260709 13:13:55.721093 22909 pending_rounds.cc:107] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:13:55.721391 22909 raft_consensus.cc:2889] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:55.717478 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92315.7us
I20260709 13:13:55.610342 28504 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:13:55.654754 28555 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:13:55.654354 28509 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.657703 28519 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:13:55.723199 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.689453 28080 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.728456 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.737887 22834 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:55.728456 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.738646 22841 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:55.731026 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 1980) took 2820 ms. Trace:
I20260709 13:13:55.664181 28473 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.732970 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.690483 28427 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:55.741595 22911 rpcz_store.cc:276] 0709 13:13:52.906803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.385804 (+479001us) service_pool.cc:224] Handling call
0709 13:13:53.409855 (+ 24051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.439131 (+ 29276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:53.439188 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:53.439261 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:53.439309 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.439414 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.726985 (+2287571us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b540010e5d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:55.727129 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18985,"raft.queue_time_us":34561,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5277,"spinlock_wait_cycles":54365568}
I20260709 13:13:55.718411 28094 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 13:13:55.748555 28483 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 2.339s	user 0.000s	sys 0.007s
W20260709 13:13:55.754194 22914 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1974) took 2857 ms (client timeout 2224 ms). Trace:
I20260709 13:13:55.718930 28400 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:13:55.754705 22914 rpcz_store.cc:269] 0709 13:13:52.896406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.237004 (+340598us) service_pool.cc:224] Handling call
0709 13:13:55.753935 (+2516931us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540020a3d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:55.754089 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432490752}
I20260709 13:13:55.758353 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:55.759958 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.760272 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.761134 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:55.761441 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.763443 28473 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.729944 28577 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:13:55.764213 28080 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.747685 28515 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:55.764858 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:55.766001 28509 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.767529 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 758) took 2406 ms. Trace:
I20260709 13:13:55.770154 28400 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:55.799290 22910 rpcz_store.cc:276] 0709 13:13:53.361335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.478910 (+117575us) service_pool.cc:224] Handling call
0709 13:13:53.480355 (+  1445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:53.891797 (+411442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.767242 (+1875445us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b54000fad54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:55.767464 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924425984}
I20260709 13:13:55.773765 28427 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.775367 28546 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.755530 22879 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:55.780433 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:55.802201 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46790.9us
I20260709 13:13:55.803915 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:55.805188 23464 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:55.784958 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:55.797433 23458 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:55.799290 22758 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:13:55.806533 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8670cf017371429ea31529b6f247493e"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:55.763499 28545 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.806216 22908 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:55.809650 23462 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:55.816814 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:55.808030 28519 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:13:55.800879 28400 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:55.832119 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.833300 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.808353 28531 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:55.829855 28515 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:55.842844 28531 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.853385 28515 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:55.858932 28531 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:55.871996 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:55.868937 28515 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:55.864542 28510 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.891108 28495 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:55.891724 28495 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:55.901464 27899 raft_consensus.cc:993] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:55.905560 28219 raft_consensus.cc:1081] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:13:55.892753 28495 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:55.918156 28541 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.288s	user 0.004s	sys 0.008s
I20260709 13:13:55.920568 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3090) took 1911 ms. Trace:
I20260709 13:13:55.921778 22908 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:55.921553 22915 rpcz_store.cc:276] 0709 13:13:54.014788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.059290 (+ 44502us) service_pool.cc:224] Handling call
0709 13:13:54.060032 (+   742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.159974 (+ 99942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.160017 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.187808 (+ 27791us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.433781 (+245973us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:54.433851 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.433974 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:55.915004 (+1481030us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:13:55.919977 (+  4973us) raft_consensus.cc:1685] finished
0709 13:13:55.920029 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.920398 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41757,"raft.queue_time_us":169519,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":5414,"spinlock_wait_cycles":8846720,"thread_start_us":253757,"threads_started":2,"wal-append.queue_time_us":337578}
I20260709 13:13:55.862010 28400 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.912458 28519 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.939105 28558 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.726s	user 0.000s	sys 0.001s
I20260709 13:13:55.942360 28431 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.943804 28442 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:55.944137 28442 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.944880 28442 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.946211 28127 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:55.946784 28127 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.947635 28127 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.948827 28127 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.950101 28568 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:55.951159 28568 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:55.951969 28568 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:55.952287 28568 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:55.952628 28219 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:55.953015 28219 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:55.953919 28219 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.955335 28383 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.956645 28383 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:55.958010 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:55.959534 22912 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:55.959895 22912 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.960738 22912 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.961856 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.933370 28519 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.963702 28577 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:55.967728 22912 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.968814 28519 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:55.969688 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.969861 28441 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:55.970556 28577 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:55.971820 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.973133 28577 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:55.973850 28577 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:55.974154 28538 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:55.974611 28538 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.974973 28578 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:55.975368 28578 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:55.975595 28538 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:55.976186 28578 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:55.976691 28538 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:55.950441 28579 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:55.965302 22911 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:55.981106 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:55.981412 28431 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:55.983556 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:55.984869 28546 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:55.985026 28579 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:55.985375 28546 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:55.946276 28442 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:55.975598 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:55.955789 28219 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:55.929608 28495 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:55.984234 28384 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:55.999338 28587 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:56.000023 28587 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:56.000293 28546 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.001753 28546 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:55.999430 28545 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:56.001093 28587 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.004477 28587 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:56.008661 22838 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:56.006919 28579 consensus_queue.cc:1243] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:13:56.014109 28494 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
W20260709 13:13:56.028769 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 748292us
I20260709 13:13:56.032366 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.032744 22912 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:13:56.033358 22912 pending_rounds.cc:85] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 10
I20260709 13:13:56.033226 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.033761 22912 pending_rounds.cc:107] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:13:56.035053 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.036398 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.036888 22765 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:56.034088 22912 raft_consensus.cc:2889] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:56.038784 22766 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:56.040356 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3083) took 2040 ms. Trace:
I20260709 13:13:56.038653 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.034660 22757 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:13:56.034073 22764 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:56.046645 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.037705 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.054677 22765 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:56.073783 28602 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:56.093918 28604 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.095041 28604 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:55.992839 28549 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.004s
I20260709 13:13:56.098712 28572 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:56.099540 28572 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:56.099889 28604 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.100059 28572 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.101989 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.102283 28531 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.102717 22836 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:56.042520 22912 rpcz_store.cc:276] 0709 13:13:53.999743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.999936 (+   193us) service_pool.cc:224] Handling call
0709 13:13:55.959163 (+1959227us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:13:55.959376 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.037637 (+ 78261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.040295 (+  2658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3496832}
I20260709 13:13:56.106160 28429 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.108595 28429 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:56.104655 28531 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:56.110392 22909 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:13:56.111248 22907 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:56.111110 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.113261 22763 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:56.010113 28214 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.127913 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.128180 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:56.128664 23465 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:56.128813 22836 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.125001 28604 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:56.130996 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.132413 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.132993 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.133633 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.134003 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.134558 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.135167 28563 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.136063 28563 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:56.141664 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 111270us
I20260709 13:13:56.143198 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:55.953143 28568 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.144917 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.145838 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.146421 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.147240 22761 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:56.147118 28214 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.148911 23458 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 16. Ops: [14.11-14.11]
W20260709 13:13:56.164258 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.165145 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.168869 22730 connection.cc:256] RPC call timeout handler was delayed by -1.5428370419999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
I20260709 13:13:56.167593 23459 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:13:56.168308 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.174614 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.192368 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.193199 22763 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:13:56.197583 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:56.203017 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.203184 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.204862 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:56.110498 28545 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.204208 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.212371 22757 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:56.203758 22759 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:56.212778 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:56.213146 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.210516 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.214325 22838 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:56.211684 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.214880 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:56.215920 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.219118 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.219739 22759 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:56.219091 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.220728 22912 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:56.220933 22838 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:56.130901 28604 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.179961 28403 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:56.223852 22832 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:56.125674 28549 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:56.229140 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.221938 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:56.248236 22912 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:56.197101 28573 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.196238 28494 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.249234 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:56.250844 22838 raft_consensus.cc:2368] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:56.250912 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.219379 28504 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.218449 28510 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:56.204794 28509 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:56.218055 28568 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:56.297046 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.254899 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.261549 28214 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.267731 22910 raft_consensus.cc:606] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
W20260709 13:13:56.270866 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.300022 22910 raft_consensus.cc:614] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:56.277828 22759 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:56.282032 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.285576 28384 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.287170 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.292681 22765 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:56.310889 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.311537 22765 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:56.253700 22836 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:13:56.295120 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74830.4us
I20260709 13:13:56.302501 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:56.302686 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.313809 22764 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:56.314136 22759 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:13:56.312742 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.224397 28545 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.315230 22879 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:56.319734 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.255308 28403 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:56.325014 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:56.333961 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.335739 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.336362 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.336943 22759 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:56.337666 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.338402 28563 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:56.340409 28577 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.340955 28577 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:56.342751 28606 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:13:56.343237 28577 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.343796 28577 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:56.344679 28577 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.345433 28577 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:56.346840 28564 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.104s	user 0.003s	sys 0.000s
I20260709 13:13:56.352042 22765 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:56.353051 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.360611 22759 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:13:56.254268 28549 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.364589 22912 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:56.362394 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.371s (SENT)
W20260709 13:13:56.344570 28527 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:13:56.367718 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.287721 28494 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.362972 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.362874 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.330765 28530 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:56.297726 28510 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.386078 22912 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:56.376335 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.373978 22765 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:56.387660 22759 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:13:56.390421 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.294979 28504 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:56.377115 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 147298us
I20260709 13:13:56.258751 28573 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.398844 22834 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:56.400300 22765 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:56.403287 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:56.300794 28509 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.318480 28384 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:56.420792 28599 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 13:13:56.402951 28572 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:56.435251 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 105513us
I20260709 13:13:56.410297 28530 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:56.423812 28383 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:13:56.429781 28515 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.000s
W20260709 13:13:56.450222 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125298us
W20260709 13:13:56.466833 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.472967 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:56.482522 28605 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:56.489436 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.490242 22766 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:13:56.491181 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.491750 22766 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:56.492381 28604 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.494033 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.504702 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:56.505203 28442 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 13:13:56.505586 22766 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:56.512938 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.515722 22766 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10.
I20260709 13:13:56.517184 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.518968 28384 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:56.520619 28376 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:56.523765 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.417999 28403 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.477488 22835 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
W20260709 13:13:56.656657 22906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3092) took 2594 ms (client timeout 2049 ms). Trace:
I20260709 13:13:56.615907 22757 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
W20260709 13:13:56.618276 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.682986 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 898) took 1083 ms. Trace:
I20260709 13:13:56.815562 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.815765 22837 rpcz_store.cc:276] 0709 13:13:55.599807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.599905 (+    98us) service_pool.cc:224] Handling call
0709 13:13:55.600216 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.695914 (+ 95698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.682722 (+986808us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b540019f0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:56.682922 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218044288}
I20260709 13:13:56.682646 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3088) took 2673 ms. Trace:
I20260709 13:13:56.814263 22766 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:56.684944 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:56.817873 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.657373 22731 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items [suppressed 2 similar messages]
W20260709 13:13:56.814640 22906 rpcz_store.cc:269] 0709 13:13:54.064608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.064766 (+   158us) service_pool.cc:224] Handling call
0709 13:13:56.656368 (+2591602us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001e32d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:56.656534 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2669741952}
I20260709 13:13:56.816761 22913 rpcz_store.cc:276] 0709 13:13:54.008954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.014580 (+  5626us) service_pool.cc:224] Handling call
0709 13:13:54.048742 (+ 34162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.222781 (+1174039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.682354 (+1459573us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001d0e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:56.682569 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2652489600}
W20260709 13:13:56.565231 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68258.2us
I20260709 13:13:56.827152 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.827170 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.828217 22906 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:56.829373 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.831651 23457 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:56.528800 28598 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.623s	user 0.000s	sys 0.001s
W20260709 13:13:56.835498 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2305) took 3228 ms (client timeout 1722 ms). Trace:
W20260709 13:13:56.525374 28587 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.000s
W20260709 13:13:56.846969 22758 rpcz_store.cc:269] 0709 13:13:53.599721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.599916 (+   195us) service_pool.cc:224] Handling call
0709 13:13:56.827495 (+3227579us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:56.827635 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5007011072}
I20260709 13:13:56.837615 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:56.848625 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:56.849445 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.841375 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.849602 22766 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:13:56.850112 22913 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
W20260709 13:13:56.560755 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.802605 28079 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.468956 28515 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:56.852553 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.735821 28442 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.832017 28127 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:56.865155 28609 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.239s	user 0.000s	sys 0.002s
I20260709 13:13:56.854027 22759 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:56.531693 28504 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.527974 28510 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.856621 22838 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:13:56.534839 28494 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:56.529121 28509 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:56.540398 28545 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:56.535863 28549 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:56.862520 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.827004 28384 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:56.892060 28384 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.893080 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2001) took 1316 ms. Trace:
I20260709 13:13:56.893483 22911 rpcz_store.cc:276] 0709 13:13:55.576564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.980750 (+404186us) service_pool.cc:224] Handling call
0709 13:13:55.981069 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.267448 (+286379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.267482 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.267526 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:56.267565 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.892802 (+625237us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b540010f4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:56.892866 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.893017 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4733,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":657817472}
I20260709 13:13:56.894421 28384 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:56.847774 28546 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.846s	user 0.016s	sys 0.000s
I20260709 13:13:56.895164 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:56.895725 22911 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:56.882164 22841 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:56.878722 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:56.597625 28573 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:56.883431 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1935) took 2811 ms. Trace:
I20260709 13:13:56.897706 22833 rpcz_store.cc:276] 0709 13:13:54.069774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.135750 (+ 65976us) service_pool.cc:224] Handling call
0709 13:13:54.136406 (+   656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.311454 (+1175048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.880377 (+1568923us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540028aad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:56.880600 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2674324736}
I20260709 13:13:56.863233 28079 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.900712 28079 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:56.880512 28127 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.906030 28127 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:56.907290 28614 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:13:56.887761 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:56.912951 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.028s	sys 0.046s
W20260709 13:13:56.913364 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.028s	sys 0.046s
I20260709 13:13:56.679332 22732 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2356)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2339)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2345)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2351)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2347)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2939)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2331)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2332)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46956 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2921)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2338)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2923)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2929)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2931)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2353)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 788)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2354)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2945)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2355)
W20260709 13:13:56.914906 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.913175 22913 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:56.909196 28578 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:56.526041 28530 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:56.916589 22911 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:13:56.920074 23431 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.536s (SENT)
I20260709 13:13:56.957340 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:56.925594 28494 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:56.927045 28568 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.626s	user 0.008s	sys 0.003s
W20260709 13:13:56.928323 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.929191 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2303) took 4935 ms. Trace:
I20260709 13:13:56.929996 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 905) took 1151 ms. Trace:
I20260709 13:13:56.961369 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:56.931715 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 1995) took 1356 ms (client timeout 1789 ms). Trace:
I20260709 13:13:56.962255 22912 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:56.931430 28515 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:56.935752 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.963677 22914 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:56.941996 28554 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:13:56.962679 22909 rpcz_store.cc:269] 0709 13:13:55.575164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.722607 (+147443us) service_pool.cc:224] Handling call
0709 13:13:56.931508 (+1208901us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54001b9ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:56.931628 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861648256}
I20260709 13:13:56.959554 22760 rpcz_store.cc:276] 0709 13:13:51.994313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.138930 (+1144617us) service_pool.cc:224] Handling call
0709 13:13:54.635181 (+1496251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.635529 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:54.748602 (+113073us) log.cc:844] Serialized 14 byte log entry
0709 13:13:55.300611 (+552009us) log.cc:844] Serialized 14 byte log entry
0709 13:13:56.013003 (+712392us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b18002cc0e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:13:56.021315 (+  8312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.030567 (+  9252us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:56.030626 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.030721 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.928994 (+898273us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002224d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:56.929125 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":794328,"prepare.queue_time_us":296130,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":2788,"raft.queue_time_us":952974,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":7972,"spinlock_wait_cycles":3397005440,"thread_start_us":641123,"threads_started":3,"wal-append.queue_time_us":520584,"wal-append.run_cpu_time_us":2203,"wal-append.run_wall_time_us":1075951}
I20260709 13:13:56.946434 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 776) took 1440 ms. Trace:
I20260709 13:13:56.966838 22907 rpcz_store.cc:276] 0709 13:13:55.505738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.795265 (+289527us) service_pool.cc:224] Handling call
0709 13:13:56.946163 (+1150898us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54000ea954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:56.946348 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704822656}
I20260709 13:13:56.947527 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 774) took 1470 ms. Trace:
I20260709 13:13:56.968266 22915 rpcz_store.cc:276] 0709 13:13:55.476717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.924437 (+447720us) service_pool.cc:224] Handling call
0709 13:13:55.924776 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.986130 (+ 61354us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:56.008452 (+ 22322us) log.cc:844] Serialized 14 byte log entry
0709 13:13:56.190432 (+181980us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.190492 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:56.190546 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.947215 (+756669us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b540020b7d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:56.947284 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.947453 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447,"prepare.queue_time_us":22445,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":5811,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":15474,"spinlock_wait_cycles":140505344,"thread_start_us":180884,"threads_started":1,"wal-append.queue_time_us":251759}
W20260709 13:13:56.955533 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.957945 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.963099 23460 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:56.961658 22840 rpcz_store.cc:276] 0709 13:13:55.778602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.796856 (+ 18254us) service_pool.cc:224] Handling call
0709 13:13:55.802662 (+  5806us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:55.803075 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:55.803107 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:56.350702 (+547595us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:13:56.351440 (+   738us) log.cc:844] Serialized 52 byte log entry
0709 13:13:56.658462 (+307022us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:56.658543 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.658726 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:56.929668 (+270942us) raft_consensus.cc:1685] finished
0709 13:13:56.929691 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.929933 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3592,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3112,"spinlock_wait_cycles":5254400,"thread_start_us":306333,"threads_started":1,"wal-append.queue_time_us":524696}
I20260709 13:13:56.971261 23460 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:13:56.973315 22806 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.079s (SENT)
I20260709 13:13:56.973999 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:56.977746 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:56.925364 28219 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.932s	user 0.011s	sys 0.000s
I20260709 13:13:56.972592 23460 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.981317 23462 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:13:56.877025 28442 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:56.978960 22913 raft_consensus.cc:2376] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15. Re-sending same reply.
I20260709 13:13:56.981608 23460 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:56.983508 22839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.060s	sys 0.075s
W20260709 13:13:56.988245 22839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.060s	sys 0.075s
I20260709 13:13:56.990411 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:56.985050 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:56.959573 28494 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.995008 28546 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:56.965305 28604 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:13:56.996137 28495 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.989s	user 0.004s	sys 0.008s
I20260709 13:13:56.997108 28576 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:56.997609 28576 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.997958 28576 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:56.965181 28079 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:56.995092 28494 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:56.997151 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:13:56.984522 23429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.051s	sys 0.079s
W20260709 13:13:57.008564 23429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.052s	sys 0.079s
I20260709 13:13:56.960763 28127 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.000980 28079 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.012539 22762 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:13:57.011947 22806 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.454s (SENT)
I20260709 13:13:57.014571 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:57.015271 22906 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:57.016043 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:57.018589 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:57.019272 22909 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:57.020684 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:56.996198 28546 consensus_queue.cc:1243] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:13:57.010026 28515 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:57.033630 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:57.030476 22912 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:56.981637 28554 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.039860 22912 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:57.040063 22909 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:57.041188 22913 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:57.041673 22912 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:57.035976 28538 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.055s	user 0.016s	sys 0.000s
I20260709 13:13:57.044330 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:57.013031 28127 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.048084 28079 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:57.052907 22914 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:57.060910 28127 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:57.036888 28568 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:57.043318 28219 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:57.057920 28515 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:57.064219 22911 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:13:57.045528 27899 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
W20260709 13:13:57.063017 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.066636 28219 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:57.066562 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:57.075268 23457 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:13:57.062238 28577 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.716s	user 0.012s	sys 0.000s
I20260709 13:13:57.077651 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.091424 22757 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:57.078034 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:57.083813 23462 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:57.084456 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:57.069547 28515 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:13:57.101615 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.103567 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.064512 28276 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:57.107031 22913 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:57.074373 28543 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:57.110103 28543 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.114339 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.115859 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.116556 22760 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:57.118192 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.119607 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:57.117266 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.122190 22760 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:57.123742 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.126422 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.120697 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:57.102850 28515 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:57.131798 22761 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:57.133126 22760 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:57.104142 28421 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:57.145473 22757 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:57.154220 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.156699 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:57.158234 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.158896 22757 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:57.161360 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.162308 22761 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.163529 23460 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:13:57.171618 22832 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:57.172317 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 82570.9us
I20260709 13:13:57.174980 22760 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:57.176033 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.102538 28604 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:13:57.134063 27899 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:57.184600 22832 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:57.181766 22760 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:57.134443 28568 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:57.184965 23460 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:13:57.147430 28442 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:57.191628 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.160609 28421 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:57.104550 28577 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.160413 28509 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.199740 22760 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:57.206383 22757 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:57.124825 28276 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.171331 28613 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:13:57.221630 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63132.9us
I20260709 13:13:57.221208 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.223949 22760 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:57.224860 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.225662 22760 raft_consensus.cc:2410] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:13:57.218485 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.170724 28587 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:57.183563 28530 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:13:57.227649 28616 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:57.228737 28276 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.228796 28616 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:57.187295 28568 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.230163 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:57.170500 28504 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:13:57.221232 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:57.231873 22757 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:13:57.240631 23431 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.249s (SENT)
I20260709 13:13:57.242682 23460 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:57.246691 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2332) took 1074 ms. Trace:
I20260709 13:13:57.247213 22757 rpcz_store.cc:276] 0709 13:13:56.172594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.245674 (+1073080us) service_pool.cc:224] Handling call
0709 13:13:57.246129 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.246336 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:57.246361 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.246412 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:57.246459 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.246525 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.246631 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.233728 28433 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:57.240953 28568 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:57.234016 28616 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:57.235605 28613 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:57.235260 28587 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:57.234061 28427 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:13:57.194941 28219 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.226254 28509 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:57.269012 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46680.1us
I20260709 13:13:57.272019 28576 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.277547 22761 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:13:57.281451 22760 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:57.282755 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 783) took 1066 ms. Trace:
I20260709 13:13:57.283555 22761 rpcz_store.cc:276] 0709 13:13:56.216112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.161020 (+944908us) service_pool.cc:224] Handling call
0709 13:13:57.282659 (+121639us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.258867 28403 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.262094 22757 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:13:57.291543 22757 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:57.292011 22757 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:57.229935 28421 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.293792 28421 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:57.293952 28616 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.295317 28616 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.296975 28613 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.308517 22840 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:57.279156 28554 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:57.278764 28620 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:13:57.292335 27899 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:57.312747 28591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.010s	sys 0.011s
I20260709 13:13:57.315316 28634 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 13:13:57.330173 27899 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.341856 28631 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:57.342350 28631 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:57.327371 28591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.011s	sys 0.012s
I20260709 13:13:57.326092 28554 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.356402 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1950) took 1109 ms. Trace:
I20260709 13:13:57.358215 22763 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:57.358659 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.358829 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 781) took 1184 ms. Trace:
I20260709 13:13:57.359305 22763 rpcz_store.cc:276] 0709 13:13:56.174376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.357528 (+1183152us) service_pool.cc:224] Handling call
0709 13:13:57.357867 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.358598 (+   731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.358774 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.360152 22759 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:13:57.360759 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2921) took 1183 ms. Trace:
I20260709 13:13:57.361093 22763 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:13:57.361959 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2338) took 1154 ms. Trace:
I20260709 13:13:57.355087 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1955) took 1043 ms. Trace:
I20260709 13:13:57.406075 22763 rpcz_store.cc:276] 0709 13:13:56.207704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.360345 (+1152641us) service_pool.cc:224] Handling call
0709 13:13:57.360696 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.361673 (+   977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.361894 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.406194 22841 rpcz_store.cc:276] 0709 13:13:56.311987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.316011 (+  4024us) service_pool.cc:224] Handling call
0709 13:13:56.316373 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.896549 (+580176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.354800 (+458251us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400321ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:57.355020 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298736896}
I20260709 13:13:57.301664 28403 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.361181 22759 rpcz_store.cc:276] 0709 13:13:56.177474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.359587 (+1182113us) service_pool.cc:224] Handling call
0709 13:13:57.359929 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:57.360529 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.360706 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.356863 22834 rpcz_store.cc:276] 0709 13:13:56.246611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.247947 (+  1336us) service_pool.cc:224] Handling call
0709 13:13:56.248341 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:56.399483 (+151142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.356146 (+956663us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54001bc954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:57.356345 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345225600}
W20260709 13:13:57.382563 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.428580 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.429021 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66331.3us
I20260709 13:13:57.430899 28639 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:57.431469 28639 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.428575 22759 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:57.433947 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 785) took 1125 ms. Trace:
I20260709 13:13:57.434314 22759 rpcz_store.cc:276] 0709 13:13:56.308276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.427901 (+1119625us) service_pool.cc:224] Handling call
0709 13:13:57.428237 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.433595 (+  5358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.433884 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.435750 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:57.436065 22759 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:57.436868 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2346) took 1121 ms. Trace:
W20260709 13:13:57.439229 22806 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.634s (SENT)
W20260709 13:13:57.443480 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.449530 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.358089 22757 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.315748 28427 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:13:57.434764 28639 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.408730 22763 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:13:57.479858 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.770946 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2923) took 1519 ms. Trace:
I20260709 13:13:57.771286 28638 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.771876 28638 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:57.771322 22763 rpcz_store.cc:276] 0709 13:13:56.251530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.407774 (+1156244us) service_pool.cc:224] Handling call
0709 13:13:57.408243 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.770630 (+362387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.770876 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.773840 22757 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.774340 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46956 (request call id 2334) took 1601 ms. Trace:
I20260709 13:13:57.774652 22757 rpcz_store.cc:276] 0709 13:13:56.190471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.248547 (+1058076us) service_pool.cc:224] Handling call
0709 13:13:57.774268 (+525721us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.327560 28587 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:57.426647 28094 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:13:57.347047 28537 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:57.452265 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.343389 28633 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.147s
I20260709 13:13:57.444005 28504 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.182s
I20260709 13:13:57.717414 27899 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.446938 28494 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:57.760311 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.812s (SENT)
W20260709 13:13:57.544399 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.646085 28630 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:13:57.427033 28554 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:57.516193 28623 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:13:57.437155 22759 rpcz_store.cc:276] 0709 13:13:56.315796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.435469 (+1119673us) service_pool.cc:224] Handling call
0709 13:13:57.435837 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.436584 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.436806 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.765930 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.809913 22841 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:13:57.812043 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.766654 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.767311 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.769598 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:57.769995 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.783957 22757 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:57.814422 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2929) took 1474 ms. Trace:
I20260709 13:13:57.814416 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.814857 22757 rpcz_store.cc:276] 0709 13:13:56.340504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.783233 (+1442729us) service_pool.cc:224] Handling call
0709 13:13:57.783718 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.814175 (+ 30457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.814365 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19328}
I20260709 13:13:57.815441 22837 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:13:57.788051 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.794176 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:57.784523 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2928) took 1444 ms. Trace:
I20260709 13:13:57.818395 22763 rpcz_store.cc:276] 0709 13:13:56.340208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.783314 (+1443106us) service_pool.cc:224] Handling call
0709 13:13:57.783767 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.784029 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:57.784069 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.784131 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:57.784183 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.784271 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.784438 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17280}
W20260709 13:13:57.824689 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.824913 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.828990 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:57.829878 22832 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:57.833935 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 786) took 1479 ms. Trace:
I20260709 13:13:57.835176 22763 rpcz_store.cc:276] 0709 13:13:56.355061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.832779 (+1477718us) service_pool.cc:224] Handling call
0709 13:13:57.833244 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.833483 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:57.833537 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.833598 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:57.833646 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.833723 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.833861 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.843376 28647 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.844063 28647 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.854058 22763 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:13:57.855938 22763 pending_rounds.cc:85] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 13
I20260709 13:13:57.856674 22763 pending_rounds.cc:107] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 15.14
W20260709 13:13:57.859283 28620 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.058s
I20260709 13:13:57.861558 22763 raft_consensus.cc:2889] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:57.862838 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.863373 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.863205 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:57.862633 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 787) took 1491 ms. Trace:
I20260709 13:13:57.863843 22836 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.865164 22763 rpcz_store.cc:276] 0709 13:13:56.370988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.839459 (+1468471us) service_pool.cc:224] Handling call
0709 13:13:57.839878 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.862299 (+ 22421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.862568 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.866775 22841 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 13:13:57.868842 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 108781us
W20260709 13:13:57.860090 28620 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.005s	sys 0.058s
I20260709 13:13:57.845319 28401 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:57.872677 28401 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.875912 28401 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:57.876322 28554 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:57.880676 28578 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.920s	user 0.007s	sys 0.012s
I20260709 13:13:57.881348 27912 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.881980 27912 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:57.885668 28638 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.893316 22763 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:57.895841 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2352) took 1502 ms. Trace:
I20260709 13:13:57.902453 22763 rpcz_store.cc:276] 0709 13:13:56.393876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.866609 (+1472733us) service_pool.cc:224] Handling call
0709 13:13:57.867013 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.895465 (+ 28452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.895770 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.904783 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2353) took 1510 ms. Trace:
I20260709 13:13:57.905296 22763 rpcz_store.cc:276] 0709 13:13:56.394198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.903669 (+1509471us) service_pool.cc:224] Handling call
0709 13:13:57.904097 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.904339 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:57.904378 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.904438 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:57.904482 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.904562 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.904716 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:57.907508 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 788) took 1506 ms. Trace:
I20260709 13:13:57.907653 22836 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:13:57.908042 22763 rpcz_store.cc:276] 0709 13:13:56.400867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.906506 (+1505639us) service_pool.cc:224] Handling call
0709 13:13:57.906861 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.907084 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:57.907120 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.907177 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:57.907223 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.907299 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.907444 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:58.015646 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.016835 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.921866 28633 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.923009 27912 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.929761 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46252 (request call id 43) took 2355 ms. Trace:
I20260709 13:13:57.870846 28576 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.936350 28494 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.938906 28577 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:57.941102 28638 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:57.946630 28643 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.038650 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 780) took 1864 ms. Trace:
I20260709 13:13:57.946465 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:57.809644 27899 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.034550 28401 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:13:57.948462 28647 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.038604 28604 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:13:57.949793 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:57.950719 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 785) took 1127 ms. Trace:
W20260709 13:13:57.952128 28421 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.004s
W20260709 13:13:57.952400 28515 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.773s	user 0.015s	sys 0.004s
I20260709 13:13:57.954135 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 911) took 1687 ms. Trace:
I20260709 13:13:58.069689 22838 rpcz_store.cc:276] 0709 13:13:56.266615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.286887 (+ 20272us) service_pool.cc:224] Handling call
0709 13:13:57.953896 (+1667009us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:57.954064 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255640448}
I20260709 13:13:57.955209 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.071425 22766 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:57.957619 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.076704 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:58.076766 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:57.968931 28616 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.006s
W20260709 13:13:57.982843 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.888422 28568 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.006s
I20260709 13:13:57.988674 28431 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:13:57.988965 22806 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.945s (SENT)
I20260709 13:13:57.990345 28384 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:56.818646 22731 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:46950 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2356)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2339)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2345)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2351)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2347)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2939)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2331)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2332)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46956 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2921)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2338)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2923)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2929)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2931)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2353)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 788)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2354)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2945)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2355)
W20260709 13:13:57.991449 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.991710 28546 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.001789 28631 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.006870 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 772) took 2547 ms. Trace:
I20260709 13:13:58.010483 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.010835 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:13:58.013789 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 124549us
I20260709 13:13:58.008159 28578 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.017691 22763 raft_consensus.cc:2376] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14. Re-sending same reply.
I20260709 13:13:58.033787 22837 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 13: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:58.034804 22910 rpcz_store.cc:276] 0709 13:13:55.574564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.806357 (+231793us) service_pool.cc:224] Handling call
0709 13:13:57.929473 (+2123116us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540010f4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aeb853 00007f68a6ab960a 00007f68a003f7d4 00007f68a003f762 00007f68a003f691 00007f68a003f60c 00007f68a003e8c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:57.929648 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059140864}
I20260709 13:13:58.035091 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:57.934968 22762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2913) took 1903 ms (client timeout 1634 ms). Trace:
I20260709 13:13:58.036024 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.046931 22765 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:57.947577 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3109) took 1674 ms. Trace:
I20260709 13:13:57.975782 22660 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 7, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.082859 28376 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.110469 28376 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.083137 28515 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.112690 28376 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:58.113615 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2331) took 1941 ms. Trace:
I20260709 13:13:58.114483 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:58.115537 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126630us
I20260709 13:13:58.117889 28421 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:58.118290 28504 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:58.118734 28504 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:58.090088 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.119253 22911 rpcz_store.cc:276] 0709 13:13:56.272931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.957605 (+684674us) service_pool.cc:224] Handling call
0709 13:13:57.947333 (+989728us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b54003c87d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:57.947482 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213580032}
I20260709 13:13:58.121202 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.121862 22763 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:58.122468 28431 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:58.122834 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.123396 28431 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.124832 28431 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.045454 28638 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:58.126971 28638 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.128037 28638 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:58.128662 22833 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:58.038945 28633 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.130543 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:58.131557 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.040729 28214 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.056041 22758 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
W20260709 13:13:58.132802 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2947) took 1681 ms (client timeout 1812 ms). Trace:
I20260709 13:13:57.961927 28400 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:58.133183 22838 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:13:58.035753 28611 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:13:58.132458 28214 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:58.133222 22758 rpcz_store.cc:269] 0709 13:13:56.459946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.949465 (+1489519us) service_pool.cc:224] Handling call
0709 13:13:58.132722 (+183257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:58.088181 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.099726 28634 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:58.107184 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.130729 28543 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.152972 23464 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:58.056546 22906 rpcz_store.cc:276] 0709 13:13:56.823342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.829048 (+  5706us) service_pool.cc:224] Handling call
0709 13:13:57.950532 (+1121484us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540020a3d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:57.950657 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424173440}
I20260709 13:13:58.114056 22835 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:58.118633 22908 rpcz_store.cc:276] 0709 13:13:55.459872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.812324 (+352452us) service_pool.cc:224] Handling call
0709 13:13:55.812558 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.924004 (+111446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.006499 (+2082495us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54002111d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:58.006772 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241976192}
W20260709 13:13:58.138880 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.158147 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.086210 22836 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:13:58.139772 22762 rpcz_store.cc:269] 0709 13:13:56.031617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.031742 (+   125us) service_pool.cc:224] Handling call
0709 13:13:57.934682 (+1902940us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400369554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:57.934832 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958065920}
I20260709 13:13:58.164104 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.164983 22762 raft_consensus.cc:2376] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15. Re-sending same reply.
I20260709 13:13:58.114030 22760 rpcz_store.cc:276] 0709 13:13:56.172345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.233014 (+1060669us) service_pool.cc:224] Handling call
0709 13:13:57.281186 (+ 48172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.281984 (+   798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.113372 (+831388us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b540035eb50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:58.113543 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109686144}
I20260709 13:13:58.166853 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:58.143627 22880 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.722s (SENT)
I20260709 13:13:58.168533 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:58.046411 22761 rpcz_store.cc:276] 0709 13:13:56.174130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.284571 (+1110441us) service_pool.cc:224] Handling call
0709 13:13:57.285043 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.285320 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:57.347842 (+ 62522us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.846470 (+498628us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.932769 (+ 86299us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b1800421840. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:13:58.037645 (+104876us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:58.037723 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:58.037800 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.038446 (+   646us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.038587 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79455,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":98,"raft.queue_time_us":282015,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":204,"spinlock_wait_cycles":19053312,"thread_start_us":714352,"threads_started":4,"wal-append.queue_time_us":508996}
I20260709 13:13:58.170250 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.170986 22760 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:13:58.171128 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.171466 22906 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:13:58.173355 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.132467 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:58.124378 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.175374 22840 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:58.175518 22763 raft_consensus.cc:2376] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 10. Re-sending same reply.
W20260709 13:13:58.097083 28621 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.849s	user 0.002s	sys 0.010s
I20260709 13:13:58.148046 22766 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:13:58.178514 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2922) took 1962 ms (client timeout 2049 ms). Trace:
W20260709 13:13:58.179073 22766 rpcz_store.cc:269] 0709 13:13:56.216556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.954852 (+1738296us) service_pool.cc:224] Handling call
0709 13:13:58.178437 (+223585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:58.097945 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1948) took 1880 ms. Trace:
I20260709 13:13:58.180006 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.140941 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:58.152594 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2333) took 1979 ms (client timeout 2168 ms). Trace:
W20260709 13:13:58.181764 22764 rpcz_store.cc:269] 0709 13:13:56.172834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.106849 (+1934015us) service_pool.cc:224] Handling call
0709 13:13:58.152530 (+ 45681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:58.139214 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.183072 23466 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:58.184918 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.185613 22758 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:58.186472 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.188112 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.188879 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.187986 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.189813 22763 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:58.190145 22761 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:58.188982 22766 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:13:58.137984 22841 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:58.191074 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.192540 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.192799 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.177592 22910 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:58.193289 22760 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:58.103895 27899 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.124342 28509 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.187052 22758 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
W20260709 13:13:58.205462 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64655us
I20260709 13:13:58.148977 28634 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:58.180709 22839 rpcz_store.cc:276] 0709 13:13:56.217119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.222816 (+  5697us) service_pool.cc:224] Handling call
0709 13:13:56.223120 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.427005 (+203885us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:56.442546 (+ 15541us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.007651 (+565105us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:57.007731 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:57.007803 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.097632 (+1089829us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:58.097688 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.097872 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19448,"prepare.queue_time_us":15926,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2898,"raft.queue_time_us":22063,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":11116,"spinlock_wait_cycles":468192128,"thread_start_us":545954,"threads_started":1,"wal-append.queue_time_us":663788,"wal-append.run_cpu_time_us":1378,"wal-append.run_wall_time_us":1003333}
I20260709 13:13:58.194654 22906 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:58.193390 22761 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:58.202256 22765 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:58.195428 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.215616 22760 raft_consensus.cc:2376] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15. Re-sending same reply.
I20260709 13:13:58.178542 28465 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:58.225445 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.230528 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.232779 28633 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:58.239084 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.209208 28543 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:58.196484 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.247051 22910 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:58.153765 28578 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:58.197824 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54244.9us
I20260709 13:13:58.202620 22660 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.202656 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.196058 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.253873 22762 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:58.255808 22758 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:58.212963 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.270849 22766 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:13:58.271198 23466 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:13:58.214594 22833 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:58.270879 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.214501 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:58.272331 22762 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:58.190435 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:58.271308 22660 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.170559 28403 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.090039 28568 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.058437 28643 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:58.217574 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2335) took 2026 ms (client timeout 1797 ms). Trace:
W20260709 13:13:58.276593 22765 rpcz_store.cc:269] 0709 13:13:56.190777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.946152 (+1755375us) service_pool.cc:224] Handling call
0709 13:13:58.217473 (+271321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:58.277839 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.119434 28384 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.220920 22907 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:58.242954 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.196609 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.153612 28546 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:58.197397 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 184360us
W20260709 13:13:58.301678 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.301898 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.212706 28425 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:58.250394 28214 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:58.187135 28616 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:13:58.259284 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:58.245922 28465 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:58.168375 28538 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.238690 28515 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:58.273229 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 82876.6us
I20260709 13:13:58.276690 28543 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:58.273877 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 59468us
I20260709 13:13:58.220489 28421 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:13:58.278945 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.296296 28376 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.211803 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:13:58.323275 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64095.9us
I20260709 13:13:58.295418 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:58.330749 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.330935 22766 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:58.302783 22758 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:13:58.329414 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.332751 22765 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17. Re-sending same reply.
I20260709 13:13:58.291940 22840 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:58.333247 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:13:58.304781 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.308888 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:58.334565 22806 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.049s (SENT)
I20260709 13:13:58.301016 22660 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.216735 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.335978 22760 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:58.293962 28510 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:58.322548 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.324800 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:58.344636 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:58.314103 22880 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:58.340582 22730 leader_election.cc:400] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:58.305374 28425 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.357538 28079 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:58.357740 28425 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:58.358156 28079 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:58.358640 28079 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:58.358453 28554 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:58.329811 22761 raft_consensus.cc:2376] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:13:58.345683 22730 leader_election.cc:403] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:58.361203 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.361203 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.362469 22660 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.363458 28384 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.335683 28214 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:58.364308 28214 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.364715 28214 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.364518 22660 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.363214 22761 raft_consensus.cc:2376] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15. Re-sending same reply.
W20260709 13:13:58.350210 28219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.020s	sys 0.020s
I20260709 13:13:58.369057 22762 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:58.369632 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:58.357563 28433 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:58.370523 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 56522.3us
I20260709 13:13:58.328361 28538 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.371423 28537 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.372997 28643 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.373632 28543 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:58.374480 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.373926 22660 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.250679 28504 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.317888 28403 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.382988 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.387396 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:58.410395 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:58.269837 28634 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.320700 28655 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:13:58.422749 22760 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:13:58.399875 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:58.424160 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:58.393637 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.404556 22732 connection.cc:256] RPC call timeout handler was delayed by -3.59992284000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
I20260709 13:13:58.406059 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:58.425972 22732 connection.cc:256] RPC call timeout handler was delayed by -3.5399978340001326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:13:58.426293 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:58.401919 22766 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:58.410936 22763 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:13:58.379482 27912 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:58.377381 28602 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.342504 28510 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:58.434284 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.449280 22660 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:58.427384 22732 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.537s (SENT)
I20260709 13:13:58.427835 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.450672 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:58.429168 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:13:58.451390 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.320924 28658 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:58.388159 28433 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.430451 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.385574 28554 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:13:58.453229 28543 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:58.453486 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.453832 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.455112 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:13:58.455798 27899 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:13:58.456152 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 242727us
I20260709 13:13:58.456318 22660 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.411037 28384 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:58.432942 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.451510 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.472107 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.472138 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:58.369047 28376 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.465070 22762 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13. Re-sending same reply.
I20260709 13:13:58.450228 28510 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:58.463948 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 76297.1us
I20260709 13:13:58.489859 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.457420 28219 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:58.435797 28639 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:13:58.472905 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.473069 22764 raft_consensus.cc:2410] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:13:58.468734 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 106615us
I20260709 13:13:58.408252 28537 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.485071 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.432209 27912 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.462471 28433 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.422854 28403 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.480611 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.467130 28515 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:58.432543 28651 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:58.530862 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.530786 22911 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:13:58.477415 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:58.531870 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 98061.8us
I20260709 13:13:58.471204 28384 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.500274 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:58.446123 28587 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:13:58.518170 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.557272 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.559157 22766 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:58.559918 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.560093 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.446473 28465 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.521471 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.523156 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.484246 28612 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:58.524428 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.570955 22836 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
W20260709 13:13:58.518236 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.539523 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.520257 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.542372 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.572757 22762 raft_consensus.cc:2376] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17. Re-sending same reply.
I20260709 13:13:58.572790 22915 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:13:58.572949 22906 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:13:58.468761 28654 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:13:58.559442 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.539428 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.470408 28658 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:58.466784 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 79490.9us
I20260709 13:13:58.574294 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.574384 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:13:58.548720 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.499363 27899 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:58.524271 28578 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:58.473073 28602 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:58.556152 28663 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:13:58.555371 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.576495 28530 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:58.555452 28079 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.583868 28546 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.538182 27912 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.438973 28655 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.553731 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:58.605759 22911 raft_consensus.cc:2410] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:13:58.587087 22915 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:58.606298 22915 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:58.525490 28660 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:58.594281 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:58.608601 23458 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:58.578406 28384 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.610142 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:58.594501 28613 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:58.631498 28431 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 13:13:58.636803 28504 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.602458 28612 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:58.608862 28568 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:58.608369 28660 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.675450 28660 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:58.650439 28666 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:13:58.658270 28657 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:13:58.659224 28665 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:13:58.664445 28631 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:58.670794 28403 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.616586 28546 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:13:58.616010 28613 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:13:58.606069 28670 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:58.608708 28654 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:13:58.601202 28530 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.674281 28655 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:58.708823 28661 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.247s	user 0.000s	sys 0.004s
I20260709 13:13:58.720068 28654 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:58.720626 28654 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:58.721674 28654 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.723336 28654 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:58.712531 28613 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:58.790416 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.791923 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 265753us
I20260709 13:13:58.799301 22763 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:58.577879 22764 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:13:58.919872 28675 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:58.800001 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2930) took 2458 ms. Trace:
I20260709 13:13:58.800644 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2931) took 2459 ms. Trace:
I20260709 13:13:58.585959 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:58.587760 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87666.2us
I20260709 13:13:58.800843 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:58.942652 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.942893 22766 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:58.943416 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2960) took 1003 ms. Trace:
I20260709 13:13:58.944069 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:58.944986 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 517421us
I20260709 13:13:58.589206 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:58.953862 22837 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
W20260709 13:13:58.814389 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 204185us
I20260709 13:13:58.956429 28602 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:58.807269 22660 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 3 to 8, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 8 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.957193 28680 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:58.958330 28538 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.958523 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.959306 28680 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:58.959764 22758 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:58.959942 28680 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:58.611723 22915 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.963598 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:58.964460 22915 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:58.827469 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3117) took 1478 ms. Trace:
I20260709 13:13:58.965711 22914 rpcz_store.cc:276] 0709 13:13:57.348549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.348765 (+   216us) service_pool.cc:224] Handling call
0709 13:13:57.443982 (+ 95217us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:58.003399 (+559417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.003454 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:58.004437 (+   983us) log.cc:844] Serialized 52 byte log entry
0709 13:13:58.192818 (+188381us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:58.212960 (+ 20142us) log.cc:844] Serialized 14 byte log entry
0709 13:13:58.213175 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.213339 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.826589 (+613250us) raft_consensus.cc:1685] finished
0709 13:13:58.826656 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.827077 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244582,"prepare.queue_time_us":23336,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":9633,"raft.queue_time_us":247155,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":9029,"thread_start_us":187903,"threads_started":1,"wal-append.queue_time_us":393562}
I20260709 13:13:58.672030 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1020) took 1324 ms. Trace:
I20260709 13:13:58.858242 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:58.861457 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.659646 27899 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.739262 28658 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.755698 28660 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.902938 28530 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.874429 28401 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.972090 22838 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:13:58.969388 23460 rpcz_store.cc:276] 0709 13:13:57.347921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.348087 (+   166us) service_pool.cc:224] Handling call
0709 13:13:57.348420 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:57.364737 (+ 16317us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:57.844975 (+480238us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b18003923e0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:13:58.037045 (+192070us) log.cc:844] Serialized 14 byte log entry
0709 13:13:58.350374 (+313329us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.350935 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.351105 (+   170us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:58.351145 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.351220 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.671644 (+320424us) raft_consensus.cc:1685] finished
0709 13:13:58.671771 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.671950 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6127,"prepare.queue_time_us":136702,"prepare.run_cpu_time_us":8691,"prepare.run_wall_time_us":257030,"raft.queue_time_us":16375,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":20338,"spinlock_wait_cycles":12071808,"thread_start_us":496858,"threads_started":2,"wal-append.queue_time_us":420552}
I20260709 13:13:58.930938 28554 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:58.984933 28554 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.931784 28604 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:13:58.938514 28638 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.810s	user 0.014s	sys 0.002s
I20260709 13:13:58.921155 22759 rpcz_store.cc:276] 0709 13:13:56.340802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.860256 (+1519454us) service_pool.cc:224] Handling call
0709 13:13:57.860841 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.349966 (+489125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.351318 (+  1352us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.369067 (+ 17749us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.798781 (+429714us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:58.798966 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.799119 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.799230 (+   111us) raft_consensus.cc:1685] finished
0709 13:13:58.799250 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.799612 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67460,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"thread_start_us":864367,"threads_started":2,"wal-append.queue_time_us":263511}
I20260709 13:13:58.806584 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:58.943339 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.713132 28546 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.921294 22757 rpcz_store.cc:276] 0709 13:13:56.341055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.827690 (+1486635us) service_pool.cc:224] Handling call
0709 13:13:57.828064 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.044797 (+216733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.048560 (+  3763us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.059960 (+ 11400us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.368101 (+308141us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:58.368140 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.368228 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.799898 (+431670us) raft_consensus.cc:1685] finished
0709 13:13:58.799943 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.800346 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99936,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":63,"thread_start_us":432110,"threads_started":2,"wal-append.queue_time_us":521179}
I20260709 13:13:58.969329 28537 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.977986 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2273) took 1193 ms. Trace:
I20260709 13:13:58.971047 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.980054 28660 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:58.979934 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.982316 22836 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:13:58.982018 28401 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:58.915947 28612 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:58.985917 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:58.943925 22762 rpcz_store.cc:276] 0709 13:13:57.939911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.573646 (+633735us) service_pool.cc:224] Handling call
0709 13:13:58.943311 (+369665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:58.919916 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:13:58.989634 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.985574 28554 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:58.977340 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:58.993273 22765 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:13:58.999835 28509 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:59.000159 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.001670 28401 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:58.989948 22835 rpcz_store.cc:276] 0709 13:13:57.784800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.788892 (+  4092us) service_pool.cc:224] Handling call
0709 13:13:58.977333 (+1188441us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54000c80d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:58.977702 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691773184}
I20260709 13:13:58.999943 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:59.002197 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:59.001227 28677 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:59.008585 28421 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:59.009063 28421 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.009457 28421 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:59.013842 28604 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.015034 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:59.016249 22765 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:13:59.006224 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2961) took 1062 ms. Trace:
I20260709 13:13:59.005841 22761 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:13:59.004702 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 198757us
I20260709 13:13:59.017845 22764 rpcz_store.cc:276] 0709 13:13:57.940206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.919251 (+979045us) service_pool.cc:224] Handling call
0709 13:13:59.002452 (+ 83201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.003502 22758 raft_consensus.cc:2376] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:13:59.020794 28680 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.026142 28680 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:59.025862 27912 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.028656 27912 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.029934 28677 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:58.993453 22660 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.031570 28677 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.017263 28604 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:58.927909 28675 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:59.065626 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.067679 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:13:59.068806 28675 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:59.069114 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 124970us
I20260709 13:13:59.044582 28546 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.046337 28079 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:59.047317 28214 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.682s	user 0.017s	sys 0.013s
I20260709 13:13:59.033258 22660 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 7, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 7 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.051339 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2355) took 2608 ms. Trace:
I20260709 13:13:59.052214 22759 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:13:59.053238 28376 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.053617 22760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46950
I20260709 13:13:59.084044 22763 rpcz_store.cc:276] 0709 13:13:56.443060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.049784 (+2606724us) service_pool.cc:224] Handling call
0709 13:13:59.050453 (+   669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.050882 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.050924 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:59.051006 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:59.051046 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.051102 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.051227 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.084704 22760 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:59.085173 22760 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:59.086292 22763 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:13:59.086673 22763 pending_rounds.cc:85] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:13:59.086989 22763 pending_rounds.cc:107] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:13:59.087293 22763 raft_consensus.cc:2889] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:59.056947 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.088056 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2959) took 1148 ms. Trace:
I20260709 13:13:59.090114 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.090957 23460 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:13:59.092444 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.088390 22763 rpcz_store.cc:276] 0709 13:13:57.939583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.085244 (+1145661us) service_pool.cc:224] Handling call
0709 13:13:59.085733 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.087778 (+  2045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.087987 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:59.060750 28510 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.012s
W20260709 13:13:59.061450 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 411856us
W20260709 13:13:59.062410 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.068825 22757 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:13:59.023581 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75277.7us
I20260709 13:13:59.021996 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2952) took 1081 ms. Trace:
W20260709 13:13:59.072695 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47188us
I20260709 13:13:59.099776 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:13:59.101581 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.101724 28214 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.103518 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.107862 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:13:59.110496 22915 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:13:59.107699 22761 rpcz_store.cc:276] 0709 13:13:57.937293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.976326 (+1039033us) service_pool.cc:224] Handling call
0709 13:13:59.018414 (+ 42088us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.108737 22909 raft_consensus.cc:606] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.115168 22909 raft_consensus.cc:614] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:59.116225 22766 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:13:59.124766 22758 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:59.128284 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.130260 23457 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:59.124749 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.128542 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.127848 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:59.129287 28425 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.738s	user 0.004s	sys 0.014s
I20260709 13:13:59.150470 22760 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:59.140897 22732 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.570s (SENT)
I20260709 13:13:59.146347 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.130053 28431 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.140537 28214 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.178596 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.179250 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:59.130165 28634 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:59.186589 28504 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.547s	user 0.016s	sys 0.002s
I20260709 13:13:59.194181 22765 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:59.195063 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2357) took 1070 ms. Trace:
I20260709 13:13:59.195472 22765 rpcz_store.cc:276] 0709 13:13:58.125040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.141709 (+1016669us) service_pool.cc:224] Handling call
0709 13:13:59.193911 (+ 52202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.194754 (+   843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.195002 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.197084 22765 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:59.112452 28510 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.199703 28510 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.200600 22757 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:59.200207 28421 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.201328 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 793) took 1222 ms. Trace:
I20260709 13:13:59.202168 22833 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:59.201783 22757 rpcz_store.cc:276] 0709 13:13:57.978822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.115457 (+1136635us) service_pool.cc:224] Handling call
0709 13:13:59.144300 (+ 28843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.201040 (+ 56740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.201265 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.215946 28554 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.222218 28431 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.225330 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 794) took 1246 ms. Trace:
I20260709 13:13:59.149318 28425 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.225867 22761 rpcz_store.cc:276] 0709 13:13:57.979083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.113045 (+1133962us) service_pool.cc:224] Handling call
0709 13:13:59.204958 (+ 91913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.205372 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.205411 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:59.205468 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:59.225035 (+ 19567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.225113 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.225254 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.227744 28425 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:59.229751 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:59.230844 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 120434us
I20260709 13:13:59.231719 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.232704 28675 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:59.233228 28675 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.233707 28675 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:59.233848 28214 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:59.229768 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:59.234376 28214 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.235504 28214 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:59.235621 22806 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.886s (SENT)
I20260709 13:13:59.236099 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.237895 28633 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:59.237962 23431 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.746s (SENT)
I20260709 13:13:59.227761 22761 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:13:59.156464 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2945) took 2709 ms. Trace:
I20260709 13:13:59.243278 22760 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.157560 22759 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:59.197444 22765 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:13:59.239635 22761 pending_rounds.cc:85] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 5
I20260709 13:13:59.246197 22761 pending_rounds.cc:107] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:13:59.246529 22761 raft_consensus.cc:2889] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:59.250922 22762 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:13:59.251176 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46950 (request call id 2948) took 2326 ms. Trace:
I20260709 13:13:59.251572 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 801) took 1270 ms. Trace:
I20260709 13:13:59.252048 22762 rpcz_store.cc:276] 0709 13:13:57.980890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.999388 (+1018498us) service_pool.cc:224] Handling call
0709 13:13:59.251508 (+252120us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:59.250419 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 127253us
I20260709 13:13:59.251555 22760 rpcz_store.cc:276] 0709 13:13:56.917493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.053091 (+2135598us) service_pool.cc:224] Handling call
0709 13:13:59.243769 (+190678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.240589 22758 rpcz_store.cc:276] 0709 13:13:56.440539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.049616 (+2609077us) service_pool.cc:224] Handling call
0709 13:13:59.120726 (+ 71110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.126659 (+  5933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.148167 (+ 21508us) spinlock_profiling.cc:243] Waited 69.2 ms on lock 0x7b5400369550. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:59.150075 (+  1908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159256448}
W20260709 13:13:59.254751 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.079s (SENT)
I20260709 13:13:59.270891 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:13:59.256186 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 81626.4us
I20260709 13:13:59.253158 28504 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:13:59.278990 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 145996us
W20260709 13:13:59.288283 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 157257us
I20260709 13:13:59.288678 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2955) took 1340 ms. Trace:
W20260709 13:13:59.287127 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 141883us
W20260709 13:13:59.292160 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 170016us
I20260709 13:13:59.277020 28376 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:59.289341 22759 rpcz_store.cc:276] 0709 13:13:57.938311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.085088 (+1146777us) service_pool.cc:224] Handling call
0709 13:13:59.085555 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.247369 (+161814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.278501 (+ 31132us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b540036f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:59.278800 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160169728}
I20260709 13:13:59.287286 28400 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:13:59.273463 28079 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 13:13:59.299621 28631 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.003s
I20260709 13:13:59.300568 28554 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.302330 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:59.303452 28384 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.682s	user 0.017s	sys 0.003s
W20260709 13:13:59.304236 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.625s (SENT) [suppressed 57 similar messages]
I20260709 13:13:59.297644 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:13:59.308606 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.292227 22765 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:59.313989 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67917us
I20260709 13:13:59.292385 28431 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.298861 28376 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.297973 28509 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:59.324545 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.324469 22836 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:13:59.341008 28631 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:59.341554 28631 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.341917 28631 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:59.341566 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.337615 28079 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:59.344672 28079 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:59.345072 28079 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:59.346594 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.347259 23460 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:59.349963 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.351420 28658 raft_consensus.cc:2764] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:59.353030 23457 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:59.330155 28384 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.357673 28679 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.358399 28679 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:59.364828 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.369628 28616 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.370574 28616 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.375525 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.380132 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2360) took 1254 ms. Trace:
I20260709 13:13:59.381836 28421 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:59.387403 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.389997 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 90656.5us
I20260709 13:13:59.383030 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.325042 28376 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:59.398365 28679 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:59.401700 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 127213us
W20260709 13:13:59.404757 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.403522 22732 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.659s (SENT)
I20260709 13:13:59.412802 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.413015 22764 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:59.415071 22765 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:59.412768 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:59.380730 22766 rpcz_store.cc:276] 0709 13:13:58.126122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.144046 (+1017924us) service_pool.cc:224] Handling call
0709 13:13:59.193712 (+ 49666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.194073 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:59.194482 (+   409us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.322181 (+127699us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.379731 (+ 57550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:59.379807 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:59.379861 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.379939 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.380067 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16197,"prepare.queue_time_us":1523,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":21937,"spinlock_wait_cycles":959744,"thread_start_us":102360,"threads_started":1}
I20260709 13:13:59.425530 22911 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
W20260709 13:13:59.418366 28433 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.011s
I20260709 13:13:59.409729 28633 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.418200 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:59.429437 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.429561 22914 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:59.430481 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.411305 28384 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.442996 28654 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:59.464037 22914 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
W20260709 13:13:59.477408 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55489.7us
I20260709 13:13:59.479631 23460 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:59.473203 22837 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:59.459324 28554 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:59.491295 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:59.491582 22806 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.159s (SENT)
W20260709 13:13:59.494354 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.497610 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.499572 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:59.502895 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.501700 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:59.508558 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.485086 28504 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.496322 28554 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:59.514285 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.779s (SENT)
I20260709 13:13:59.457803 28433 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:59.515400 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 115930us
I20260709 13:13:59.510569 28613 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.519855 28431 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:59.530902 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2354) took 3098 ms. Trace:
I20260709 13:13:59.528415 22911 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:59.536970 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.531490 22764 rpcz_store.cc:276] 0709 13:13:56.432850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.054586 (+2621736us) service_pool.cc:224] Handling call
0709 13:13:59.055151 (+   565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.416961 (+361810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.530464 (+113503us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54003403d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:13:59.530787 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819070208}
I20260709 13:13:59.510617 28509 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:59.538898 22833 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:13:59.529855 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:59.488265 28633 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:59.528425 28695 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.129s	user 0.002s	sys 0.000s
I20260709 13:13:59.521685 28427 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:59.545737 22913 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:59.531948 28613 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.524109 28576 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:59.510452 28604 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:13:59.552357 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.559660 23460 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:59.520622 28554 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:59.552951 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.542620 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49092.7us
I20260709 13:13:59.552807 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 792) took 1574 ms. Trace:
I20260709 13:13:59.559868 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:13:59.520327 28504 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.582070 22907 raft_consensus.cc:2368] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
W20260709 13:13:59.591558 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52988.5us
W20260709 13:13:59.578328 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 101853us
I20260709 13:13:59.580703 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.578018 28693 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:13:59.578339 22732 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.820s (SENT)
I20260709 13:13:59.558939 28576 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:59.562220 28427 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:13:59.578527 28698 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:13:59.573400 22763 rpcz_store.cc:276] 0709 13:13:57.978541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.107317 (+1128776us) service_pool.cc:224] Handling call
0709 13:13:59.132508 (+ 25191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.213572 (+ 81064us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:59.214056 (+   484us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.552066 (+338010us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.552417 (+   351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:59.552470 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:59.552523 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.552634 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.552747 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133578,"prepare.queue_time_us":139563,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":111,"raft.queue_time_us":320,"raft.run_cpu_time_us":442,"raft.run_wall_time_us":136613,"thread_start_us":325148,"threads_started":2,"wal-append.queue_time_us":198939}
I20260709 13:13:59.586354 28433 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:59.577876 28400 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:59.613934 28693 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.629475 28651 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:59.630617 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52373.5us
I20260709 13:13:59.615918 28431 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:59.613318 28616 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:59.626245 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.245s (SENT)
I20260709 13:13:59.577488 28079 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.651376 22908 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:13:59.652563 28079 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:13:59.652794 22911 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:59.653836 28707 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:13:59.655741 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.635787 28431 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.637529 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:59.662425 22765 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:13:59.662827 22765 pending_rounds.cc:85] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:13:59.663028 22765 pending_rounds.cc:107] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:13:59.663242 22765 raft_consensus.cc:2889] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:59.664206 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43952.1us
I20260709 13:13:59.665174 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.669727 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2368) took 1405 ms. Trace:
I20260709 13:13:59.670121 22765 rpcz_store.cc:276] 0709 13:13:58.264519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.196403 (+931884us) service_pool.cc:224] Handling call
0709 13:13:59.196886 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.669452 (+472566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.669675 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:59.631461 28504 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:13:59.671643 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 161936us
I20260709 13:13:59.674500 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:13:59.675763 22732 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.872s (SENT)
I20260709 13:13:59.677863 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.678818 22836 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:59.680348 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.681053 22841 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:13:59.677727 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.667024 28431 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:59.689932 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:59.701164 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:59.702102 28427 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.713753 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:13:59.716034 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.681564 28546 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:59.719717 23458 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:59.720268 28633 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.721890 28633 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:59.723582 28576 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.725606 22833 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:59.726781 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:13:59.727325 22914 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:13:59.689406 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2002) took 1151 ms. Trace:
I20260709 13:13:59.738196 22907 rpcz_store.cc:276] 0709 13:13:58.527424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.527580 (+   156us) service_pool.cc:224] Handling call
0709 13:13:59.667081 (+1139501us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b54001d1d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:13:59.678591 (+ 11510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278110592}
W20260709 13:13:59.747752 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 69544us
I20260709 13:13:59.751523 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:59.757045 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72899.9us
W20260709 13:13:59.769822 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59656us
I20260709 13:13:59.749725 28651 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:59.772868 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.774257 28546 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:59.777626 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93866.4us
W20260709 13:13:59.780004 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 110982us
W20260709 13:13:59.781083 22732 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.909s (SENT)
I20260709 13:13:59.781522 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.782440 28651 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:59.783864 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.785589 28537 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:59.789629 22911 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:13:59.765790 28433 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:59.792101 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.767916 28576 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:59.794095 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 129080us
I20260709 13:13:59.794242 22913 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:13:59.768373 28693 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.774715 28546 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.796808 22833 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:13:59.810524 28675 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.011s
I20260709 13:13:59.796707 28537 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:13:59.815971 28214 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.002s
I20260709 13:13:59.816521 28214 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:59.817692 22732 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.035s (SENT)
I20260709 13:13:59.818300 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:59.820168 22732 connection.cc:256] RPC call timeout handler was delayed by -3.4010875199999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28750000000000003s for a response
W20260709 13:13:59.821065 22732 connection.cc:256] RPC call timeout handler was delayed by -3.3951732760001505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2537s for a response
W20260709 13:13:59.821641 23431 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.279s (SENT)
I20260709 13:13:59.796219 28504 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.826399 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.825578 28425 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.827687 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.827890 28425 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.811345 28675 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.828938 22832 raft_consensus.cc:2368] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18.
I20260709 13:13:59.803314 28546 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:59.839676 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.837899 23457 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:59.833783 22841 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:59.843879 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:13:59.838145 28708 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:13:59.845381 28433 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:59.846099 28433 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:59.847211 28433 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.848278 28376 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.849781 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.850591 28376 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:59.850611 28537 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:59.852397 22879 leader_election.cc:400] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18.
I20260709 13:13:59.852594 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.852833 22879 leader_election.cc:403] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:13:59.853564 28537 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:59.853812 28376 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18.
W20260709 13:13:59.854707 22806 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.457s (SENT)
I20260709 13:13:59.855034 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.824198 28633 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:59.826084 28651 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:59.858824 28376 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:13:59.855921 22732 connection.cc:256] RPC call timeout handler was delayed by -3.2544998339999438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
I20260709 13:13:59.791873 28631 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:13:59.883855 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.862392 22832 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:13:59.872649 22732 connection.cc:256] RPC call timeout handler was delayed by -2.9825875199999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.334s for a response
W20260709 13:13:59.897832 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 53737.5us
I20260709 13:13:59.874876 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:59.898538 22732 connection.cc:256] RPC call timeout handler was delayed by -2.9019732760002626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3085s for a response
I20260709 13:13:59.872376 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.834069 28431 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:13:59.839982 28425 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:59.910043 22732 connection.cc:256] RPC call timeout handler was delayed by -2.2283875200000693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4178s for a response
I20260709 13:13:59.918308 22835 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:13:59.859617 28376 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:59.859297 28554 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.910552 22841 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:13:59.875416 28633 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:13:59.940148 28633 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.943096 28214 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:13:59.832845 28504 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:13:59.925992 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104415us
I20260709 13:13:59.952073 28504 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.952814 28675 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:13:59.953018 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.953877 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.954712 28631 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:13:59.928041 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.933590 22732 connection.cc:256] RPC call timeout handler was delayed by -2.2143422160002046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
I20260709 13:13:59.876916 28613 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:13:59.955670 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.955780 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:13:59.910182 28510 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 13:13:59.955785 22732 connection.cc:256] RPC call timeout handler was delayed by -2.054142216000173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 13:13:59.957756 22732 connection.cc:256] RPC call timeout handler was delayed by -2.0001422160000857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 13:13:59.958055 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83391.8us
W20260709 13:13:59.958256 22732 connection.cc:256] RPC call timeout handler was delayed by -1.9776422160002767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 13:13:59.958575 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6518422160002046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
I20260709 13:13:59.957616 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:13:59.958731 28616 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:59.958863 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.959473 23466 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:13:59.959422 28616 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:13:59.954571 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.960384 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.960731 23464 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:13:59.958931 22732 connection.cc:256] RPC call timeout handler was delayed by -1.6203422160001537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 13:13:59.962859 28554 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:13:59.926671 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 102185us
I20260709 13:13:59.904184 28693 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:59.957212 28425 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:59.957160 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.956378 23463 raft_consensus.cc:2368] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18.
I20260709 13:13:59.967018 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:13:59.967773 22835 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:59.970887 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:13:59.972573 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.462s (SENT)
I20260709 13:13:59.971959 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:13:59.967027 28425 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:13:59.972451 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:13:59.978502 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 118248us
I20260709 13:13:59.980460 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:13:59.980798 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:13:59.970484 22841 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:13:59.982712 28376 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.983146 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:13:59.985231 28433 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:59.986029 28433 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:13:59.987251 28433 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:13:59.967196 28537 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:59.998229 23460 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:00.001176 28651 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:00.004694 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47612.8us
I20260709 13:13:59.980067 28079 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:00.010349 28079 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:00.011595 28079 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:00.013398 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.013550 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49740.4us
I20260709 13:13:59.978121 28425 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:00.016436 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.995302 28675 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:00.023900 28680 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.025969 28537 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:00.026129 28431 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:00.027128 28537 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:00.028030 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.960718 28616 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:00.027573 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67284.6us
I20260709 13:14:00.031433 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:13:59.995040 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.034934 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:13:59.989703 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:00.041826 22832 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:00.048074 28504 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:00.048008 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53224.3us
I20260709 13:14:00.048645 28504 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.049013 28504 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:00.050176 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.051826 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.049862 28546 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.053745 28431 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:00.054524 28431 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:00.055586 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83084us
W20260709 13:14:00.055770 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.056188 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.059389 22835 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:00.061333 22879 leader_election.cc:400] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18.
I20260709 13:14:00.062731 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:00.063560 22833 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:00.050148 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.070397 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.043763 28554 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:00.047886 28718 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:00.087183 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.088330 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:00.097219 23460 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:00.071647 28431 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:00.101689 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.205s (SENT)
I20260709 13:14:00.105116 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.106133 22911 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:00.117899 22913 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:00.078835 28509 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:00.117076 22833 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:00.083905 28693 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:00.097401 28530 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:00.133971 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 117679us
I20260709 13:14:00.103580 28431 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.136320 28693 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:00.139454 28693 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.145448 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.145797 28546 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:00.146780 28504 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.147524 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:00.148141 22757 raft_consensus.cc:2376] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16. Re-sending same reply.
I20260709 13:14:00.149000 28504 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:00.149861 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:00.150177 28651 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:00.150599 22760 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:00.151561 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:00.149189 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:00.151137 28530 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:00.152410 22757 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:00.162593 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:00.163502 22762 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 10.
W20260709 13:14:00.171993 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 118467us
I20260709 13:14:00.157255 22832 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:00.175510 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:00.177666 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:00.179216 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.151314 28613 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.182478 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:00.137718 28431 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:00.187608 22760 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:14:00.184396 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:00.189078 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.178296 22766 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15. Re-sending same reply.
W20260709 13:14:00.191907 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96322.6us
I20260709 13:14:00.193241 22758 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:00.191155 28433 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.190860 28530 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.194525 27899 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.194698 28433 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.195156 27899 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.197305 28433 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:00.199955 28425 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.203294 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:00.204109 22762 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:00.164572 28509 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:00.208062 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.219990 28425 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:00.219434 28509 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:00.224606 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 106166us
I20260709 13:14:00.206535 28633 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.225283 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:00.226042 22806 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.626s (SENT)
I20260709 13:14:00.237417 22762 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:00.245961 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.238404 22913 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:00.205267 28613 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.262912 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:00.266136 23431 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.534s (SENT)
I20260709 13:14:00.254753 27899 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:00.254173 28675 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:00.242291 28633 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.274310 28680 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:00.275198 28728 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:00.277988 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132698us
I20260709 13:14:00.259093 28731 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.288313 22758 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:00.276176 28634 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:00.296551 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.307085 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:00.309285 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47172.5us
W20260709 13:14:00.310237 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84227.2us
W20260709 13:14:00.310425 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.767s (SENT)
W20260709 13:14:00.311072 22806 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.730s (SENT)
W20260709 13:14:00.311949 23431 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.915s (SENT)
I20260709 13:14:00.312994 28633 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:00.310467 28613 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:00.315568 28576 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.002s
I20260709 13:14:00.316947 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:00.317638 23457 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:14:00.319514 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:00.320344 23464 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:00.302889 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.286468 28680 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.333115 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.334272 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:00.339920 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:00.342844 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.344856 23463 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:00.347018 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99887.2us
I20260709 13:14:00.350263 22835 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:00.331714 28537 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:00.337584 28680 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:00.365334 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.368170 23464 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
W20260709 13:14:00.377028 28736 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:14:00.359226 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:00.407944 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48847.6us
I20260709 13:14:00.411073 28616 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:00.411623 28616 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:00.412027 28616 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:00.412091 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54848.4us
I20260709 13:14:00.414186 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.424649 23457 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:00.428860 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.365685 28726 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:00.393049 28537 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:00.376161 28731 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:00.439851 22832 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:00.440353 22839 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:14:00.420008 28530 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:00.448999 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:00.453954 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:00.462029 28433 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:14:00.458900 28726 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:00.464919 28726 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.453778 28675 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.468739 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:00.469542 23463 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:00.473974 22839 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:14:00.475266 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61252.5us
I20260709 13:14:00.476574 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:00.480587 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58195.6us
I20260709 13:14:00.481797 28731 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.482339 28731 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.482741 28537 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.484737 28731 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:00.486821 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:00.487462 23463 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:00.488462 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:00.489132 22836 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:00.489553 22840 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:00.490911 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:00.491132 22832 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:00.492091 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:00.482321 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.498299 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.476835 28680 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.500381 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:00.500938 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:00.501139 22839 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:00.501613 22839 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:00.500042 28604 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:00.507105 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.508443 28651 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:00.509831 28745 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:14:00.512667 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 216537us
I20260709 13:14:00.523444 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:00.526021 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 189476us
W20260709 13:14:00.527184 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.801s (SENT)
I20260709 13:14:00.533838 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:00.502269 28727 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.519070 22839 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.500648 28731 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:00.540812 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.544669 28680 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:00.549454 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.549877 28738 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.000s	sys 0.003s
I20260709 13:14:00.551141 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.517599 28732 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:00.534212 28718 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:00.557988 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:00.567960 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.534099 28631 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:00.518365 28546 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:14:00.551297 28731 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.538419 28433 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.571540 28693 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:00.587888 28384 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:14:00.591544 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:00.598738 23463 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:00.561606 28554 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:00.583406 28631 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.562105 28718 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.589357 28731 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:00.605624 28631 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:00.614699 28744 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:14:00.596153 28634 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:00.621775 22911 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:00.624215 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:00.624639 22909 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:00.612030 28718 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:00.627991 22841 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:00.637882 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:00.639107 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 119684us
I20260709 13:14:00.601229 28433 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.625799 28728 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.656157 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.624744 28732 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:00.653770 23432 leader_election.cc:336] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.321s (SENT)
I20260709 13:14:00.679195 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.654474 28750 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.667603 28748 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:14:00.680944 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:00.683764 22907 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:00.641207 28431 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.689208 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:00.693379 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:00.694047 22915 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:00.693511 28750 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.698612 28751 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:00.699276 28751 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:00.703004 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:00.650823 28616 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.714929 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.706723 22909 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:14:00.706770 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109709us
I20260709 13:14:00.735850 28758 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:00.744481 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.753932 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 235636us
I20260709 13:14:00.755338 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.758594 22915 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:00.742561 28732 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:14:00.761318 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:00.770124 22908 raft_consensus.cc:2393] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:00.766407 28755 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:14:00.765950 28752 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:00.795661 28616 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:00.795552 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.753255 28754 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.826812 22912 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:00.802879 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.783084 28693 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:00.804608 28431 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:00.801188 28732 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:00.805151 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.769779 28509 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:14:00.765426 28757 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:14:00.832386 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 167786us
I20260709 13:14:00.833017 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:00.833945 22915 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:00.765362 27899 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:00.813022 28745 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:00.818397 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 197514us
I20260709 13:14:00.819665 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.822494 22907 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:00.818580 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:00.754014 28751 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:00.840811 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:00.839896 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 237744us
I20260709 13:14:00.847872 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:00.851475 22912 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:00.822033 28634 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:00.853389 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:14:00.861243 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.835466 28509 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:00.851634 28754 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:00.864833 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:00.871019 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:00.872355 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:00.872969 22759 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:00.874204 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:00.875582 28433 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:00.872848 22764 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15. Re-sending same reply.
I20260709 13:14:00.882431 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:14:00.883627 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.836440 28732 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:00.850287 28745 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:00.909278 22806 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.018s (SENT)
I20260709 13:14:00.896219 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:00.911631 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:00.913998 22912 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:00.888948 28433 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:00.892256 28728 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:00.913461 28766 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:00.956593 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50281.9us
W20260709 13:14:00.959623 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.959903 28728 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:01.006630 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111104us
I20260709 13:14:00.944547 28764 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:00.956228 28433 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:01.024353 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.155s (SENT)
I20260709 13:14:01.024921 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:01.024026 28764 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.027655 22835 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:01.035557 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:01.027325 28754 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:00.954576 28634 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.027400 28766 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:00.944972 28693 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.047650 22762 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:01.080005 28376 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:01.080526 28376 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:01.048481 28773 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:01.081493 28376 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.081457 28773 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:01.050179 28761 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:14:01.050195 22841 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:14:01.083432 28773 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.069788 28728 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.071336 28433 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.073138 28732 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.074436 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:01.080873 28680 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:01.083302 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:01.089062 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52755.1us
I20260709 13:14:01.095173 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:01.103086 22759 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 11.
I20260709 13:14:01.107224 22834 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:14:01.107019 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:01.117352 22838 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:01.083416 28376 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:01.120038 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74559.5us
I20260709 13:14:01.120812 22840 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:01.122406 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:01.123382 23457 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:01.123669 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:01.126104 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:01.125304 28693 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.128135 28604 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:01.128412 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:01.130257 28759 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:14:01.140002 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 115688us
I20260709 13:14:01.140033 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:01.115855 28680 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:01.142132 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.142467 22841 raft_consensus.cc:1217] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Deduplicated request from leader. Original: 13.7->[15.8-18.9]   Dedup: 15.8->[18.9-18.9]
I20260709 13:14:01.144373 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:01.147037 22841 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: [15.8-18.9]
I20260709 13:14:01.144106 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:01.147760 22909 raft_consensus.cc:2376] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16. Re-sending same reply.
W20260709 13:14:01.152019 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.152618 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:01.146443 22906 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15. Re-sending same reply.
I20260709 13:14:01.142940 28431 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:14:01.163102 22907 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:01.164709 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.225s (SENT)
I20260709 13:14:01.128424 28693 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.165818 28769 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:01.125927 28718 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.166466 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:01.167599 28764 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:01.168129 28718 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.166484 28769 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:01.169260 28769 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.175650 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.175916 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.176544 22762 raft_consensus.cc:2376] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16. Re-sending same reply.
I20260709 13:14:01.177654 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.178180 22909 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:01.178678 22764 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
W20260709 13:14:01.162932 28631 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.000s
I20260709 13:14:01.198437 28631 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.183966 22910 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:01.155843 28425 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:01.169202 28754 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.189716 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.190971 28745 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.191586 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:01.192447 28731 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 13:14:01.165913 28651 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:01.142957 28546 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:14:01.193346 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:01.193291 28772 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:14:01.205575 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.210434 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.217900 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.221673 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.222718 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.223307 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.219108 28651 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:01.223997 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:01.222703 22757 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:01.227046 22762 raft_consensus.cc:2368] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15.
I20260709 13:14:01.228240 22761 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:01.238808 28776 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:01.242285 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83504us
W20260709 13:14:01.243441 22806 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 3.165s (SENT)
I20260709 13:14:01.243906 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:01.245009 28760 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:14:01.245707 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45881.3us
I20260709 13:14:01.246728 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.248322 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.251461 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.259688 22757 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:01.266305 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.266896 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.268095 22764 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:01.268724 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.269285 22762 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.266999 22761 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.221567 28731 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.272688 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.273253 22760 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:01.269093 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.274514 22764 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:01.275035 22766 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:01.217475 28680 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.280470 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.281629 22766 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:14:01.284286 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.287056 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.294199 22766 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:01.296985 22762 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:01.298919 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.302158 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:01.309721 22766 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.310971 28727 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.311587 28727 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.314306 22841 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:14:01.315536 28756 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:01.300980 28745 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:01.302793 28780 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:14:01.317678 28778 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:14:01.322402 22766 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:14:01.325341 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55017.5us
W20260709 13:14:01.308881 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44570.8us
I20260709 13:14:01.327605 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.307271 28731 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.308843 28750 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:01.281433 28651 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.302326 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.333763 22764 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:01.331426 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.335501 22762 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 15: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:01.330739 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.295810 28680 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:01.338559 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:01.322980 28631 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:01.347283 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:01.347724 22766 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:01.332029 28756 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:01.354349 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.355515 22758 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:01.355641 28214 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:01.356823 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:01.358021 28214 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:01.359346 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.360132 28214 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:01.360929 22761 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:14:01.369649 28777 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:14:01.385742 28693 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:01.387627 22764 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:14:01.391357 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 131469us
I20260709 13:14:01.348850 28776 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:14:01.399513 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52394.7us
I20260709 13:14:01.391465 28651 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:01.430127 23466 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:14:01.432865 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.445s (SENT)
I20260709 13:14:01.433190 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:01.433532 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:01.433964 23466 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:01.392472 28731 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:01.352672 28727 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:01.437183 28421 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:01.441847 22806 leader_election.cc:336] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.690s (SENT)
I20260709 13:14:01.442413 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.443998 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.445137 28750 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:01.445664 28731 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:01.442761 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:01.449085 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.454316 23457 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:01.455881 28718 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:14:01.463357 28634 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:01.463630 28613 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:01.454018 28631 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:01.464205 28634 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:01.465523 28747 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.864s	user 0.013s	sys 0.000s
I20260709 13:14:01.467417 28634 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.470134 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:01.472361 28731 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.472877 28731 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.473609 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:01.474269 22834 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:01.475100 28731 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:01.486300 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.488898 28756 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:01.497992 23466 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:01.498456 28731 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:01.499751 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:01.506394 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.511817 28731 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:01.512243 28731 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.508153 22908 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:01.458942 28546 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:01.510879 22765 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:14:01.513118 28546 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.515307 28546 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:01.516844 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:01.517953 28421 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:01.518577 28421 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:01.529982 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.542411 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.543138 22758 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:01.545421 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:01.541239 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:01.492857 28728 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.554876 22763 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:01.538098 27899 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:01.545063 28726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.010s	sys 0.016s
W20260709 13:14:01.632395 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 146253us
I20260709 13:14:01.562697 28728 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.553905 28546 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:01.550763 28776 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:01.566843 22761 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:01.576365 28616 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 13:14:01.519829 28421 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.779951 22839 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:01.791991 28726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.324s	user 0.011s	sys 0.017s
I20260709 13:14:01.815126 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:01.816846 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:01.818662 22911 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:01.809983 28425 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:01.821920 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.821357 28425 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.826911 28425 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.828581 22908 pending_rounds.cc:85] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 6
I20260709 13:14:01.829277 22908 pending_rounds.cc:107] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:14:01.805814 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:01.778319 28376 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.008s
W20260709 13:14:01.789604 28769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.008s	sys 0.014s
I20260709 13:14:01.813235 28554 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:01.813525 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:01.813525 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 272352us
I20260709 13:14:01.862074 23466 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:01.805606 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:01.777292 28731 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.833796 28616 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:14:01.836927 28728 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:01.833832 28787 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.520s	user 0.006s	sys 0.004s
I20260709 13:14:01.829838 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:01.802971 27899 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:01.843108 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.830868 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:01.888242 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:01.847488 22908 raft_consensus.cc:2889] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.848822 22841 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:01.802565 28776 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.849186 28765 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.849546 28651 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.896030 28776 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.900480 28651 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.903458 28651 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.847443 28680 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:01.869671 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:01.871210 28554 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.873203 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.878332 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:01.879098 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.904867 22912 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:01.861164 28769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.009s	sys 0.014s
I20260709 13:14:01.892385 22837 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:01.896400 28728 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.908875 22837 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:01.920358 23460 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:01.921124 22834 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:01.923162 28728 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.941838 28728 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:01.944041 28731 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:01.946045 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83950.3us
I20260709 13:14:01.946914 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:01.948582 22660 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 242) took 72968 ms (client timeout 15000 ms). Trace:
I20260709 13:14:01.918957 28765 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.951437 28504 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.951973 28504 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:01.952334 28765 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:01.954383 28504 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:01.954947 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 85475.9us
I20260709 13:14:01.956858 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:01.959467 28776 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.919878 28680 raft_consensus.cc:2764] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:14:01.935345 22837 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:01.966300 22837 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:01.976089 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49111.9us
I20260709 13:14:01.977878 28554 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:01.981850 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:01.989568 22912 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:01.992470 22914 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
W20260709 13:14:01.992610 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 210471us
I20260709 13:14:01.987946 28680 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:02.009289 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.985633 28726 raft_consensus.cc:2764] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:14:02.015076 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.018630 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:02.020434 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63717.9us
I20260709 13:14:02.021757 28651 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:02.022193 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:02.023438 28764 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:02.023654 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:02.026185 28776 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:02.014374 28425 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:14:02.014660 28795 raft_consensus.cc:2764] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:14:02.015090 28765 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:02.043372 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96569.6us
W20260709 13:14:01.949108 22660 rpcz_store.cc:269] 0709 13:12:48.979961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.981718 (+  1757us) service_pool.cc:224] Handling call
0709 13:14:01.948430 (+72966712us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:59.519709 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:59.526132 (+  6423us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:59.526198 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:13:59.526235 (+    37us) tablet.cc:662] Decoding operations
0709 13:13:59.527803 (+  1568us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:59.527931 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:59.527962 (+    31us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:13:59.619472 (+ 91510us) tablet.cc:701] Row locks acquired
0709 13:13:59.631311 (+ 11839us) write_op.cc:260] PREPARE: finished
0709 13:13:59.675242 (+ 43931us) write_op.cc:270] Start()
0709 13:13:59.695706 (+ 20464us) write_op.cc:276] Timestamp: P: 1783602839667083 usec, L: 0
0709 13:13:59.695755 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:13:59.779035 (+ 83280us) log.cc:844] Serialized 4729 byte log entry
0709 13:13:59.942269 (+163234us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b
0709 13:14:00.274452 (+332183us) op_driver.cc:464] REPLICATION: finished
0709 13:14:00.448448 (+173996us) write_op.cc:301] APPLY: starting
0709 13:14:00.498534 (+ 50086us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:00.675776 (+177242us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:00.675821 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:00.736844 (+ 61023us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:00.741112 (+  4268us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:14:00.741189 (+    77us) write_op.cc:312] APPLY: finished
0709 13:14:00.765463 (+ 24274us) log.cc:844] Serialized 113 byte log entry
0709 13:14:00.798852 (+ 33389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:00.799034 (+   182us) write_op.cc:454] Released schema lock
0709 13:14:00.799106 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:00.799614 (+   508us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:00.799646 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":854452,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":162720,"apply.run_cpu_time_us":29668,"apply.run_wall_time_us":358072,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54311,"num_ops":12,"prepare.queue_time_us":245765,"prepare.run_cpu_time_us":34692,"prepare.run_wall_time_us":589571,"raft.queue_time_us":114212,"raft.run_cpu_time_us":45970,"raft.run_wall_time_us":94677,"replication_time_us":578470,"spinlock_wait_cycles":40844032,"thread_start_us":559191,"threads_started":4,"wal-append.queue_time_us":355996,"wal-append.run_cpu_time_us":17336,"wal-append.run_wall_time_us":1666372}]]}
I20260709 13:14:01.998215 28693 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:14:02.045286 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:02.003500 22839 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:02.046967 22807 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:02.049628 22835 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:02.050204 22839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 1965) took 1564 ms (client timeout 1972 ms). Trace:
I20260709 13:14:02.051282 28421 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:02.054307 22839 rpcz_store.cc:269] 0709 13:14:00.485780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.500074 (+ 14294us) service_pool.cc:224] Handling call
0709 13:14:00.654928 (+154854us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b1800020160. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1b2c646 00007f68a1a6658e 00007f68a1ae7f9e 00007f68a1ae97de 00007f68a1af527c 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c
0709 13:14:01.634439 (+979511us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b58008d0bd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a4d0aa 00007f68a1acf447 00007f68a1ae7fe7 00007f68a1ae97de 00007f68a1af527c 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c
0709 13:14:02.050130 (+415691us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192536704}
I20260709 13:14:02.056695 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.071805 22912 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:02.080308 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:02.048249 28751 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:02.110087 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57231.5us
I20260709 13:14:02.112258 22880 leader_election.cc:400] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15.
I20260709 13:14:02.112632 22880 leader_election.cc:403] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:14:02.113361 28728 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15.
W20260709 13:14:02.117767 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:14:02.118739 22649 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44261.2us
I20260709 13:14:02.123714 27899 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:02.042017 28763 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:02.141757 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:02.049203 28765 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:02.152616 28765 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:02.103574 28504 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:02.165985 28504 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:02.167047 28504 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:02.150100 28804 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:02.165198 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:02.153301 28765 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:02.144236 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:02.186344 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45420.5us
W20260709 13:14:02.186309 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 165158us
I20260709 13:14:02.193035 22835 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
W20260709 13:14:02.195426 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 184557us
I20260709 13:14:02.211012 22764 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:02.215758 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:02.216646 22759 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:02.227447 22762 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:02.199551 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:02.239086 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.248545 22915 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:02.261391 22760 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:02.105194 28751 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:02.212244 28634 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:02.251382 28809 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:14:02.277583 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:02.278856 23457 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:02.283054 28214 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:02.277329 28376 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:02.283999 28376 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:02.284432 28376 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:02.285449 28214 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:02.286612 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.279345 22759 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:02.291069 28751 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:02.257189 28756 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:02.301489 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:02.274539 28634 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.317097 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:02.322296 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 298715us
I20260709 13:14:02.319482 28634 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:02.316520 28756 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:02.386175 22732 leader_election.cc:400] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:02.388177 28766 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:02.408030 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 281520us
I20260709 13:14:02.408052 28376 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:02.421166 28376 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:02.436515 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.441397 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:02.447449 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.459677 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:02.460454 22837 raft_consensus.cc:2376] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14. Re-sending same reply.
I20260709 13:14:02.460598 23458 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:02.463542 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.388865 28766 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:02.447224 28504 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.473263 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63092.3us
I20260709 13:14:02.469156 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.477300 22835 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:02.479324 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:02.480055 28815 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:14:02.483784 28504 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:02.490370 28504 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:02.491232 28766 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:02.498679 22658 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:02.477191 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 264787us
I20260709 13:14:02.502125 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:02.503235 28728 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.503732 28728 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:02.504350 23458 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:14:02.513041 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 56031.7us
I20260709 13:14:02.516502 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:02.517318 23458 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:02.514055 28728 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:02.523814 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:02.522652 23463 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:14:02.530843 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.532871 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.533183 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.534866 23460 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:02.518173 28726 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:14:02.541343 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.542656 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.519861 28727 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.544597 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65370.3us
I20260709 13:14:02.546336 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:02.552722 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.567728 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:02.543532 28795 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:02.545547 28726 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.579329 22908 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:02.556335 28834 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:02.582243 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.580682 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:02.507186 28756 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:02.559286 28651 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:02.562592 28728 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:02.656410 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69122.4us
W20260709 13:14:02.646631 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66100.6us
W20260709 13:14:02.621140 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 159638us
I20260709 13:14:02.672832 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:02.675426 28651 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:02.579028 28726 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:02.566259 28769 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:14:02.677521 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:02.678470 28839 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:14:02.580075 28504 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:02.802027 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 124653us
I20260709 13:14:02.802289 28504 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.802803 28504 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:02.803578 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:02.679994 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.805769 28795 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:02.556583 28727 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:02.584368 28751 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:14:02.578810 28827 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:14:02.674593 28546 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:02.729923 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.730792 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.812112 22907 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:02.731338 28384 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.732270 28840 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:02.729990 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.571744 28818 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.156s
I20260709 13:14:02.801990 28554 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:02.736261 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:02.742285 28805 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:14:02.720340 22658 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:02.746167 28634 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:02.716485 28728 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:02.749856 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:02.768698 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.769863 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:02.779502 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:02.789762 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 204221us
W20260709 13:14:02.796080 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 134046us
I20260709 13:14:02.798233 28604 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.753111 28425 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:14:02.807960 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:02.808086 28727 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:02.917833 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 109958us
I20260709 13:14:02.829870 22909 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:02.831041 22757 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:14:02.921129 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:02.831276 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:02.922360 22761 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:02.832283 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:02.921160 22912 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:14:02.931653 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.923278 22836 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:02.921850 22757 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:02.828516 28854 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.970755 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 200981us
I20260709 13:14:02.834621 28818 consensus_queue.cc:1243] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:03.010524 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.014357 22757 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:02.968266 22765 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:03.019743 22840 raft_consensus.cc:2368] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
W20260709 13:14:03.020087 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104065us
I20260709 13:14:03.015300 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:03.038165 28850 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:14:03.008996 28727 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:14:03.042833 28837 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:02.930655 28680 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:02.951426 28604 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.050585 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.038725 28833 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:14:02.970767 28840 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:02.906443 28400 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.063373 22761 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:03.063340 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:02.968001 28554 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:02.906190 28384 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.248448 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.249338 22761 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:03.010825 28546 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:03.149530 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.168758 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:03.170620 22806 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.736s (SENT)
I20260709 13:14:03.254220 22759 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:03.254272 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:03.170270 22658 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.173399 28651 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:03.179265 28634 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:03.019124 28854 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.050540 28731 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.237242 28554 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:03.182170 28756 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.268200 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.211953 28838 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:03.211663 28776 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:03.278936 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 108385us
I20260709 13:14:03.280016 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:03.282785 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ad66b5c9842437d9320507b92b78fd5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:03.284116 22761 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:03.283273 28554 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:03.206997 28680 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:03.286985 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:03.299628 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 234559us
W20260709 13:14:03.281615 28852 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:14:03.247807 28400 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.294473 28776 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.295406 28421 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:03.296777 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.296866 28510 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:03.297410 28751 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:03.246547 28727 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:03.277240 22759 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:03.317955 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:03.335814 22914 raft_consensus.cc:2368] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
W20260709 13:14:03.319537 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.332683 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.334250 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.287173 28827 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:03.331725 22757 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:03.342410 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.343210 22759 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18. Re-sending same reply.
I20260709 13:14:03.340734 22909 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:03.345150 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.345922 22759 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:03.347002 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.347815 22759 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:03.348713 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.349088 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 840) took 1047 ms. Trace:
I20260709 13:14:03.349932 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.350637 22914 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:03.351166 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.351781 22913 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:03.351815 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:03.352377 28776 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:03.352811 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.349318 22759 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18. Re-sending same reply.
W20260709 13:14:03.353578 28858 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:14:03.354067 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.354480 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.354709 22759 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:03.355558 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.355726 22906 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:03.356101 22759 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:03.356662 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.331844 28509 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:03.358647 28509 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.359169 28728 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:03.360894 28854 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:03.361804 22658 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 14, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 14 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.299839 28838 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.364320 22658 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.286296 28756 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.369982 22910 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:03.371564 28756 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:03.373546 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:03.374435 22908 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:03.375734 22908 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:03.378868 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.379637 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:03.380620 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.349586 23464 rpcz_store.cc:276] 0709 13:14:02.301749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.307004 (+  5255us) service_pool.cc:224] Handling call
0709 13:14:03.348876 (+1041872us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400375354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:03.349023 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156090240}
I20260709 13:14:03.381903 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:03.382712 22909 raft_consensus.cc:2368] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:03.362591 28214 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:03.353991 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.381254 22757 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
W20260709 13:14:03.384306 22833 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.009s	sys 0.014s
I20260709 13:14:03.283661 28731 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:03.385847 22913 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 13:14:03.385995 22833 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.014s
I20260709 13:14:03.392118 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:03.389536 22909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "023c6d83f2864eedb2775b615045955e"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:03.394368 22909 raft_consensus.cc:606] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:03.402150 22761 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:14:03.403376 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.404036 22757 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:03.400743 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.406307 22911 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:03.382593 28509 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:03.412592 22835 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.014s	sys 0.035s
W20260709 13:14:03.415761 22835 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.014s	sys 0.035s
I20260709 13:14:03.391429 28795 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:03.415033 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.418653 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.419312 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.419984 22759 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:03.422267 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.419605 22757 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
W20260709 13:14:03.414330 22762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.012s	sys 0.029s
W20260709 13:14:03.425369 22762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.012s	sys 0.030s
I20260709 13:14:03.434607 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.435472 22759 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18. Re-sending same reply.
I20260709 13:14:03.443481 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.447716 22914 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:03.449645 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.467070 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.467419 22757 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:03.467792 22835 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:03.454859 28425 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.470957 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.470767 28425 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:03.471740 22760 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:03.472592 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.456215 22658 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.463831 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.465797 22764 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:03.476720 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.466253 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.466583 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.478580 22760 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:03.474898 28731 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:03.528216 23466 pending_rounds.cc:85] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 11
I20260709 13:14:03.528713 23466 pending_rounds.cc:107] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:14:03.529038 23466 raft_consensus.cc:2889] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:03.537959 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.531071 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:03.538658 22760 raft_consensus.cc:2376] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:14:03.536249 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:03.544550 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.477068 28425 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:03.552704 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.555447 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.558354 22761 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:03.561902 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.565711 22760 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 13:14:03.541706 28726 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.000s
I20260709 13:14:03.525513 28765 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:03.568744 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:03.562862 22759 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:03.541545 28384 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:03.607956 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:03.638535 22761 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:03.548972 28830 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:03.435031 28795 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.644419 22759 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:03.574309 28731 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:03.674986 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 196567us
I20260709 13:14:03.684625 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 840) took 1126 ms. Trace:
I20260709 13:14:03.695588 22762 rpcz_store.cc:276] 0709 13:14:02.557808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.557972 (+   164us) service_pool.cc:224] Handling call
0709 13:14:02.568939 (+ 10967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.587040 (+ 18101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.587076 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.657033 (+ 69957us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.426374 (+769341us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:03.426419 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.426492 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.684321 (+257829us) raft_consensus.cc:1685] finished
0709 13:14:03.684346 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.684525 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17670,"raft.queue_time_us":109698,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":6657,"thread_start_us":611509,"threads_started":1,"wal-append.queue_time_us":893616}
I20260709 13:14:03.698611 27899 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:03.699335 27899 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.685309 28680 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:03.574653 28867 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:03.691427 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.693617 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.004s	sys 0.028s
W20260709 13:14:03.709429 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.004s	sys 0.028s
W20260709 13:14:03.709829 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 915546us
I20260709 13:14:03.709959 22915 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:03.694566 22757 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:03.530885 28829 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.177s
I20260709 13:14:03.645550 28651 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:03.726929 22757 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:03.727779 22762 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:03.727135 22759 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:14:03.728776 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 339176us
I20260709 13:14:03.732856 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.742137 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:03.748198 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.748688 22914 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:03.715092 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:03.575244 28726 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:03.747761 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2333) took 1166 ms. Trace:
I20260709 13:14:03.733760 22909 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:03.764281 27899 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:03.764848 22763 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:14:03.789686 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:03.793464 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43950.3us
I20260709 13:14:03.690927 28765 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:03.799903 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 496236us
I20260709 13:14:03.654098 28830 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.745262 28651 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.747264 22761 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:03.802686 23462 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:03.725879 28867 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:03.831594 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:03.839957 22914 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:03.800438 23458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.021s	sys 0.054s
I20260709 13:14:03.823880 22763 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:03.811403 22833 rpcz_store.cc:276] 0709 13:14:02.568791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.569977 (+  1186us) service_pool.cc:224] Handling call
0709 13:14:02.581311 (+ 11334us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:02.649897 (+ 68586us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:02.722887 (+ 72990us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.298113 (+575226us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b1800374da0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04557 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:03.388718 (+ 90605us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:03.398640 (+  9922us) log.cc:844] Serialized 78 byte log entry
0709 13:14:03.399108 (+   468us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:03.402686 (+  3578us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.403090 (+   404us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.405376 (+  2286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.434073 (+ 28697us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.457306 (+ 23233us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b1800374da0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:03.734661 (+277355us) raft_consensus.cc:1685] finished
0709 13:14:03.734699 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.734913 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126041,"prepare.queue_time_us":78053,"prepare.run_cpu_time_us":173,"prepare.run_wall_time_us":2226,"raft.queue_time_us":79227,"raft.run_cpu_time_us":588,"raft.run_wall_time_us":80167,"spinlock_wait_cycles":26121728,"thread_start_us":665310,"threads_started":2,"wal-append.queue_time_us":193171}
I20260709 13:14:03.841084 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:03.841830 23458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.021s	sys 0.054s
I20260709 13:14:03.791102 28731 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:03.802305 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.000s	sys 0.013s
W20260709 13:14:03.854600 22806 leader_election.cc:336] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.519s (SENT)
W20260709 13:14:03.855115 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.001s	sys 0.013s
I20260709 13:14:03.855293 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:03.855576 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 834724us
I20260709 13:14:03.816866 28680 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:03.861850 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.862767 22913 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:03.822167 28876 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:03.843624 22914 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:03.845494 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:03.874049 22906 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:03.847304 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:03.847723 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:03.868708 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.872207 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.880096 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:03.862303 28829 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:03.889818 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132347us
I20260709 13:14:03.858523 28739 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:03.898674 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.899500 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.900401 22835 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
W20260709 13:14:03.904597 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57420.6us
I20260709 13:14:03.899667 22841 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:03.908272 22835 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:14:03.906528 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.610s (SENT)
I20260709 13:14:03.915271 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:03.813292 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71196.1us
I20260709 13:14:03.923475 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:03.878360 28854 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:03.873652 28876 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:03.930074 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.880934 28510 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:03.937768 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:03.944635 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:03.944942 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49195.8us
I20260709 13:14:03.943977 22832 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:03.945515 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:03.946079 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:03.946696 23463 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:14:03.883203 28868 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:03.893878 28680 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:03.956290 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:03.894681 28829 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.959489 22913 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:14:03.866586 28731 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.962440 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:03.788512 22658 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 8 to 9, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 9 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.943145 28425 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:03.938500 28854 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.938541 28816 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:03.984196 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77711.5us
W20260709 13:14:03.965154 28504 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.159s	user 0.014s	sys 0.002s
I20260709 13:14:03.999178 28504 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.001745 28868 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:04.003775 28868 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:03.946535 28876 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:04.010825 28504 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:03.911670 28739 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:03.950264 28878 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:03.960582 28384 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:03.964550 28829 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:03.989988 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44005.5us
I20260709 13:14:03.968791 28843 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:04.026058 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 80385.3us
I20260709 13:14:03.949410 28769 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:14:03.968933 28879 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:03.975122 28425 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.955349 28510 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.032461 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:04.033154 22913 raft_consensus.cc:606] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:04.033640 22913 raft_consensus.cc:614] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:04.017141 28816 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:03.995890 28854 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:04.027444 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:04.043053 28864 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 13:14:04.055100 23430 connection.cc:256] RPC call timeout handler was delayed by -1.0466217299999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:14:04.048590 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 146618us
I20260709 13:14:04.034751 28425 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:04.067955 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:04.052314 28879 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:04.029631 28384 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.088027 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:04.088925 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 207412us
I20260709 13:14:04.031828 28878 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.091046 22907 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 19 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:14:04.090889 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45259.8us
I20260709 13:14:04.091849 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:04.091949 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:04.057749 28726 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.066126 28773 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:04.100988 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:04.103324 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105111us
I20260709 13:14:04.066751 28837 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:14:04.105443 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:04.088863 28765 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.109051 27899 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:04.111763 27899 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:04.109719 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:04.102095 28880 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:14:04.081351 28791 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:14:04.073478 28865 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:14:04.093858 28795 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.095813 28651 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.118345 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:04.095325 28843 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:04.133774 28843 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.136102 28843 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:04.137017 28877 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:14:04.139943 28893 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:04.100255 28879 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.140887 28879 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.142977 28384 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.102262 22658 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:04.086995 28845 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 13:14:04.144637 28888 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:04.144438 22658 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 8 to 16, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 16 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:04.148381 22658 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 13 to 14, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 14 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:14:04.154469 28885 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:14:04.110715 28869 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:14:04.115757 27899 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:04.145347 28554 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.828s	user 0.011s	sys 0.000s
I20260709 13:14:04.168015 28554 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.169675 28791 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:04.170136 28791 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.170297 28554 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.170522 28791 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:04.149598 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:04.158099 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 847) took 1242 ms. Trace:
I20260709 13:14:04.102716 28867 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:04.149537 28871 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:14:04.181968 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.161162 22906 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:04.164698 28845 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:04.187103 28845 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:04.188313 28845 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.166428 28878 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.111027 28837 consensus_queue.cc:1243] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 18.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:14:04.171888 22758 rpcz_store.cc:276] 0709 13:14:02.915629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.915790 (+   161us) service_pool.cc:224] Handling call
0709 13:14:02.916148 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:03.011772 (+ 95624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.011813 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:03.169216 (+157403us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b1800430cc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a
0709 13:14:03.478439 (+309223us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:03.548800 (+ 70361us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.941578 (+392778us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:03.943813 (+  2235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.943889 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.157855 (+213966us) raft_consensus.cc:1685] finished
0709 13:14:04.157876 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.158032 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19746,"raft.queue_time_us":106870,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":14148,"spinlock_wait_cycles":94619776,"thread_start_us":293457,"threads_started":1,"wal-append.queue_time_us":533990}
I20260709 13:14:04.192862 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.199308 28845 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:04.217099 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84392.4us
I20260709 13:14:04.143226 28879 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.162365 28504 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:04.138827 28816 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.102633 28773 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.162909 28765 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:04.163086 28680 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:04.237630 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:04.238204 28867 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:04.238672 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:04.291149 22762 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:04.239583 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1059) took 1063 ms. Trace:
I20260709 13:14:04.292160 23460 rpcz_store.cc:276] 0709 13:14:03.176370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.177626 (+  1256us) service_pool.cc:224] Handling call
0709 13:14:03.178286 (+   660us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:03.178805 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.178843 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:03.179571 (+   728us) log.cc:844] Serialized 52 byte log entry
0709 13:14:03.637950 (+458379us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:03.638521 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.638597 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.239210 (+600613us) raft_consensus.cc:1685] finished
0709 13:14:04.239247 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.239458 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4895,"raft.queue_time_us":2011,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":14317,"thread_start_us":428407,"threads_started":1,"wal-append.queue_time_us":608678}
I20260709 13:14:04.236442 28843 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:04.223523 28893 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:04.277231 28862 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.510s	user 0.001s	sys 0.001s
I20260709 13:14:04.277621 28868 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:04.277841 28903 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:04.279879 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 2988) took 1239 ms. Trace:
I20260709 13:14:04.216185 28376 raft_consensus.cc:2764] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:14:04.332208 28765 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:04.153604 22658 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 11 to 14, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 14 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.276602 28876 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.303117 28879 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.304252 28728 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:04.305408 28884 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:14:04.250778 28830 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.306073 28384 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.308964 22906 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:04.309098 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1058) took 1133 ms. Trace:
I20260709 13:14:04.329699 28816 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:04.311311 28739 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:04.312878 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:04.322357 28838 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.326020 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:04.326233 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 88732.1us
I20260709 13:14:04.334404 22764 rpcz_store.cc:276] 0709 13:14:03.040764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.050313 (+  9549us) service_pool.cc:224] Handling call
0709 13:14:04.279655 (+1229342us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:04.279780 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321321728}
I20260709 13:14:04.286203 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.236791 28504 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:04.335891 28843 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:04.337292 28888 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:14:04.367193 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 295170us
I20260709 13:14:04.309896 28827 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:04.375101 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 274215us
I20260709 13:14:04.289841 28731 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.409041 22913 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 11: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:04.370680 23458 rpcz_store.cc:276] 0709 13:14:03.176044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.180130 (+  4086us) service_pool.cc:224] Handling call
0709 13:14:03.180459 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:03.214318 (+ 33859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.214368 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:03.229124 (+ 14756us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.844790 (+615666us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:03.844998 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.845069 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.307893 (+462824us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b2400396098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea42fb0 00007f68a16b9488 00007f68a16b9be8 00007f68a1691398 00007f68a168c76a 00007f68a168bbb6
0709 13:14:04.308671 (+   778us) raft_consensus.cc:1685] finished
0709 13:14:04.308703 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.309015 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33381,"raft.queue_time_us":76025,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":24650,"spinlock_wait_cycles":31274240,"thread_start_us":562803,"threads_started":1,"wal-append.queue_time_us":692960}
I20260709 13:14:04.354482 28510 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.468495 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:04.469347 22758 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:04.479192 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.484041 22764 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:04.488068 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:04.372270 28604 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.343322 28868 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:04.511346 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48628us
I20260709 13:14:04.525341 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:04.411180 28739 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.410506 28504 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:04.528671 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.559360 28739 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.559952 22760 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:04.561086 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:04.563199 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 495327us
I20260709 13:14:04.527794 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:04.514504 22763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.004s
I20260709 13:14:04.430230 28898 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:04.370395 28879 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.561707 22760 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:04.574193 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.575426 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:04.566941 22763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.011s	sys 0.005s
I20260709 13:14:04.355515 28903 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.334331 28893 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.513101 22762 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:04.463690 28554 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.500303 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3022) took 1082 ms. Trace:
I20260709 13:14:04.591084 22761 rpcz_store.cc:276] 0709 13:14:03.417803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.835049 (+417246us) service_pool.cc:224] Handling call
0709 13:14:03.835444 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:03.835680 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:04.069246 (+233566us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.499846 (+430600us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.499918 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:04.499984 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.500077 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.500222 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37685,"prepare.queue_time_us":159121,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":128,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":35110,"thread_start_us":586619,"threads_started":2}
I20260709 13:14:04.463407 28865 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.577268 22764 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:04.601943 22764 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:14:04.572237 28862 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:14:04.576174 22760 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:04.580083 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:04.372525 28728 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.466956 28769 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:14:04.573900 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2425) took 1235 ms. Trace:
I20260709 13:14:04.616164 22759 rpcz_store.cc:276] 0709 13:14:03.338317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.730205 (+391888us) service_pool.cc:224] Handling call
0709 13:14:03.730592 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:03.731054 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.731084 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:03.746328 (+ 15244us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.954853 (+208525us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:04.108281 (+153428us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.573555 (+465274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.573655 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.573700 (+    45us) raft_consensus.cc:1685] finished
0709 13:14:04.573713 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.573840 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11043,"prepare.queue_time_us":142586,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":37,"raft.queue_time_us":58299,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":26537,"thread_start_us":746280,"threads_started":3,"wal-append.queue_time_us":299966}
I20260709 13:14:04.378255 28384 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.624914 22759 raft_consensus.cc:1217] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 13.7->[15.8-19.10]   Dedup: 15.8->[17.9-19.10]
I20260709 13:14:04.470316 28827 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.570511 28898 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:04.638294 22911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.000s	sys 0.016s
W20260709 13:14:04.633448 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.015s	sys 0.062s
I20260709 13:14:04.464555 28843 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:04.638463 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58471.4us
I20260709 13:14:04.625658 22759 pending_rounds.cc:85] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:14:04.644080 22759 pending_rounds.cc:107] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:14:04.644438 22759 raft_consensus.cc:2889] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:04.509456 28425 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.640081 22758 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:04.589735 28731 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:04.645686 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.015s	sys 0.062s
I20260709 13:14:04.654662 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:04.655133 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 563321us
I20260709 13:14:04.655344 23464 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:04.645972 22911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.000s	sys 0.016s
I20260709 13:14:04.609840 28693 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.498136 28765 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.655865 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:04.663146 22730 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:04.663439 22730 leader_election.cc:403] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:04.660614 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 841) took 1133 ms. Trace:
I20260709 13:14:04.634382 28739 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:04.664227 23466 rpcz_store.cc:276] 0709 13:14:03.527234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.527383 (+   149us) service_pool.cc:224] Handling call
0709 13:14:03.527890 (+   507us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:03.549273 (+ 21383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.549324 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:03.550315 (+   991us) log.cc:844] Serialized 78 byte log entry
0709 13:14:03.879252 (+328937us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:03.879356 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.879490 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.304388 (+424898us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:04.648560 (+344172us) raft_consensus.cc:1685] finished
0709 13:14:04.648596 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.660534 (+ 11938us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":702,"raft.queue_time_us":1368,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":12993,"spinlock_wait_cycles":6703360,"thread_start_us":232067,"threads_started":1,"wal-append.queue_time_us":453720}
W20260709 13:14:04.681896 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 156692us
I20260709 13:14:04.682097 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:04.682919 22732 leader_election.cc:400] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15.
I20260709 13:14:04.642756 28898 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:04.646054 28837 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:04.687525 22765 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:04.614014 28554 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.711620 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:04.712116 22757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.047s	sys 0.094s
I20260709 13:14:04.712301 22906 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:14:04.712657 22757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.047s	sys 0.094s
W20260709 13:14:04.686167 22806 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.061s (SENT) [suppressed 32 similar messages]
I20260709 13:14:04.430249 22658 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 10 to 13, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 13 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.703135 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 846) took 2116 ms. Trace:
W20260709 13:14:04.709558 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:04.716696 23464 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:04.623358 28865 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.718971 22766 rpcz_store.cc:276] 0709 13:14:02.571495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.571613 (+   118us) service_pool.cc:224] Handling call
0709 13:14:02.571987 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:02.783766 (+211779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.783888 (+   122us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:02.791150 (+  7262us) log.cc:844] Serialized 52 byte log entry
0709 13:14:03.391378 (+600228us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:03.567825 (+176447us) log.cc:844] Serialized 14 byte log entry
0709 13:14:03.574163 (+  6338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.574257 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.276512 (+702255us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:04.662691 (+386179us) raft_consensus.cc:1685] finished
0709 13:14:04.686959 (+ 24268us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.687237 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115155,"prepare.queue_time_us":441887,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":113976,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":39335,"spinlock_wait_cycles":12111744,"thread_start_us":740056,"threads_started":3,"wal-append.queue_time_us":830411}
I20260709 13:14:04.730386 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:04.706308 28833 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.256s	user 0.011s	sys 0.017s
I20260709 13:14:04.742803 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:04.704970 28680 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.706566 28400 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:04.752956 28910 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:04.756253 22838 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:04.727563 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3020) took 1331 ms. Trace:
W20260709 13:14:04.738842 28833 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.294s	user 0.011s	sys 0.017s
W20260709 13:14:04.773362 28917 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:14:04.774672 28894 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:14:04.779124 28865 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.779856 28865 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:04.793358 22658 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.685200 28693 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.740444 28731 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:04.790848 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 108052us
I20260709 13:14:04.832072 28893 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.832809 22732 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:04.833148 28879 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:04.834856 28845 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
I20260709 13:14:04.803619 28868 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:04.743374 28795 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:14:04.758680 28876 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.770131 28680 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.817970 28773 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:04.818578 28854 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.747s	user 0.014s	sys 0.000s
W20260709 13:14:04.819253 28829 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.779s	user 0.003s	sys 0.011s
I20260709 13:14:04.822222 28384 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.844640 28425 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.825853 28728 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:04.828783 28908 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:14:04.846118 28829 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:04.830473 28554 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:04.833164 22732 leader_election.cc:403] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:04.843441 28680 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:04.877486 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 233227us
I20260709 13:14:04.882956 28773 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:04.894455 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2432) took 1002 ms. Trace:
I20260709 13:14:04.855201 28893 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
W20260709 13:14:04.905382 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 250918us
I20260709 13:14:04.906466 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:04.881237 28881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.007s	sys 0.012s
I20260709 13:14:04.844699 28854 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:04.954533 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 268384us
W20260709 13:14:04.975737 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 143068us
I20260709 13:14:04.977170 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:04.951990 22880 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:04.984556 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.985536 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:04.986241 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:04.929080 28881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.008s	sys 0.013s
I20260709 13:14:04.987008 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.011137 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:05.011680 22766 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:05.012286 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 2989) took 1969 ms. Trace:
I20260709 13:14:05.012739 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:05.013191 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:05.013448 22766 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:04.987861 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.014354 22761 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:05.012657 22765 rpcz_store.cc:276] 0709 13:14:03.043192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.049025 (+  5833us) service_pool.cc:224] Handling call
0709 13:14:04.687251 (+1638226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.710508 (+ 23257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.012065 (+301557us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400311d50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:05.012224 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3766635392}
I20260709 13:14:05.010622 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:05.014148 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.014219 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:05.010885 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:04.997211 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:04.883778 28773 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:04.932390 28879 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:05.019483 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:05.017319 22760 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:14:05.031224 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:04.974828 28510 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.037809 22732 leader_election.cc:400] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:05.043678 22732 leader_election.cc:403] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:14:05.007761 28838 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:05.014968 28893 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:05.022249 28731 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:05.029538 28879 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.026868 28816 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:05.040908 28919 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:14:05.068488 22766 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:14:05.056591 28791 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.883s	user 0.014s	sys 0.000s
W20260709 13:14:05.076282 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 413253us
I20260709 13:14:05.039355 28769 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.021191 28680 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:14:05.086519 22730 leader_election.cc:400] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
W20260709 13:14:05.086418 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58073us
I20260709 13:14:05.092142 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:05.087667 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:05.045373 28911 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:05.053459 28827 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.028126 28910 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:14:05.063977 28731 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.074376 28879 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:05.064507 28932 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:05.093269 22766 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:05.106204 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 361984us
I20260709 13:14:05.092789 28425 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:05.153944 28425 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.154382 28425 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:05.107307 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:05.101290 22730 leader_election.cc:403] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:14:05.101526 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 354020us
I20260709 13:14:05.157379 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.085696 28739 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:05.120383 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.053218 28510 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.049880 28837 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:05.162529 22840 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:14:05.090063 28878 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:05.091724 28651 raft_consensus.cc:476] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Not starting pre-election -- already a leader
W20260709 13:14:05.186391 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 100111us
I20260709 13:14:05.187291 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:05.171469 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.188380 23460 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:05.092569 28868 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.092726 28384 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:05.083629 28791 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.147413 28731 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:05.185385 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 78219.8us
I20260709 13:14:05.194965 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:05.197748 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:05.200855 28905 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:14:05.201352 28922 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.090859 28769 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.202423 28868 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:05.204243 28927 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:14:05.205231 28924 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:14:05.271759 23457 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:05.205964 28693 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:05.207188 28933 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:14:05.209542 28576 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.170948 28739 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.190121 28911 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
W20260709 13:14:05.215929 28928 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:14:05.122648 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.175618 28878 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.219089 28837 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.220570 28934 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:05.224260 28937 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:05.227623 28935 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:14:05.171837 28829 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:05.235872 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:04.771225 22757 rpcz_store.cc:276] 0709 13:14:03.395865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.800142 (+404277us) service_pool.cc:224] Handling call
0709 13:14:03.816258 (+ 16116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:03.816479 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:04.045220 (+228741us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.727195 (+681975us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.727255 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:04.727307 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.727380 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.727502 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54957,"prepare.queue_time_us":97259,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":113714,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"thread_start_us":820259,"threads_started":2,"wal-append.queue_time_us":822857}
W20260709 13:14:05.241694 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46814.9us
I20260709 13:14:05.298251 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:05.243932 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46319us
I20260709 13:14:05.201912 28943 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:05.251556 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 827) took 1514 ms. Trace:
I20260709 13:14:05.150372 28765 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:05.247242 22766 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:05.196723 28384 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.322405 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.273087 28854 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.249838 28376 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:05.162432 28932 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.311283 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:05.332410 22835 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:05.311764 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.316356 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.318050 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.262055 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:05.306512 22911 rpcz_store.cc:276] 0709 13:14:03.738446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.738622 (+   176us) service_pool.cc:224] Handling call
0709 13:14:03.739100 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:03.757764 (+ 18664us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:03.926778 (+169014us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.900753 (+973975us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:04.914119 (+ 13366us) log.cc:844] Serialized 26 byte log entry
0709 13:14:04.914345 (+   226us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:05.248508 (+334163us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.251056 (+  2548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.251178 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:05.251243 (+    65us) raft_consensus.cc:1685] finished
0709 13:14:05.251262 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.251472 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62366,"prepare.queue_time_us":336817,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":103,"raft.queue_time_us":176159,"raft.run_cpu_time_us":530,"raft.run_wall_time_us":35996,"thread_start_us":1321487,"threads_started":4,"wal-append.queue_time_us":849928}
W20260709 13:14:05.334122 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 72260.8us
I20260709 13:14:05.335568 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:05.337339 28827 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:14:05.325914 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87904.2us
W20260709 13:14:05.328251 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.310s (SENT)
W20260709 13:14:05.329072 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 93241.3us
I20260709 13:14:05.341331 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.255702 28425 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.291719 28878 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:05.374320 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.372354 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:05.367553 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.371061 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.366523 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.366273 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.289168 28934 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.384327 23463 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
W20260709 13:14:05.342918 28869 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:14:05.387480 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2428) took 1496 ms. Trace:
I20260709 13:14:05.306852 28911 raft_consensus.cc:2764] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:14:05.392614 28898 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.005s
I20260709 13:14:05.393106 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.256191 28922 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.293689 28576 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.398273 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.398883 28931 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:05.428540 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 93038.9us
I20260709 13:14:05.385416 28861 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:05.411927 28846 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:05.411834 22762 rpcz_store.cc:276] 0709 13:14:03.891231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.580182 (+688951us) service_pool.cc:224] Handling call
0709 13:14:04.626883 (+ 46701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.630683 (+  3800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.630733 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:04.643424 (+ 12691us) log.cc:844] Serialized 26 byte log entry
0709 13:14:04.953875 (+310451us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:04.953940 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.232932 (+278992us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b540035eb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:05.233016 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:05.387098 (+154082us) raft_consensus.cc:1685] finished
0709 13:14:05.387122 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.387411 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10219,"raft.queue_time_us":56951,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":16118,"spinlock_wait_cycles":7544576,"thread_start_us":249486,"threads_started":1,"wal-append.queue_time_us":409914}
I20260709 13:14:05.342798 28947 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:05.323668 28680 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:05.417480 27899 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:05.418728 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.420737 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.342525 28876 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:14:05.332732 28384 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:05.408704 28934 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:05.411175 28903 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.451071 28948 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.451763 28948 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:05.454227 28946 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:14:05.455726 28769 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:05.438146 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 97168us
I20260709 13:14:05.444458 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.423405 28931 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:05.448447 28942 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:05.478943 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:05.479929 22840 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:05.423130 28898 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.449086 28876 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:14:05.469779 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 155667us
I20260709 13:14:05.403916 28943 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:14:05.479220 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.438446 28947 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:05.437477 28846 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:05.444749 28827 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:05.491663 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:05.401891 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73671.8us
I20260709 13:14:05.504606 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.428745 28838 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:05.512195 22806 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.651s (SENT)
W20260709 13:14:05.522958 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 183303us
I20260709 13:14:05.523324 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.523942 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:05.525271 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.527743 22836 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:05.478282 28937 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.478336 28931 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:05.530709 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:05.541165 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:05.487289 28876 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:05.506165 28846 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:05.440855 28861 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.541963 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:05.564316 22915 raft_consensus.cc:606] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.564960 22915 raft_consensus.cc:614] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:05.547088 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.531019 28829 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:05.541934 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:05.506335 28947 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:05.555234 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.487012 28837 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 13:14:05.532351 28942 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:05.604444 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57467us
I20260709 13:14:05.552208 28867 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:05.556675 28376 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:05.605716 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:05.607189 23462 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:05.576303 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.610263 28956 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:05.611809 28977 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:05.614567 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:05.604933 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:05.551977 28937 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.624172 28963 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:05.624576 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69430.5us
I20260709 13:14:05.624883 28963 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.624081 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.628551 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.628978 22766 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:05.629189 22760 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
W20260709 13:14:05.653157 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48287.7us
I20260709 13:14:05.631664 22760 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:05.612789 28948 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.618216 28868 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.663769 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:05.671527 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.669915 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:05.665267 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.701982 28920 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:05.701740 28952 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:05.630578 28425 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:05.649376 28376 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:05.720832 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.666277 28964 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:05.723583 28967 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.657088 28854 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:05.705647 28938 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:05.743688 28980 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:14:05.724290 28981 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.743680 28963 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.725317 28865 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:05.744740 28954 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:14:05.747612 22837 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:05.685729 28903 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:05.622133 28838 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.683789 28576 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:05.760692 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 90915.3us
I20260709 13:14:05.723857 28898 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.684191 28932 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:05.767534 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.756153 28981 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:05.778955 28973 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:14:05.745229 28948 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:05.777491 28964 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:05.759378 28952 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:05.627223 28827 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.746107 28922 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.752784 28911 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:05.724852 28868 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:14:05.784895 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:05.786485 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:05.770417 28968 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:14:05.752631 28978 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.797938 28978 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.799840 28931 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.763193 28903 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.801582 28903 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:05.802538 28960 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:14:05.805351 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:05.806464 22910 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:05.806085 28952 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.809574 22915 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:05.760679 28943 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:05.809926 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:05.794782 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:05.811136 22837 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:05.811050 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:05.755409 28854 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.821149 28943 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.759315 28967 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.823390 28510 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.823650 28943 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.823992 28967 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.824786 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.825449 22835 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:14:05.826479 28854 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.828047 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.828734 22834 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
W20260709 13:14:05.830967 28731 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.007s
I20260709 13:14:05.831442 28731 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.812000 28911 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.832005 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:05.833806 28911 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:05.834322 28504 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.274s	user 0.006s	sys 0.013s
I20260709 13:14:05.834496 28911 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.834810 28504 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.835140 28911 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:05.791553 28948 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:05.838145 28948 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.840117 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.840823 22761 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:05.841786 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.843263 22840 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:05.845088 22840 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:05.848104 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.849148 22834 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:05.848800 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:05.898190 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44614.2us
I20260709 13:14:05.899734 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.901337 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.801040 28978 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:05.826471 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.913350 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:05.908125 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.915305 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:05.919359 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:05.921339 22908 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:05.921985 22761 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:05.923813 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.926756 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.922084 22908 raft_consensus.cc:606] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:05.928783 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:05.928036 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.929188 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:05.929118 22908 raft_consensus.cc:614] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:05.848186 28911 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:05.931600 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.915899 28932 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:05.939896 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.958473 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.958973 22837 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:05.925916 28376 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:05.926245 28943 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:05.970544 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:06.009877 22834 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:05.935035 28978 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:06.013123 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.937242 28975 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:06.014817 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:05.957055 28932 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:05.932119 28425 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:14:05.956553 28861 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.011786 28978 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:06.034564 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:05.969633 28943 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:06.035616 22908 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:06.040877 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:06.013095 28376 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:06.144819 22908 raft_consensus.cc:606] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:06.026260 28932 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:06.048879 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.053320 22913 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:06.152071 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.153296 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.154795 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:06.056038 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:06.149772 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:06.061026 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.078070 28816 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:06.080812 28739 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.179350 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.091707 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:06.101200 28843 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:06.111900 28876 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:06.036902 28978 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:06.122889 28879 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.955s	user 0.013s	sys 0.000s
I20260709 13:14:06.193789 28879 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.040033 28931 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:06.051527 28878 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.635s	user 0.012s	sys 0.004s
I20260709 13:14:06.195652 28878 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.196879 28879 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:06.198020 28878 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:06.136468 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.199034 22906 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:14:06.200376 22906 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:14:06.200699 28384 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.007s
I20260709 13:14:06.137101 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.137256 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.137966 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.138154 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:06.142531 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.023406 28975 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:06.147783 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:06.148710 22908 raft_consensus.cc:614] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:06.159564 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.205899 22907 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:06.160398 28929 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.068s	user 0.004s	sys 0.000s
I20260709 13:14:06.160808 28868 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:06.148401 28943 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:06.170778 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:06.179639 23462 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.004s
I20260709 13:14:06.212224 23462 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:06.087002 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:06.188926 28947 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.001s
I20260709 13:14:06.179131 28816 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:06.178833 28893 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:06.190260 28876 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:06.189862 28376 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.191802 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.203821 22914 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:06.206393 22913 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:06.201112 28384 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.234326 28731 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:06.211220 22766 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:06.188709 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:06.219151 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:06.219702 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:06.221369 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1067) took 1318 ms. Trace:
I20260709 13:14:06.237128 22761 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:06.237380 23458 rpcz_store.cc:276] 0709 13:14:04.903521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.903701 (+   180us) service_pool.cc:224] Handling call
0709 13:14:04.904112 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.046479 (+142367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.046530 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:05.046623 (+    93us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:05.046680 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.221145 (+1174465us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400093dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:06.221197 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.221317 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326789504}
I20260709 13:14:06.225912 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:06.230026 22907 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:06.231741 28981 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.232703 28940 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:06.245589 22911 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:06.233649 28425 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:06.208985 28975 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:06.240587 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:06.240517 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:06.241634 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:06.243536 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.245069 22912 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:06.251442 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.249393 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:06.257124 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:06.259526 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.220533 28893 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:06.254778 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.263734 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.265110 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.266316 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.269882 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.245843 28940 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:06.277355 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:06.278141 22762 raft_consensus.cc:2376] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17. Re-sending same reply.
I20260709 13:14:06.244501 28865 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:06.285624 28940 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.281800 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:06.263223 28914 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:06.294098 22912 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:06.295831 28957 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.670s	user 0.002s	sys 0.000s
W20260709 13:14:06.298275 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 257477us
I20260709 13:14:06.299089 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.303285 22761 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:06.303592 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.304257 22762 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:06.304519 22915 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:06.306262 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:06.312352 22915 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:06.219800 28947 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.317739 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:06.318219 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.314370 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:06.254266 28425 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:06.319892 22762 raft_consensus.cc:2410] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:14:06.321910 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:06.312587 28934 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.860s	user 0.012s	sys 0.012s
I20260709 13:14:06.325914 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:06.306792 28931 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:06.324414 28903 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.000s
I20260709 13:14:06.340420 22761 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:06.326443 28948 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:06.293020 28865 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.328035 28976 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:06.445385 22915 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:06.345539 28948 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:06.316743 28893 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.507158 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.336184 28934 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.453598 28680 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:06.347465 28903 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:06.534852 28993 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:14:06.459625 28974 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:06.525189 28995 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:14:06.507367 28948 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:06.542575 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:06.545198 22911 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:06.558543 22834 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:06.572494 22915 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:14:06.575904 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2447) took 2218 ms. Trace:
I20260709 13:14:06.576509 22758 rpcz_store.cc:276] 0709 13:14:04.357658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.662914 (+305256us) service_pool.cc:224] Handling call
0709 13:14:04.663288 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.027619 (+364331us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:05.072521 (+ 44902us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.422850 (+350329us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:05.423476 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:14:05.423662 (+   186us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:05.424071 (+   409us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.429051 (+  4980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.429135 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.292891 (+863756us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:06.575652 (+282761us) raft_consensus.cc:1685] finished
0709 13:14:06.575684 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.575835 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27375,"prepare.queue_time_us":59933,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":98662,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":37570,"spinlock_wait_cycles":28803328,"thread_start_us":680087,"threads_started":2,"wal-append.queue_time_us":537008}
I20260709 13:14:06.590471 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.586311 28865 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:06.595217 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1009) took 1066 ms. Trace:
I20260709 13:14:06.549533 28974 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:06.596487 22841 rpcz_store.cc:276] 0709 13:14:05.528531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.893057 (+364526us) service_pool.cc:224] Handling call
0709 13:14:05.893602 (+   545us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:05.894109 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.894140 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:05.920864 (+ 26724us) log.cc:844] Serialized 52 byte log entry
0709 13:14:06.126340 (+205476us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:06.126409 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.126520 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.594900 (+468380us) raft_consensus.cc:1685] finished
0709 13:14:06.594934 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.595131 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1242,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10555,"thread_start_us":166010,"threads_started":1,"wal-append.queue_time_us":277314}
I20260709 13:14:06.606388 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.612129 28984 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:06.663775 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:06.613723 28981 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:06.613015 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:06.664815 22912 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:06.613796 28934 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:06.647109 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.568936 28903 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.689945 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:06.691079 22912 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:06.691431 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 78509.8us
I20260709 13:14:06.696208 28984 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:06.698473 28994 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:14:06.699622 28981 consensus_queue.cc:1243] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:06.699388 28953 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:06.698045 28984 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:06.702775 28981 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:14:06.708014 28953 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:06.708464 28953 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:06.709425 28953 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.710796 28953 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:06.716503 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.721195 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 466171us
I20260709 13:14:06.692663 28425 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:06.734606 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 408789us
I20260709 13:14:06.740805 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:06.746995 29000 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:06.749444 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:06.754963 29000 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:06.745944 28953 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:14:06.761615 28953 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:06.763386 29000 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:06.766314 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:06.774741 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 857) took 1108 ms. Trace:
I20260709 13:14:06.774771 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:06.775370 22757 rpcz_store.cc:276] 0709 13:14:05.666836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.667041 (+   205us) service_pool.cc:224] Handling call
0709 13:14:05.721946 (+ 54905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.848027 (+126081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.848060 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:05.848102 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:05.848135 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.774449 (+926314us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400220bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:06.774504 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.774640 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289442176}
I20260709 13:14:06.722612 28903 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:06.724767 28947 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.780311 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:06.800065 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 77732.5us
I20260709 13:14:06.805013 22912 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:06.778116 28731 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:06.788002 28903 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:04.895139 22764 rpcz_store.cc:276] 0709 13:14:03.892285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.603155 (+710870us) service_pool.cc:224] Handling call
0709 13:14:04.603639 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.604052 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:04.636697 (+ 32645us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.893920 (+257223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.894010 (+    90us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:04.894079 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.894175 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.894368 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33568,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":1004,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":15540,"thread_start_us":255560,"threads_started":1}
I20260709 13:14:06.836428 28981 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:06.837237 28981 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:06.838507 28981 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.812968 28931 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:06.848255 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46784.7us
I20260709 13:14:06.835109 28731 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:06.851289 29000 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:06.918380 28791 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.888216 28974 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:06.929085 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 206193us
I20260709 13:14:06.933138 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:06.893653 28731 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.934445 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 95715.3us
W20260709 13:14:06.925457 28816 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.000s
W20260709 13:14:06.940887 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72856.4us
I20260709 13:14:06.925595 28893 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:06.947849 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61340.6us
I20260709 13:14:06.949795 28816 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.954963 28948 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:06.956911 28816 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:06.961275 29002 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:06.962235 28947 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:06.962170 29002 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.965919 29002 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.943060 28974 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.969436 28947 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:06.970310 28974 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.971414 29009 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:06.971998 29009 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:06.973169 29009 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:06.974210 29003 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:14:06.974475 29002 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.975219 29002 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:06.978653 29002 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:06.980793 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:06.982353 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:06.984668 28948 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:06.993628 28948 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:06.994560 28948 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:06.998086 28948 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:06.999032 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2459) took 1251 ms. Trace:
I20260709 13:14:06.999634 22763 rpcz_store.cc:276] 0709 13:14:05.747146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.748077 (+   931us) service_pool.cc:224] Handling call
0709 13:14:05.755402 (+  7325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.845381 (+ 89979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.845413 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:05.845452 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:05.845485 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.998681 (+1153196us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b540035dc54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:06.998756 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.998949 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187259136}
I20260709 13:14:07.003033 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:07.003942 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2365) took 1607 ms (client timeout 1788 ms). Trace:
W20260709 13:14:07.004304 22833 rpcz_store.cc:269] 0709 13:14:05.396213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.002734 (+1606521us) service_pool.cc:224] Handling call
0709 13:14:07.003878 (+  1144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:07.004587 28975 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.005s
I20260709 13:14:07.012760 22763 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:07.013877 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3046) took 1004 ms. Trace:
I20260709 13:14:07.015101 22763 rpcz_store.cc:276] 0709 13:14:06.009577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.001157 (+991580us) service_pool.cc:224] Handling call
0709 13:14:07.006439 (+  5282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.013461 (+  7022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.013794 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:07.019305 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.019s	sys 0.023s
W20260709 13:14:07.019719 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.019s	sys 0.023s
W20260709 13:14:07.007625 28846 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.429s	user 0.017s	sys 0.003s
W20260709 13:14:07.020046 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 649464us
I20260709 13:14:06.901830 28903 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:07.014535 28876 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.778s	user 0.015s	sys 0.000s
I20260709 13:14:07.007305 29002 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.942425 28731 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.009150 28974 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:07.040369 23431 connection.cc:256] RPC call timeout handler was delayed by -1.1299602219999088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
I20260709 13:14:07.055066 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:06.941715 28931 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.029022 28967 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.030097 29011 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:07.012462 28975 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:07.077329 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 93836.6us
I20260709 13:14:07.042652 29000 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.041329 29012 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:07.052295 29001 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.057806 28903 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.027228 28846 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.127611 29011 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:07.149631 28943 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.908s	user 0.017s	sys 0.000s
W20260709 13:14:07.169406 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.171617 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3028) took 1320 ms. Trace:
I20260709 13:14:07.172854 22758 rpcz_store.cc:276] 0709 13:14:05.851532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.580685 (+729153us) service_pool.cc:224] Handling call
0709 13:14:06.581478 (+   793us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.582362 (+   884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.582402 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:06.587620 (+  5218us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.897365 (+309745us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:07.041278 (+143913us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.047495 (+  6217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.047635 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.171359 (+123724us) raft_consensus.cc:1685] finished
0709 13:14:07.171382 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.171544 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5725,"prepare.queue_time_us":30548,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":7364,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":8846,"thread_start_us":420444,"threads_started":2,"wal-append.queue_time_us":337473}
I20260709 13:14:07.049965 29002 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.150549 29018 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.151059 28791 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:07.126880 28967 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.157558 29011 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.138161 29000 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:07.200278 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 145376us
I20260709 13:14:07.200754 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3025) took 1371 ms. Trace:
I20260709 13:14:07.201370 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.201233 22762 rpcz_store.cc:276] 0709 13:14:05.828905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.329212 (+500307us) service_pool.cc:224] Handling call
0709 13:14:06.329727 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.330327 (+   600us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:06.444963 (+114636us) log.cc:844] Serialized 14 byte log entry
0709 13:14:06.723730 (+278767us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:06.724811 (+  1081us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.725007 (+   196us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:06.725317 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 13:14:06.728832 (+  3515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.728929 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.200509 (+471580us) raft_consensus.cc:1685] finished
0709 13:14:07.200534 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.200687 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121352,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":3580,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":49330,"spinlock_wait_cycles":108108672,"thread_start_us":232241,"threads_started":1,"wal-append.queue_time_us":356581}
I20260709 13:14:07.204522 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.206387 22839 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 11.
I20260709 13:14:07.191017 28791 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.210843 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.148579 29001 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.214422 22839 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:07.214751 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:07.221568 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.222447 22841 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
W20260709 13:14:07.215741 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.215158 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.183818 28943 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:07.209227 28932 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.046s	user 0.011s	sys 0.005s
I20260709 13:14:07.216007 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.204975 29024 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:07.231564 29024 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:07.233186 29024 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:07.226241 22838 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:07.211159 28978 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.020s	user 0.013s	sys 0.003s
I20260709 13:14:07.234788 28978 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.212683 28791 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:07.254021 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 121017us
I20260709 13:14:07.271360 28943 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.250396 29012 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:07.197873 29018 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:07.282398 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71659.3us
I20260709 13:14:07.273671 28975 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:07.286204 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 109061us
I20260709 13:14:07.242865 28932 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.291356 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:07.292374 23464 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:07.295193 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:07.295310 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:07.299691 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:07.294236 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.311110 23464 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:07.314340 28504 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.284186 28731 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:07.316277 29025 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:14:07.320648 29022 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:07.323772 29018 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.324301 22909 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:07.326575 28932 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:07.329145 28967 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.313313 28791 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.336431 28791 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.343760 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.287689 28846 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.350914 28773 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.354032 28903 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.354799 28903 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.360014 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2460) took 1612 ms. Trace:
I20260709 13:14:07.360524 22765 rpcz_store.cc:276] 0709 13:14:05.747494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.747795 (+   301us) service_pool.cc:224] Handling call
0709 13:14:05.755743 (+  7948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.843503 (+ 87760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.843536 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:05.843577 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:05.843880 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.359711 (+1515831us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.359780 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.359941 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201330560}
I20260709 13:14:07.366353 28937 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.371727 29002 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.365155 29012 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.382017 28923 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:07.383081 28923 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:07.384413 28923 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.401914 28903 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.402757 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1012) took 1851 ms. Trace:
I20260709 13:14:07.402556 28504 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.404407 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:07.403156 22840 rpcz_store.cc:276] 0709 13:14:05.551594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.896305 (+344711us) service_pool.cc:224] Handling call
0709 13:14:05.896584 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.400933 (+1504349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.400967 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.401013 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:07.401058 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.402494 (+  1436us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003228d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.402546 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.402693 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3459273472}
I20260709 13:14:07.406061 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.408823 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:07.411993 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.414047 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3220) took 1168 ms. Trace:
I20260709 13:14:07.414557 22906 rpcz_store.cc:276] 0709 13:14:06.245637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.245767 (+   130us) service_pool.cc:224] Handling call
0709 13:14:06.250995 (+  5228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.453264 (+202269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.453313 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:06.453376 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:06.453421 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.413730 (+960309us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001e32d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.413799 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.413969 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464465280}
I20260709 13:14:07.416910 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.417918 22906 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:07.418183 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.409600 28504 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.424823 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:07.429251 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 1997) took 1662 ms. Trace:
I20260709 13:14:07.430853 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3221) took 1184 ms. Trace:
I20260709 13:14:07.432911 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2051) took 1297 ms. Trace:
I20260709 13:14:07.458386 22911 rpcz_store.cc:276] 0709 13:14:06.246219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.246519 (+   300us) service_pool.cc:224] Handling call
0709 13:14:06.246779 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.549961 (+303182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.430503 (+880542us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54001257d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.430763 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685639936}
I20260709 13:14:07.432734 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.436568 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3030) took 1584 ms. Trace:
W20260709 13:14:07.441422 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.442828 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 151433us
I20260709 13:14:07.462121 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:07.454768 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 155220us
I20260709 13:14:07.457309 22834 rpcz_store.cc:276] 0709 13:14:05.766493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.010973 (+244480us) service_pool.cc:224] Handling call
0709 13:14:06.026173 (+ 15200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.562944 (+536771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.428807 (+865863us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540019ebd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.429152 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221971200}
I20260709 13:14:07.459209 22909 rpcz_store.cc:276] 0709 13:14:06.135089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.144276 (+  9187us) service_pool.cc:224] Handling call
0709 13:14:06.144578 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.324805 (+1180227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.432584 (+107779us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540010f4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.432832 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712629504}
W20260709 13:14:07.464023 22806 connection.cc:256] RPC call timeout handler was delayed by -1.5246996750001927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
W20260709 13:14:07.465126 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.721s (SENT)
W20260709 13:14:07.466130 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4769996750001155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 13:14:07.466550 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4337996750002731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 13:14:07.466904 22806 connection.cc:256] RPC call timeout handler was delayed by -1.4175996750002469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 13:14:07.467275 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3662996750001639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:14:07.467617 22806 connection.cc:256] RPC call timeout handler was delayed by -1.3509996750001392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:14:07.468410 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.460899 22764 rpcz_store.cc:276] 0709 13:14:05.856851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.825926 (+969075us) service_pool.cc:224] Handling call
0709 13:14:06.826563 (+   637us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:07.019258 (+192695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.019302 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:07.031021 (+ 11719us) log.cc:844] Serialized 52 byte log entry
0709 13:14:07.129158 (+ 98137us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:07.139015 (+  9857us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.154469 (+ 15454us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.435528 (+281059us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b1800300b80. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:14:07.435894 (+   366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.436034 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.436243 (+   209us) raft_consensus.cc:1685] finished
0709 13:14:07.436268 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.436476 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45818,"prepare.queue_time_us":46735,"prepare.run_cpu_time_us":168,"prepare.run_wall_time_us":3056,"raft.queue_time_us":391975,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":14554,"spinlock_wait_cycles":16571008,"thread_start_us":284462,"threads_started":2,"wal-append.queue_time_us":181630}
I20260709 13:14:07.472432 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:07.472292 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.474085 22911 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:14:07.457989 28975 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.476122 28773 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:07.476594 28773 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:07.476338 28975 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.476994 28773 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.478585 29024 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:07.479063 28975 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.456826 28923 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.495929 28893 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:07.490851 28903 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.502792 22834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:14:07.483263 29014 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.503110 28903 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.503667 29014 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.502398 28893 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.500896 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.509330 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:07.506254 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.517191 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.517377 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:07.480592 28947 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.518965 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.520309 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.518002 22835 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:07.522799 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.524005 22840 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:07.521199 22838 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:07.525977 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:07.527001 22836 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:07.529898 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.525966 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.531505 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:07.530064 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.532157 23459 raft_consensus.cc:2376] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18. Re-sending same reply.
I20260709 13:14:07.491417 29024 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:07.532984 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:07.538237 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.539902 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:07.495378 29002 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.545794 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.550243 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:07.550323 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.553445 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.505914 29013 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:14:07.533870 28931 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:07.541958 29012 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:07.534121 29024 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:07.561898 29002 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.524200 28893 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.563614 28921 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:07.564409 28903 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.565315 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:07.531672 28910 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.531379 28975 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.542061 28846 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:07.571698 28854 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:07.572326 29011 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.533556 28942 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:07.576467 28981 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:07.508540 28765 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:14:07.502478 28967 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.531900 28947 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.581274 29014 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.584777 23462 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:07.585916 29033 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:07.586666 29037 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.599083 28921 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.554422 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:07.592397 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:07.595023 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2058) took 1457 ms (client timeout 1671 ms). Trace:
I20260709 13:14:07.588812 28978 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.599224 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:07.592895 28874 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:07.615700 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3029) took 1763 ms. Trace:
I20260709 13:14:07.544189 29030 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:07.620757 22832 raft_consensus.cc:2368] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:07.612947 28909 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:07.622403 22764 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.623291 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:07.622022 22913 rpcz_store.cc:269] 0709 13:14:06.137613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.153033 (+ 15420us) service_pool.cc:224] Handling call
0709 13:14:07.594757 (+1441724us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b540010efd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:07.594936 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120235776}
I20260709 13:14:07.625200 28868 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:07.625324 29011 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.625941 28942 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:07.631302 29002 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.618281 28967 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:07.632534 29004 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.728s	user 0.000s	sys 0.002s
I20260709 13:14:07.659255 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3024) took 1830 ms. Trace:
I20260709 13:14:07.632772 29024 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.614046 28910 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.634444 28931 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.634654 28773 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.666551 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3236) took 1059 ms. Trace:
I20260709 13:14:07.638851 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:07.622459 22757 rpcz_store.cc:276] 0709 13:14:05.851916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.780170 (+928254us) service_pool.cc:224] Handling call
0709 13:14:06.810322 (+ 30152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.007502 (+197180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.007554 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.020795 (+ 13241us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.225868 (+205073us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:07.258658 (+ 32790us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.615312 (+356654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.615392 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.615435 (+    43us) raft_consensus.cc:1685] finished
0709 13:14:07.615448 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.615621 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189724,"prepare.queue_time_us":32283,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":265917,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":24616,"thread_start_us":468814,"threads_started":3,"wal-append.queue_time_us":231144}
I20260709 13:14:07.646147 22764 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:07.668522 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.614099 28975 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.649936 28731 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:07.651793 28947 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:07.652374 28997 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.864s	user 0.000s	sys 0.006s
I20260709 13:14:07.641348 29030 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:07.641614 28868 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:07.657363 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2050) took 1607 ms. Trace:
I20260709 13:14:07.667810 22908 rpcz_store.cc:276] 0709 13:14:06.607480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.607688 (+   208us) service_pool.cc:224] Handling call
0709 13:14:06.608256 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.608561 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:06.916278 (+307717us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b18002c2780. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf
0709 13:14:06.916467 (+   189us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.666208 (+749741us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.666268 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:07.666314 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.666378 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.666494 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":609480,"prepare.queue_time_us":203175,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"spinlock_wait_cycles":46812032,"thread_start_us":351213,"threads_started":2,"wal-append.queue_time_us":283519}
I20260709 13:14:07.683907 22910 rpcz_store.cc:276] 0709 13:14:06.049763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.050101 (+   338us) service_pool.cc:224] Handling call
0709 13:14:06.060638 (+ 10537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.061068 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.061102 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:06.061679 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.315242 (+253563us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:06.728578 (+413336us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.656530 (+927952us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18002f05c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:14:07.656933 (+   403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.657017 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.657063 (+    46us) raft_consensus.cc:1685] finished
0709 13:14:07.657170 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.657301 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37676,"prepare.queue_time_us":450943,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":951754,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":199,"spinlock_wait_cycles":5262208,"thread_start_us":654229,"threads_started":2,"wal-append.queue_time_us":497853}
I20260709 13:14:07.659651 22766 rpcz_store.cc:276] 0709 13:14:05.828608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.326951 (+498343us) service_pool.cc:224] Handling call
0709 13:14:06.332114 (+  5163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.343361 (+ 11247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.343400 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:06.343924 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.855461 (+511537us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:07.012493 (+157032us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.013427 (+   934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.013564 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.631927 (+618363us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:07.658920 (+ 26993us) raft_consensus.cc:1685] finished
0709 13:14:07.659025 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.659194 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":71901,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":14106,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":36234,"spinlock_wait_cycles":50523904,"thread_start_us":526373,"threads_started":2,"wal-append.queue_time_us":525499}
I20260709 13:14:07.664181 28830 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.694654 28830 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:07.664746 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 669505us
I20260709 13:14:07.698464 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:07.665398 28838 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.637109 29000 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.718509 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.672927 28731 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:07.669392 23462 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:07.729581 28838 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.730731 28967 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.669639 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:07.671679 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.723204 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.734047 22764 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
W20260709 13:14:07.734454 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62850.2us
I20260709 13:14:07.735342 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.735419 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:07.665699 28942 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:14:07.680871 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:07.738003 22757 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:07.739213 22757 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:14:07.739554 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:07.683115 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:07.741128 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58154.8us
I20260709 13:14:07.670032 28931 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:07.672652 28554 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.674556 29011 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:07.678341 28874 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:07.677347 28975 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.756022 29011 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.688122 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:07.762122 29011 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.762467 22766 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:07.680250 28879 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.689379 28868 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.703202 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:07.706945 23465 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:07.672695 28932 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.623034 28921 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:07.722710 22913 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:07.723316 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.723623 28773 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:07.742601 28981 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:07.713492 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.727764 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:07.687028 29024 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.736609 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:07.783272 22908 raft_consensus.cc:2393] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:07.746861 28830 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.754547 22757 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:14:07.759183 28965 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:07.729159 28731 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.758991 29030 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:07.751276 29049 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:14:07.765702 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:07.750319 28843 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:07.828089 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:07.777595 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109820us
I20260709 13:14:07.727010 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.779259 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3234) took 1182 ms. Trace:
I20260709 13:14:07.749125 28893 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.756544 28934 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:07.838838 22757 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:07.846247 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3051) took 1804 ms. Trace:
I20260709 13:14:07.783424 28876 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.823807 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.821473 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:07.834614 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.830859 22915 rpcz_store.cc:276] 0709 13:14:06.596555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.597135 (+   580us) service_pool.cc:224] Handling call
0709 13:14:06.597652 (+   517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:06.615878 (+ 18226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.615926 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:06.634841 (+ 18915us) log.cc:844] Serialized 52 byte log entry
0709 13:14:06.829266 (+194425us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:06.840434 (+ 11168us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.760480 (+920046us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.778745 (+ 18265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.778875 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.778943 (+    68us) raft_consensus.cc:1685] finished
0709 13:14:07.778963 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.779174 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":866608,"prepare.queue_time_us":120278,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":152,"raft.queue_time_us":840332,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":221,"thread_start_us":206807,"threads_started":2,"wal-append.queue_time_us":220887}
I20260709 13:14:07.835008 28731 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.828317 28843 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:07.835752 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:07.845124 28984 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:07.857987 22908 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.860446 29001 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.836848 28384 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:07.836941 29048 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:14:07.837236 28893 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:07.837793 29040 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:07.882304 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3050) took 1848 ms. Trace:
I20260709 13:14:07.776096 28879 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.846781 22758 rpcz_store.cc:276] 0709 13:14:06.041650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.175424 (+1133774us) service_pool.cc:224] Handling call
0709 13:14:07.175935 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.481932 (+305997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.481980 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.489488 (+  7508us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.589338 (+ 99850us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:07.611764 (+ 22426us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.612019 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.612103 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.763044 (+150941us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b18002f7b20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f
0709 13:14:07.816022 (+ 52978us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b48002ac0d8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:14:07.845984 (+ 29962us) raft_consensus.cc:1685] finished
0709 13:14:07.846014 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.846177 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322539,"prepare.queue_time_us":21861,"raft.queue_time_us":307745,"raft.run_cpu_time_us":2584,"raft.run_wall_time_us":216822,"spinlock_wait_cycles":11171712,"thread_start_us":56429,"threads_started":1,"wal-append.queue_time_us":133876}
I20260709 13:14:07.763742 28920 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:07.764701 28554 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.825191 28868 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.879266 28934 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:07.880059 28938 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.841094 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:07.858089 28984 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:07.882977 22763 rpcz_store.cc:276] 0709 13:14:06.034061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.026539 (+992478us) service_pool.cc:224] Handling call
0709 13:14:07.027000 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.063338 (+ 36338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.063392 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.138737 (+ 75345us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.250650 (+111913us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:07.279800 (+ 29150us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.443211 (+163411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.584460 (+141249us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b180046b460. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:07.881812 (+297352us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:07.881901 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.881975 (+    74us) raft_consensus.cc:1685] finished
0709 13:14:07.881995 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.882210 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166665,"prepare.queue_time_us":25827,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":1418,"raft.queue_time_us":183048,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":73764,"spinlock_wait_cycles":162052224,"thread_start_us":63720,"threads_started":1,"wal-append.queue_time_us":130556}
I20260709 13:14:07.844081 28881 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:07.904954 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 147308us
I20260709 13:14:07.856648 28843 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:07.899930 29008 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.028s	sys 0.042s
I20260709 13:14:07.779148 28932 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.891304 28833 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:07.911058 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 145460us
I20260709 13:14:07.907732 22911 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.916056 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.899276 28920 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:07.918576 22914 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:07.927400 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:07.875473 28876 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.919037 28881 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.933120 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.934520 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:07.905061 29053 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:07.939478 28920 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.941058 28920 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.941126 28838 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:07.922544 29008 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.028s	sys 0.042s
I20260709 13:14:07.950979 22910 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:07.953567 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3053) took 1906 ms. Trace:
I20260709 13:14:07.953979 22762 rpcz_store.cc:276] 0709 13:14:06.047517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.205545 (+1158028us) service_pool.cc:224] Handling call
0709 13:14:07.440375 (+234830us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b5800002d98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:14:07.440597 (+   222us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:07.441050 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.441082 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:07.456102 (+ 15020us) log.cc:844] Serialized 78 byte log entry
0709 13:14:07.613965 (+157863us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:07.614007 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.614080 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.946187 (+332107us) raft_consensus.cc:1685] finished
0709 13:14:07.946213 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.953442 (+  7229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10328,"raft.queue_time_us":1175,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":14865,"spinlock_wait_cycles":3626752,"thread_start_us":122076,"threads_started":1,"wal-append.queue_time_us":194317}
I20260709 13:14:07.960456 22911 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:07.950143 28984 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:07.968385 28984 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:07.973814 28554 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.892637 29040 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.947265 28843 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:07.931449 29037 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:07.984664 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.998546 22911 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:07.987074 29047 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.361s	user 0.002s	sys 0.000s
I20260709 13:14:07.952862 28910 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.012717 22908 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:14:07.996174 29005 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.091s	user 0.000s	sys 0.006s
I20260709 13:14:07.977125 28879 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.019879 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.001794 29059 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:14:08.022058 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.023367 22839 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:08.027194 28881 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.012996 28923 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:08.029035 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.002990 28773 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:08.055189 28773 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:07.938696 29053 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.058908 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3026) took 2229 ms. Trace:
I20260709 13:14:08.060160 29053 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.914058 28938 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.059384 22761 rpcz_store.cc:276] 0709 13:14:05.829161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.345095 (+515934us) service_pool.cc:224] Handling call
0709 13:14:06.346059 (+   964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.346417 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.346446 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:06.370162 (+ 23716us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.821947 (+451785us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:06.927399 (+105452us) log.cc:844] Serialized 14 byte log entry
0709 13:14:06.927997 (+   598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.928096 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.980389 (+1052293us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:08.058536 (+ 78147us) raft_consensus.cc:1685] finished
0709 13:14:08.058672 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.058836 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1060,"prepare.queue_time_us":105222,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":23795,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":59783,"spinlock_wait_cycles":18399104,"thread_start_us":386975,"threads_started":2,"wal-append.queue_time_us":413145}
I20260709 13:14:08.060653 28931 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.029881 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.032202 28876 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.034468 28932 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:07.977922 29061 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 13:14:07.978753 28934 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.081854 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.062474 28938 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:08.052739 29059 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:07.995836 29040 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.064905 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.084739 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.085111 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.086236 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.087229 22832 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:14:08.088194 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.072497 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.054018 28923 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.078994 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.079370 28773 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.079913 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.083811 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.064692 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.086529 23457 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:08.093582 22841 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:08.073414 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:08.095444 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.097590 22832 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:08.098217 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:08.100261 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 103031us
I20260709 13:14:08.097391 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:08.104703 22841 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:08.109987 28504 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.112409 28923 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:08.115073 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.119298 22841 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:08.053906 29011 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.129961 22833 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:08.102810 29009 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:08.132678 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.134038 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.134523 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.135404 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.116110 29061 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:08.145891 29061 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:08.147439 22836 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:08.106429 28376 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:08.115847 28876 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:08.149209 28876 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.149760 28876 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:08.152621 22835 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:08.155193 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.155891 22833 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:08.157099 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.151576 28376 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:08.159179 28376 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.160712 28376 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:08.167723 22911 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:08.171640 23458 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:08.168905 28921 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.171564 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.174461 23465 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:14:08.176832 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.174783 28921 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.180111 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.184530 22910 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:08.184935 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:08.187476 22907 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:08.190335 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:08.192242 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.187495 29053 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:08.194070 28868 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:08.194180 29053 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.194746 28868 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.195264 28868 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:08.196123 28940 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:08.196705 29053 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.199060 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.192203 28920 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:08.201225 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.201900 23465 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:08.200980 28947 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:14:08.204614 28920 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:08.202663 28923 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 13:14:08.207654 28920 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.204994 28947 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:08.209162 28947 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:08.205830 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:08.214284 22839 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:08.200553 22807 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:08.224181 22907 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:14:08.228534 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.223685 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:08.245115 22834 raft_consensus.cc:606] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:08.256575 22834 raft_consensus.cc:614] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:08.222137 28816 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:08.224547 29061 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.210044 28947 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.232788 28923 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:08.257480 22837 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:14:08.257002 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:08.257813 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3238) took 1648 ms. Trace:
I20260709 13:14:08.262660 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.280627 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.264449 29013 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:14:08.267643 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 713645us
I20260709 13:14:08.282678 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.282583 22761 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:08.268671 28651 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:08.269105 28931 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.270601 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2445) took 4076 ms. Trace:
I20260709 13:14:08.271641 28948 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:08.272742 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 574390us
I20260709 13:14:08.277961 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.279857 22912 rpcz_store.cc:276] 0709 13:14:06.609833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.871783 (+261950us) service_pool.cc:224] Handling call
0709 13:14:07.595674 (+723891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.595958 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.595995 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.596056 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:07.596102 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.596181 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.257600 (+661419us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540010efd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:08.257740 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663593728}
I20260709 13:14:08.289561 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.310412 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.289878 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.314761 22836 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:08.278517 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 856) took 2616 ms. Trace:
I20260709 13:14:08.282819 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.289543 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.328258 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.328994 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.329263 22912 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:08.332943 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.337361 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.226528 29011 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.345927 22832 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:08.374464 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:08.273967 28816 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:08.273746 29046 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:08.388799 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 106181us
I20260709 13:14:08.388881 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.393160 22836 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:08.379913 28816 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:08.316323 22760 rpcz_store.cc:276] 0709 13:14:05.661701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.661836 (+   135us) service_pool.cc:224] Handling call
0709 13:14:05.662184 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.662399 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:05.666311 (+  3912us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.654164 (+1987853us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b18001e8800. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a169123a 00007f68a1afba2a
0709 13:14:08.277920 (+623756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.278026 (+   106us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:08.278091 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.278206 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.278429 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":547093,"prepare.queue_time_us":3734,"prepare.run_cpu_time_us":3086,"prepare.run_wall_time_us":22331,"raft.queue_time_us":552507,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":13497984,"thread_start_us":79688,"threads_started":1,"wal-append.queue_time_us":2163237}
I20260709 13:14:08.383452 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:08.384330 22838 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:08.402896 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.384261 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.386976 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:08.403895 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.395444 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:08.313126 22759 rpcz_store.cc:276] 0709 13:14:04.194686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.624371 (+429685us) service_pool.cc:224] Handling call
0709 13:14:04.624709 (+   338us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:05.053837 (+429128us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:05.291073 (+237236us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.502036 (+210963us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:05.510004 (+  7968us) log.cc:844] Serialized 52 byte log entry
0709 13:14:05.510180 (+   176us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:05.525006 (+ 14826us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.582478 (+2057472us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b180038bd20. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287
0709 13:14:07.583035 (+   557us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.269972 (+686937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.270243 (+   271us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.270304 (+    61us) raft_consensus.cc:1685] finished
0709 13:14:08.270324 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.270529 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":668203,"prepare.queue_time_us":61803,"prepare.run_cpu_time_us":195,"prepare.run_wall_time_us":7824,"raft.queue_time_us":990098,"raft.run_cpu_time_us":685,"raft.run_wall_time_us":82167,"spinlock_wait_cycles":11328896,"thread_start_us":962457,"threads_started":5,"wal-append.queue_time_us":388757,"wal-append.run_cpu_time_us":4525,"wal-append.run_wall_time_us":1379262}
I20260709 13:14:08.231772 28876 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.287883 29009 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:14:08.407357 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.387902 28923 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:08.407872 28876 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.408650 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.408872 28923 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.415356 29009 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:08.415885 29009 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:08.415441 28868 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.395951 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.418252 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.417928 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:08.419549 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.422410 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.417934 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.430764 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:08.431947 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:08.432576 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.435446 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.435923 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.438198 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.438951 22838 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:08.290814 28910 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.443850 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.286360 28942 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:08.441092 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.441188 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.442855 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.447430 22913 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:08.449026 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:08.277544 28984 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:08.281752 28974 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:08.449324 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:08.313375 29013 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:08.443365 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.450969 22762 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:08.452441 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.400887 29061 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:08.419713 28947 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:08.487671 22835 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:08.489099 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:08.490188 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:08.459169 22832 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:08.459389 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:08.447315 28942 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:08.460120 28893 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:08.460876 28974 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.448266 28921 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:14:08.448019 28504 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:08.462687 22839 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:14:08.463945 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.464177 29053 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:08.466172 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.466347 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:08.466841 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:08.470403 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:08.471575 29051 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.573s	user 0.005s	sys 0.001s
I20260709 13:14:08.472122 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.416962 29009 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.430334 28874 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:08.480203 23465 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:08.456874 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:08.527323 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55290.5us
W20260709 13:14:08.480962 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:14:08.528437 22649 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47564.9us
I20260709 13:14:08.400381 28931 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:08.451602 28910 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.493922 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:08.496501 22762 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:08.508565 28651 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:14:08.470294 29069 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:14:08.508438 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.541486 22758 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:08.543161 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.376469 28948 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:08.513868 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50026.4us
I20260709 13:14:08.549778 28876 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:08.509711 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:08.553580 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43975.6us
I20260709 13:14:08.554307 22757 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:14:08.554736 22757 pending_rounds.cc:85] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:14:08.554960 22757 pending_rounds.cc:107] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:14:08.555210 22757 raft_consensus.cc:2889] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:08.526310 23457 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:08.557060 28773 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:08.557677 28773 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:08.558565 28773 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.559209 29038 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:08.560971 29038 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:14:08.508078 29072 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:14:08.470295 28868 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.531625 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.508911 29002 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:08.491140 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.508700 28942 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:08.512332 28504 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:08.519783 29015 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:08.539893 28651 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:08.537700 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.538180 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:14:08.516594 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.550874 28910 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.525327 28874 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:08.551360 28940 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:08.551679 22765 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:08.505995 28909 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:08.584718 22762 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:08.593648 23457 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:08.587222 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:08.604955 22838 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:08.562340 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.505718 28984 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:08.562671 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.621351 22763 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:08.563659 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:08.621610 22758 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:08.516419 29059 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:08.571897 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:08.534487 28975 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:08.564499 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73474.8us
I20260709 13:14:08.533960 29018 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:08.627758 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:08.586822 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.614363 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:08.623818 22913 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:08.584571 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 868) took 1055 ms. Trace:
I20260709 13:14:08.573984 22757 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:08.602388 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.629009 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:08.637662 22730 leader_election.cc:400] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:08.637986 22730 leader_election.cc:403] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:14:08.512332 28893 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.630934 22766 rpcz_store.cc:276] 0709 13:14:07.528835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.763569 (+234734us) service_pool.cc:224] Handling call
0709 13:14:07.766197 (+  2628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:08.287085 (+520888us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:08.287463 (+   378us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.404837 (+117374us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:08.405410 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:14:08.489754 (+ 84344us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:08.490814 (+  1060us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.492194 (+  1380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.492277 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.566664 (+ 74387us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b180006bb00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:08.584311 (+ 17647us) raft_consensus.cc:1685] finished
0709 13:14:08.584346 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.584507 (+   161us) inbound_call.cc:177] Queueing success response
0709 13:14:08.590058 (+  5551us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b180006bb00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
Metrics: {"mutex_wait_us":520351,"prepare.queue_time_us":72546,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":87,"raft.queue_time_us":537768,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":26808,"spinlock_wait_cycles":26561152,"thread_start_us":116878,"threads_started":1,"wal-append.queue_time_us":91392}
I20260709 13:14:08.649395 22908 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:14:08.650197 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:08.565917 28504 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:08.648447 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:08.661260 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:08.660889 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89059.3us
I20260709 13:14:08.564812 28868 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:08.663589 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 76884.4us
I20260709 13:14:08.679052 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:08.686482 23458 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:08.686843 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.566928 29001 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:08.616421 28874 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:08.610234 28934 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:08.561484 29038 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:08.584283 28651 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:08.661744 29000 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:08.712558 29000 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:08.713018 29000 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:08.714058 28876 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:08.717170 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.693192 28765 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:14:08.693485 28868 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.692946 28967 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:14:08.703948 28731 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:08.719741 28651 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.720862 28868 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.699148 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.647140 28931 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:08.687413 28910 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:08.722843 28651 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.722038 28934 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.630363 28909 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.701704 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:08.755115 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.729822 28765 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:08.730574 28731 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:08.593297 29013 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.677373 28978 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:08.680805 28865 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:08.685959 29058 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:08.773484 28942 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.775177 22836 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:08.776420 28934 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.776607 28942 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.779224 28893 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.783004 28816 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:08.743510 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 46022.6us
I20260709 13:14:08.563844 29002 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:08.746323 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:08.649091 28941 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:08.570068 29015 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.706974 29053 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:08.761096 22766 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:08.749949 28967 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:08.730624 29001 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:08.652751 28975 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.737730 28940 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:14:08.772946 29013 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:08.747910 28910 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.785034 28984 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.788267 22762 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:08.743661 28931 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:08.619804 28843 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:14:08.758749 28765 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:08.787757 29038 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.813421 22909 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
W20260709 13:14:08.814369 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 417494us
I20260709 13:14:08.819180 28504 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:08.822113 28731 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.825785 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:08.828351 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.828626 23458 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:08.854980 22915 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:08.809604 28948 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.830480 29002 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:08.884620 23457 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:14:08.867529 28893 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.849913 29001 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:08.882995 28910 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:08.886089 28931 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:08.907276 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81606.1us
I20260709 13:14:08.918290 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:08.919759 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:08.910913 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.892699 29054 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.895571 28967 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.904057 28893 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:08.926744 28938 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:08.930483 28909 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.931527 28974 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:08.773554 28976 consensus_queue.cc:1175] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "776baf137f8745dd866379ca15ab8cee" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: [14.10-14.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:08.938694 29015 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.968716 28504 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.970578 29000 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:08.966852 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.908632 28400 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:08.978461 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.915112 28948 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:08.914808 28680 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:08.921881 28911 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:08.973340 29000 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:08.978613 28938 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:09.003686 29046 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.006s
I20260709 13:14:08.981019 28974 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:09.022939 22907 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:09.024824 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3244) took 1203 ms. Trace:
I20260709 13:14:08.957901 28967 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:09.026114 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 351525us
I20260709 13:14:09.027510 22909 rpcz_store.cc:276] 0709 13:14:07.821034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.821161 (+   127us) service_pool.cc:224] Handling call
0709 13:14:09.024594 (+1203433us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400125cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:09.024737 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214083968}
I20260709 13:14:08.952775 29054 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.033150 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:08.987486 28400 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.021126 28816 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:08.952234 28765 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.024950 29046 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:09.018888 28978 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:08.990314 28948 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.038043 29009 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:09.084844 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 396799us
W20260709 13:14:09.049822 29075 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
I20260709 13:14:09.082949 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:09.085781 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:09.088999 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:09.093642 23458 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:14:08.999293 29001 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.095064 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:09.096230 29001 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:09.097563 28941 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:09.099136 28941 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:09.099771 28941 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:09.102387 28934 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:09.102994 28934 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.104141 28934 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.105209 28934 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:09.085588 28874 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.107756 28874 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.110363 22836 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:09.118700 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:09.121533 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.063849 29053 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:09.122704 22909 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:14:09.149127 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.150281 22907 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:09.148149 28816 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:09.153421 28893 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.152674 28975 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.113322 29054 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:09.153458 28816 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.173393 28816 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.179669 29065 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.180480 29065 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.183053 28967 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.184059 29065 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:09.122769 28938 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.190981 28941 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.192245 28938 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:09.192663 28941 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:09.180986 28948 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:09.197824 28947 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.005s
I20260709 13:14:09.201939 29058 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:14:09.220564 28773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.034s	sys 0.080s
I20260709 13:14:09.208834 28931 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:09.240393 29061 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.762s	user 0.015s	sys 0.002s
W20260709 13:14:09.279155 22759 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.025s	sys 0.054s
W20260709 13:14:09.280767 22759 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.026s	sys 0.055s
W20260709 13:14:09.286476 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:09.228505 28773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.035s	sys 0.080s
W20260709 13:14:09.374015 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 252420us
I20260709 13:14:09.374370 29080 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:09.378110 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 73650.5us
I20260709 13:14:09.380417 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:09.382376 28504 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:09.384199 28504 consensus_queue.cc:1243] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 20.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 13:14:09.387171 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.387907 28921 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:09.388521 28921 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:09.395537 28931 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:09.397738 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 274540us
I20260709 13:14:09.400310 28910 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.402199 28921 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.406991 28921 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:09.410714 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.411569 28874 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:09.412461 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 323465us
I20260709 13:14:09.412972 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.414988 28874 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:09.416617 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.416860 28874 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:09.418315 28953 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:09.421078 28984 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:09.421672 28984 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:09.422559 28953 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:09.423296 28984 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.425127 28953 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:09.376101 29080 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.432237 28976 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:09.433311 28976 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.434209 29080 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.436443 29061 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:09.439044 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61925us
I20260709 13:14:09.438755 29079 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:09.446004 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:09.447922 29058 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:09.429795 28984 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:09.449317 29058 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.451390 29058 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.464694 29038 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:09.470513 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.465464 28765 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:09.456002 28400 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:09.474125 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:09.479640 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:09.480547 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 546548us
I20260709 13:14:09.433061 28953 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.486461 28909 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:09.499416 28909 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:09.499483 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 51835.3us
I20260709 13:14:09.486465 28931 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.503151 28909 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.503756 28910 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.502085 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.514039 28384 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:09.518428 28940 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:09.517092 28384 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:09.503252 28931 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.529371 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:09.532401 28931 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.536732 28976 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.454942 28874 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.542102 29061 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:09.542682 29061 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:09.543123 29061 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:09.560585 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:09.564963 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.546480 28940 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:14:09.574534 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:09.579849 28934 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:09.590194 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:09.594637 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49686.1us
W20260709 13:14:09.598186 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 181824us
I20260709 13:14:09.596098 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:09.602064 28921 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:09.606810 28504 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:09.607070 28921 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.607442 28504 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:09.604750 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 153208us
W20260709 13:14:09.622538 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50310us
I20260709 13:14:09.625945 28976 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:09.626153 28504 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.629143 28976 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:09.630052 28976 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.636425 28923 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:09.641263 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:09.631091 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:09.620850 29080 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.641942 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:09.644979 28947 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:09.649993 28947 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.660455 28843 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.661494 28843 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.666016 28843 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:09.666815 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.686818 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:09.687743 23431 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.514s (SENT) [suppressed 15 similar messages]
W20260709 13:14:09.688053 29001 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.591s	user 0.020s	sys 0.010s
W20260709 13:14:09.689777 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46336.9us
I20260709 13:14:09.694274 28400 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:09.674836 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:09.635545 28953 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:09.650836 28947 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:09.685613 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133008us
I20260709 13:14:09.687165 28843 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.674453 28504 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.699896 28953 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.702155 29082 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:09.699973 28843 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.703462 28816 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:09.703902 28816 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.700510 28953 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:09.696661 28773 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:09.702697 22908 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:09.700383 28504 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.705287 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.649168 28976 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:09.713398 28773 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:09.720285 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 73904.5us
I20260709 13:14:09.661092 29061 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.729785 22836 raft_consensus.cc:2410] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:14:09.732477 22759 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:09.722448 28921 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.738541 28947 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:09.739336 28947 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:09.740049 28947 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.740480 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:09.741560 22761 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:09.742282 28923 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.743198 28923 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:09.739476 28938 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.546s	user 0.016s	sys 0.005s
I20260709 13:14:09.740677 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.747498 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.705416 28816 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.749738 28843 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.750996 28773 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:09.752171 29002 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.842s	user 0.004s	sys 0.022s
I20260709 13:14:09.752578 28773 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:09.754906 28504 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.752981 29002 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.730684 29080 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.769407 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:09.771888 22761 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:09.773175 28923 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.787515 28938 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:09.793017 28938 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.794512 28938 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.788848 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:09.798110 22915 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:09.789548 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:09.789165 29054 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:14:09.802006 23457 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:09.802670 29054 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:09.803145 29054 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.790509 28947 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.804340 29054 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.807827 28816 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:09.809223 28976 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.810231 28976 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.813221 29002 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.813935 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:09.815099 28941 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.565s	user 0.015s	sys 0.014s
I20260709 13:14:09.820151 29080 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.822059 29061 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:09.824153 28953 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.826433 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.827039 28953 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.828930 28940 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:09.830171 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.830390 28940 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:09.832649 29082 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:09.833226 29082 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.834818 29082 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.836524 22760 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:09.846118 29099 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:09.837349 29082 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:09.840850 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61440.1us
I20260709 13:14:09.856585 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.857527 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:09.857972 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:09.858642 22838 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:09.857631 22910 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:14:09.860136 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.862986 28976 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:09.863262 29062 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:09.866636 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.867767 22909 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:09.869684 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:09.871223 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.871563 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:09.872084 22909 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:09.872713 29062 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:09.873272 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:09.873621 29062 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:09.876473 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.876582 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.876871 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:09.878521 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 173383us
I20260709 13:14:09.880079 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:09.880270 29080 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:09.881597 29080 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:09.888258 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61651.7us
I20260709 13:14:09.891364 22760 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:14:09.889123 29080 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:09.897763 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.902112 22908 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:09.886266 29099 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:09.906054 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:09.896713 22760 pending_rounds.cc:85] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:09.910465 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:09.918701 28934 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:09.919431 28934 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:09.920841 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:09.922009 22761 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:09.922076 29054 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:09.922688 28947 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.923543 28947 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.913551 29101 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:14:09.927140 28947 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.905580 29099 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:09.936862 22910 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:09.921032 28934 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.939429 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:09.914763 22760 pending_rounds.cc:107] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:14:09.940485 22760 raft_consensus.cc:2889] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:09.947577 29101 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:14:09.950738 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53544.2us
I20260709 13:14:09.952842 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:09.957726 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:09.963248 22759 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
W20260709 13:14:09.963730 22658 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 137) took 20652 ms (client timeout 15000 ms). Trace:
I20260709 13:14:09.964491 29079 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:09.964948 29079 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.965404 29079 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:09.970738 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:09.960132 29094 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:14:09.973455 29009 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.853s	user 0.007s	sys 0.021s
I20260709 13:14:09.974182 28953 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.974876 28953 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:09.977932 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:09.979159 22765 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:09.980319 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:09.980535 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:09.982236 22762 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:09.984589 22757 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:09.972927 22655 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:09.985440 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:09.986478 23459 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:09.986471 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:09.987985 22655 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 8 to 14, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:09.989074 22764 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:09.984288 28941 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:09.990895 29062 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:09.991369 29062 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:09.991844 22758 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:09.992811 29062 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:09.994271 28953 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:09.994962 29062 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:09.995213 28984 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.545s	user 0.014s	sys 0.001s
I20260709 13:14:10.008395 28874 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:10.009003 22764 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:14:09.939937 29099 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:10.009945 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58258.9us
I20260709 13:14:10.011121 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:10.016758 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:10.024374 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 152950us
I20260709 13:14:10.027094 28384 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.027895 28384 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:10.028712 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.016961 29081 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:10.029834 29081 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.030117 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.031980 22762 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:10.032123 29081 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.032765 28947 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:10.036505 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.037189 22758 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:10.022490 29038 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.544s	user 0.015s	sys 0.001s
I20260709 13:14:10.012971 28874 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.055632 29098 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.055852 22910 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:10.056347 29098 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.058007 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.058905 22766 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 11.
I20260709 13:14:10.059361 29098 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:10.063959 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:10.077293 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.102021 22760 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:10.100898 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.048362 29058 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.065703 28874 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:09.964298 22658 rpcz_store.cc:269] 0709 13:13:49.311042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.311373 (+   331us) service_pool.cc:224] Handling call
0709 13:13:49.992104 (+680731us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5400310010. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cd0b2 00007f68a7946a89 00007f68a78deda6 00007f68a78b0088 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc
0709 13:14:05.712852 (+15720748us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b54000407a8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a419b4 00007f68a55253dc 00007f68a54e6e0b 00007f68a54e6be4 00007f68a79e72d1 00007f68a79e7b2f 00007f68a78a6d4b 00007f68a79aec3e 00007f68a5e6f2a4 00007f68a5e6f232
0709 13:14:09.963450 (+4250598us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:06.042586 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:06.091214 (+ 48628us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:06.091256 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:14:06.091274 (+    18us) tablet.cc:662] Decoding operations
0709 13:14:06.092487 (+  1213us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:06.092579 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:06.092609 (+    30us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:14:06.160869 (+ 68260us) tablet.cc:701] Row locks acquired
0709 13:14:06.160888 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:14:06.161079 (+   191us) write_op.cc:270] Start()
0709 13:14:06.161211 (+   132us) write_op.cc:276] Timestamp: P: 1783602846161023 usec, L: 0
0709 13:14:06.161236 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:14:06.162003 (+   767us) log.cc:844] Serialized 5112 byte log entry
0709 13:14:06.906453 (+744450us) op_driver.cc:464] REPLICATION: finished
0709 13:14:07.776281 (+869828us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a5527ef0
0709 13:14:07.979163 (+202882us) write_op.cc:301] APPLY: starting
0709 13:14:07.979260 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:08.138282 (+159022us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:08.138313 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:08.432260 (+293947us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:08.531224 (+ 98964us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:14:08.537682 (+  6458us) write_op.cc:312] APPLY: finished
0709 13:14:08.591061 (+ 53379us) log.cc:844] Serialized 121 byte log entry
0709 13:14:09.040771 (+449710us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:09.041022 (+   251us) write_op.cc:454] Released schema lock
0709 13:14:09.048869 (+  7847us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:09.167904 (+119035us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:09.170413 (+  2509us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":619388,"reported_tablets":28,"spinlock_wait_cycles":111897728,"child_traces":[["op",{"apply.queue_time_us":1072268,"apply.run_cpu_time_us":62766,"apply.run_wall_time_us":1135775,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58493,"num_ops":13,"prepare.queue_time_us":250506,"prepare.run_cpu_time_us":37488,"prepare.run_wall_time_us":464158,"raft.queue_time_us":290080,"raft.run_cpu_time_us":70793,"raft.run_wall_time_us":1055500,"replication_time_us":745028,"spinlock_wait_cycles":17328384,"thread_start_us":2053835,"threads_started":5,"wal-append.queue_time_us":1053160,"wal-append.run_cpu_time_us":15374,"wal-append.run_wall_time_us":1265103}]]}
I20260709 13:14:10.047974 28953 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:10.106277 28976 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:10.036741 28947 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.123216 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.128281 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:10.122188 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.134099 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.101156 29098 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:10.136471 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.138581 22906 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:14:10.109763 28843 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.112164 28953 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.101827 29080 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.053884 22655 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:10.146148 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.145303 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.129582 28938 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.154644 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.143399 28953 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.174413 22765 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:10.134690 29058 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.167838 29099 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:10.173786 22907 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:10.187935 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.148976 28976 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.194912 22836 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:10.199569 29001 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.200266 29001 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.159876 29098 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.202567 29001 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.205268 28938 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.186542 28843 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.176625 29054 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:10.214219 29054 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.168263 22655 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 5 to 10, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 10 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:10.175361 28384 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:10.217928 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:10.234916 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.232137 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:10.232815 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:10.237114 23461 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.238184 29080 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.240130 28976 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.232582 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:10.241657 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.232878 23466 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:10.233467 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.231878 23464 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:10.234086 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.243999 23463 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:10.245052 23458 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:10.234447 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:10.235241 28941 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.246455 28941 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.248822 28941 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:10.251523 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.217463 29054 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.252486 22762 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:10.254525 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:10.264603 23462 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.254005 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:10.255370 28947 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.251164 29098 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.256045 28984 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:10.257026 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:10.259711 22835 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:10.260951 28938 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.257877 22765 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:10.279759 23461 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:10.285701 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.249176 29058 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:10.231734 22655 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.374699 23462 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:10.288595 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.304781 22837 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:10.397882 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.306152 22759 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:10.392925 22765 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:10.276432 28843 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.426222 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.426981 22761 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:14:10.427222 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.427306 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.428054 23459 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:10.428730 22757 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:10.429966 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.430820 22765 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:10.432576 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.433390 22759 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:10.438047 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.438803 22764 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:10.441015 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.441612 22764 raft_consensus.cc:2376] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17. Re-sending same reply.
I20260709 13:14:10.443917 22837 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:10.445338 22759 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:10.445976 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.446221 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.446889 22764 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:14:10.448196 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.448750 22759 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:10.448837 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.449595 22757 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:14:10.451483 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.452158 22765 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:10.454000 28874 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.414228 22760 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:10.417244 22655 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.288231 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.309458 29002 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.384128 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.463076 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.234333 29062 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:10.445256 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.314201 28940 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:10.458462 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:10.309599 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55965.2us
I20260709 13:14:10.307662 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:10.473778 22759 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:10.480386 23463 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:10.309899 22762 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:10.486407 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:10.490553 22838 raft_consensus.cc:2368] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16.
W20260709 13:14:10.490005 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 64504.8us
I20260709 13:14:10.491441 22764 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:10.310725 29001 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.493199 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:10.493948 22907 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:10.494246 28953 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.501237 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.501425 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:10.477864 29002 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.504480 29002 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.505015 29002 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:10.495143 22655 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.488631 22836 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:10.280246 28976 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:10.499925 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.488270 28984 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.496030 29098 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:10.508212 22833 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:10.376627 29079 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.517050 28843 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:10.518990 28843 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.519039 29113 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:10.519452 28843 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.519665 28976 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.522123 28976 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.518484 29098 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:10.517211 28984 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.523342 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:10.525091 28984 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.525799 28984 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:10.526151 28984 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.519790 29113 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:10.526682 28984 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.523380 29098 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.527500 29113 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.508555 29002 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.534140 22765 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:10.533747 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:10.536103 28938 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:10.536620 28938 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:10.537654 28938 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.539469 28843 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.539600 22655 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 10, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 10 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:10.548185 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.551559 23462 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:10.534046 29079 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.552670 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.515152 29081 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:10.553799 29081 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:10.554236 29081 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.555574 22765 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:10.557302 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:10.574066 23464 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.573360 28953 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:10.559588 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.562209 28816 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:10.577625 28816 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:10.585695 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.586494 23462 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:10.587661 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:10.587965 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.588343 22837 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.588665 22835 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:10.589584 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.590368 22835 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:14:10.591457 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.592692 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:10.593487 22841 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 14: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:10.594336 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:10.596459 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.597864 23462 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:10.564272 28874 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.566583 29054 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:10.568811 28843 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.569300 28384 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:10.628105 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:10.571449 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:10.533797 22766 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:10.492686 28940 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:10.601473 22761 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:10.608232 22837 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
W20260709 13:14:10.619982 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.625056 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.627722 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.629420 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.634186 22765 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:10.630664 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:10.634886 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 63537.6us
I20260709 13:14:10.509641 29001 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.633064 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.638551 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.634038 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:10.637081 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.639163 22837 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:10.637081 22835 raft_consensus.cc:2368] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:10.639896 22764 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:10.640223 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.641603 22910 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:10.509388 29062 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.651026 22836 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:10.652546 22836 pending_rounds.cc:85] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 10
I20260709 13:14:10.652843 22836 pending_rounds.cc:107] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:14:10.653093 22836 raft_consensus.cc:2889] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.654723 29001 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.655616 22758 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:14:10.655978 22758 pending_rounds.cc:85] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:10.656190 22758 pending_rounds.cc:107] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:14:10.656375 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:10.657040 22914 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.627254 29002 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:10.663152 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.656407 22758 raft_consensus.cc:2889] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.664847 22835 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:10.662189 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.667694 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.667850 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.646886 22907 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:10.662081 23465 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:10.665777 22879 leader_election.cc:400] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16.
I20260709 13:14:10.669008 22879 leader_election.cc:403] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:14:10.663679 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:10.667701 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.669901 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.672215 22840 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:10.674710 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.676601 22835 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:10.646438 28976 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:10.677732 28976 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:10.678205 28976 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:10.678790 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.680168 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.675694 22763 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:10.682224 22765 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:10.640967 29082 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:10.576807 22655 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.647895 29123 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.684191 22838 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:10.637713 28874 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:10.687861 28816 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.688007 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.688669 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.687896 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.698210 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.698987 22763 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.700291 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.701206 22655 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:10.701248 29120 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:10.711148 22907 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:10.702888 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.711817 22839 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:10.703609 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.683849 28940 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.712836 22833 raft_consensus.cc:2368] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:10.684450 29082 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.715726 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:10.715338 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.722258 22765 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.727291 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:10.729226 22914 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:10.734833 23464 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:10.729216 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:10.736444 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.724448 29082 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.737877 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.738598 22766 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:14:10.740026 22762 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:10.741743 22764 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:10.688427 28773 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:14:10.743067 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.743197 28773 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:10.743696 28773 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:10.729538 29122 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:10.744669 28773 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.736553 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:10.746466 22757 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:10.753276 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:10.759433 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:10.757427 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.761731 22758 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:14:10.765829 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 129570us
I20260709 13:14:10.769830 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:10.734579 29002 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:10.773187 29002 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.773622 29002 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:10.774171 29001 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:10.774837 29001 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.775281 29001 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.777913 28941 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:10.781095 22832 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:10.781890 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:10.710898 29123 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.784199 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:10.788754 22915 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:10.786299 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57171.5us
I20260709 13:14:10.806309 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:10.788125 29054 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:10.810226 22759 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:10.813823 22759 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:10.814952 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:10.747783 28874 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:10.820858 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61517.2us
I20260709 13:14:10.788803 29098 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:10.812857 28938 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:10.829725 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:10.832366 22908 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.834885 22759 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:14:10.836875 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.842343 28984 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.831627 28938 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:10.844812 28938 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:10.825374 28773 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.839133 22765 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:10.856748 22758 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:10.830925 29098 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:10.860723 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:10.856588 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:10.879916 28923 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:10.880362 28923 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.880759 28923 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:10.822312 29122 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:10.858521 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:10.894413 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59648.5us
I20260709 13:14:10.862955 29038 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.863090 28940 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:14:10.874778 29123 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:10.895872 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 86626.8us
I20260709 13:14:10.896123 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:10.876323 22908 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
W20260709 13:14:10.897925 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125028us
I20260709 13:14:10.899297 29130 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:10.910192 29130 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.910657 29130 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.853629 28953 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:14:10.894897 28874 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:10.929116 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67708.6us
I20260709 13:14:10.926081 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:10.902805 28938 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.927469 29062 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:10.933454 29123 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:10.938004 28947 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:10.969444 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47614.8us
I20260709 13:14:10.906246 29038 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.899818 29135 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:14:10.971699 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:10.831719 22655 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 4 to 12, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:14:10.980301 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74706.8us
I20260709 13:14:10.937063 28923 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:10.985107 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81469.1us
I20260709 13:14:10.967947 29123 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:10.993247 28816 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16.
I20260709 13:14:10.995481 29139 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:11.010149 28816 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:10.996121 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:10.971247 29132 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:14:11.024469 29126 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.387s	user 0.003s	sys 0.000s
I20260709 13:14:11.026659 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:11.029624 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:11.070089 22764 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:10.981691 28947 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:10.992120 29082 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.086302 29038 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.009057 29123 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.139372 22909 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:11.069306 22757 raft_consensus.cc:2376] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19. Re-sending same reply.
I20260709 13:14:10.990949 29127 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:11.143107 29127 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.143500 29127 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:11.099962 29140 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:14:11.103159 29128 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:14:11.105611 22762 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:11.110442 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:11.171397 22766 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.073101 29098 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:10.991255 29122 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.073526 28984 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:11.119731 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.053009 29132 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:11.133602 28947 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:11.157871 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 141824us
W20260709 13:14:11.178779 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 180900us
W20260709 13:14:11.185397 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 246874us
W20260709 13:14:11.198328 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 375495us
W20260709 13:14:11.194953 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.200426 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 208263us
W20260709 13:14:11.199127 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.208606 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:11.209255 22907 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.207017 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.203183 29062 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:14:11.210970 22910 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:14:11.217883 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 385196us
I20260709 13:14:11.218827 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.205090 29082 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.221700 29144 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.222288 29144 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:11.223232 28976 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.008s
I20260709 13:14:11.223667 28976 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.224550 29144 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:11.226099 29130 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.228261 29130 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:11.229067 22761 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:11.201362 29125 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.555s	user 0.003s	sys 0.000s
W20260709 13:14:11.203325 29081 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.648s	user 0.017s	sys 0.007s
I20260709 13:14:11.235984 22914 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:11.174993 28874 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:11.245319 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:11.083220 22655 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 8, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 8 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:11.200368 29129 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:11.204905 28923 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.262863 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:11.264405 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57473.7us
I20260709 13:14:11.264382 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:11.275281 22906 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:11.275357 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.271633 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:11.286980 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:11.293689 22763 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.008s
I20260709 13:14:11.294128 22763 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:14:11.265115 28976 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.275017 29058 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:11.303957 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55873.6us
I20260709 13:14:11.313009 29154 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:11.313838 29154 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:11.314324 29154 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:11.320281 29038 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:11.320811 29038 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:11.326242 29038 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.254982 29081 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.342468 29138 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:11.343591 22835 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:11.267241 29082 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:14:11.346525 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 140436us
I20260709 13:14:11.269026 28874 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.304986 29099 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:11.347855 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:11.355361 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.301442 29135 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:14:11.356993 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:11.350394 29002 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.576s	user 0.007s	sys 0.011s
I20260709 13:14:11.350817 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:11.352382 29001 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.576s	user 0.016s	sys 0.007s
I20260709 13:14:11.346673 28976 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:14:11.359117 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49634.2us
I20260709 13:14:11.370347 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:11.338795 22655 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.384974 22838 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:11.352784 29099 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:11.366389 22915 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.559s	user 0.017s	sys 0.004s
I20260709 13:14:11.345896 29138 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:11.392114 22915 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
W20260709 13:14:11.392794 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45451.7us
I20260709 13:14:11.396859 22730 leader_election.cc:400] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:11.397814 22730 leader_election.cc:403] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:11.383939 29002 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.393697 29099 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:11.403025 22655 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 6 to 11, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 11 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:11.393623 22836 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:11.437467 29002 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:11.420414 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 349212us
W20260709 13:14:11.403999 29152 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:14:11.406121 29143 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:14:11.406776 29146 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:11.407639 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 188938us
I20260709 13:14:11.404040 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:11.416718 28976 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:11.402880 29133 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:14:11.441624 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 93863.1us
I20260709 13:14:11.439875 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1106) took 1015 ms. Trace:
I20260709 13:14:11.445045 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.387010 29129 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:11.388149 28773 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:11.386456 29001 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.454979 23461 rpcz_store.cc:276] 0709 13:14:10.424928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.425487 (+   559us) service_pool.cc:224] Handling call
0709 13:14:10.426009 (+   522us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:10.426583 (+   574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:10.426625 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:10.566219 (+139594us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b2c00040ad8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:10.566838 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 13:14:10.628244 (+ 61406us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:10.645519 (+ 17275us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.645970 (+   451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.646025 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:11.179229 (+533204us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:11.439648 (+260419us) raft_consensus.cc:1685] finished
0709 13:14:11.439671 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:11.439813 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15203,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2516,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":192,"spinlock_wait_cycles":81964032,"thread_start_us":60955,"threads_started":1,"wal-append.queue_time_us":78684}
I20260709 13:14:11.464151 22730 leader_election.cc:400] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:11.464797 22730 leader_election.cc:403] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:11.463781 28773 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.466868 29161 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:11.467438 29161 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:11.467826 29161 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:11.467945 28773 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.469681 29155 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:11.470588 29117 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:11.471208 29117 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:11.472226 29117 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.473639 29150 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:11.474704 29121 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:14:11.477703 29140 raft_consensus.cc:2764] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:14:11.478089 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:11.480470 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:11.348331 29132 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.399035 29082 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.482244 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:11.482494 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.488602 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.489089 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.492080 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:11.492734 23463 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.493048 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:11.493911 23462 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:11.494849 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:11.496399 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:11.488075 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:11.464793 29133 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.477953 29155 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:11.500411 23466 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:11.500844 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:11.501675 29129 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.502399 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:11.504459 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:11.509465 23462 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:11.440421 22655 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:11.507171 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:11.511209 23457 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:11.513726 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:11.515801 29148 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:11.516500 29148 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:11.517787 29081 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.517453 29148 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.519586 29147 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:11.519660 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.520195 29147 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:11.520692 29147 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:11.521674 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 127301us
I20260709 13:14:11.490706 29080 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:11.527290 22760 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:11.527760 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.508713 29133 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:11.543586 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.549214 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:11.550076 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 146139us
I20260709 13:14:11.513258 29054 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:11.495942 29082 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.556738 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:11.464944 29146 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.577163 23464 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:11.551271 29132 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:11.550796 29002 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:11.554404 29122 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:11.588686 29099 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.591382 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.592331 29164 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:11.594201 29145 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:11.575147 29154 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:11.598369 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100698us
I20260709 13:14:11.498457 29138 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.603856 22840 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
I20260709 13:14:11.576748 29155 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:11.575358 29098 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:11.541015 29080 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.606366 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.607146 29135 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:11.611864 29038 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:11.614809 22806 leader_election.cc:336] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.645s (SENT)
I20260709 13:14:11.626870 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:11.618742 23463 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:11.628702 23465 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:11.602324 22766 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:11.631560 29122 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.642074 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:11.584038 29081 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:14:11.645788 22838 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.653553 29163 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:11.654264 29163 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.654744 29163 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:11.560607 28976 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.625707 29098 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:11.654857 29123 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.518s	user 0.005s	sys 0.009s
I20260709 13:14:11.632447 29138 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.657003 22760 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:14:11.659144 29081 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.616335 29058 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:14:11.622123 29143 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:14:11.623940 29164 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:14:11.662678 29127 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.519s	user 0.015s	sys 0.000s
I20260709 13:14:11.664464 29147 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.666031 29149 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:11.667514 28384 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:11.674257 29158 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:11.675125 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:11.682360 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60119.2us
W20260709 13:14:11.692835 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78089us
I20260709 13:14:11.723771 22763 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:14:11.695888 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 173482us
I20260709 13:14:11.635964 29145 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:11.710489 22907 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:11.681237 29154 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.718220 29001 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.655704 28816 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:14:11.671430 29146 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.679847 29082 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:11.683916 29080 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.692703 29098 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:11.718407 29081 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.731045 29149 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:11.657392 22655 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.753747 29149 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.675204 28843 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:11.755936 22655 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.724933 28384 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:11.761724 29145 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.763039 29081 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:11.747869 22906 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:11.646608 29122 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.764156 29147 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.752216 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:11.762622 28384 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.712800 29123 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.776763 22838 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
W20260709 13:14:11.688098 28938 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.005s
I20260709 13:14:11.778173 22879 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:11.785051 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 241553us
I20260709 13:14:11.765322 29122 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.719355 29127 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.739521 29158 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:11.806195 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:11.765322 29082 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:14:11.765767 29081 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:14:11.792906 29154 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.810696 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:11.810977 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:11.823395 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:11.811426 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:11.835587 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:11.850634 23463 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 12.
I20260709 13:14:11.823038 29123 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.810316 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:11.837554 23461 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:11.842365 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64317.2us
W20260709 13:14:11.844913 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 211632us
I20260709 13:14:11.860899 22765 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:14:11.861286 22836 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:14:11.848335 22839 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:14:11.880110 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 69855.1us
W20260709 13:14:11.887447 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76143.2us
I20260709 13:14:11.824664 28874 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.892716 22906 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:11.883620 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:11.814208 29099 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:11.822809 29054 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:11.902976 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:11.903545 29149 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.904263 22914 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:11.905051 29158 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.860893 28923 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.907397 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:11.915370 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:11.910786 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:11.918323 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:11.885108 29122 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:11.833726 29133 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:11.833446 29098 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:11.891355 29001 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:11.938792 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:11.941419 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:11.981974 22766 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:11.981818 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:11.829146 29161 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:11.975273 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49382.3us
I20260709 13:14:11.930835 29146 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:11.927683 29139 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:11.984172 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44225.8us
I20260709 13:14:11.919229 29002 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:11.987167 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:11.931705 29123 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:11.920266 29054 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:11.977080 22908 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:11.990355 23465 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:11.996850 29181 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:11.997345 29182 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:12.000123 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:12.000761 29177 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:11.830009 22655 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:12.023554 23463 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:11.978543 28984 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:12.051613 29141 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:14:12.051714 29182 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.054292 29182 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:11.975605 28923 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.056401 28923 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:12.057098 29149 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:12.058941 29133 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:12.058725 22655 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:12.060297 29127 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:12.043223 28816 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 13:14:12.067175 29161 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:12.068568 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.070101 22841 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:12.014673 29123 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:12.034263 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 119018us
I20260709 13:14:11.983067 29155 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:12.040802 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:12.084143 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:11.988420 29135 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:14:12.031075 23461 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
W20260709 13:14:12.118727 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.120292 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47635.7us
I20260709 13:14:12.121236 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:12.144572 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:12.050725 29158 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:12.070188 29177 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.095803 29123 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:12.153925 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 212998us
I20260709 13:14:12.096029 29155 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:12.111029 29001 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:12.170328 29146 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:12.170867 29159 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:12.174275 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.170392 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 49274.4us
I20260709 13:14:12.174892 22835 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:12.038378 29139 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.185657 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:12.162725 29177 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:12.192507 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74304.6us
I20260709 13:14:12.193099 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:12.220655 22837 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:12.197479 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:12.190364 29117 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:14:12.221828 22879 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:12.206187 29001 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.208024 28384 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:12.211534 29098 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:12.165611 28984 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:12.215792 22806 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.857s (SENT)
I20260709 13:14:12.215886 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:12.168176 29155 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:12.214046 29079 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.226944 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:12.277122 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:12.230522 29163 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.570s	user 0.002s	sys 0.011s
I20260709 13:14:12.217298 28816 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:12.310853 29167 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:14:12.311264 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:12.321916 29145 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:12.323408 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 132078us
I20260709 13:14:12.323565 29117 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:12.330120 29146 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:12.226231 29144 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:12.177364 28940 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
W20260709 13:14:12.355647 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.065s (SENT)
I20260709 13:14:12.364188 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.297655 29159 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:12.345799 28816 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:12.364313 29192 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:14:12.374615 29145 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:12.397384 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.664s (SENT)
I20260709 13:14:12.398048 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.392292 29148 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:12.399562 29145 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:12.401218 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68886.3us
I20260709 13:14:12.410171 22832 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:12.411549 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1073) took 1165 ms. Trace:
I20260709 13:14:12.412077 22840 rpcz_store.cc:276] 0709 13:14:11.246649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.261901 (+ 15252us) service_pool.cc:224] Handling call
0709 13:14:12.401294 (+1139393us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b540028a0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:12.411451 (+ 10157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709548288}
I20260709 13:14:12.415788 23464 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:14:12.415949 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.422253 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:12.424067 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.424284 29161 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:12.425601 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.425781 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:12.425999 29174 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:12.426432 29174 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:12.426558 22913 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:12.427441 29174 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.428735 29174 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:12.370600 29144 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:12.429868 29144 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:12.429955 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.425520 29161 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:12.444635 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.343400 29079 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:12.450224 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52908us
W20260709 13:14:12.394378 29038 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.762s	user 0.006s	sys 0.010s
I20260709 13:14:12.453007 23461 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:12.457392 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:12.463747 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.219s (SENT)
I20260709 13:14:12.393618 28816 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:12.354099 29163 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.470443 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:12.475323 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:12.480187 22837 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:12.449795 28940 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:12.458894 29150 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.464583 29161 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:12.466357 29148 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:12.442117 29127 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:14:12.506218 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.482317 29002 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.540212 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.465180 29122 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
W20260709 13:14:12.547540 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87133us
W20260709 13:14:12.532317 22839 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.002s
I20260709 13:14:12.549436 22839 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:12.492388 28976 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.553429 22907 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:12.557102 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:12.561427 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.566982 22914 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:12.472949 28984 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:12.573889 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.570309 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:12.575912 23461 raft_consensus.cc:2368] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:12.500645 29150 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.545356 28923 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:12.547003 29141 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:12.578756 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:12.609886 23462 raft_consensus.cc:2368] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:12.545662 29002 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:12.553788 28874 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:12.598623 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:12.495596 28940 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.585327 22907 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:14:12.638016 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82964.5us
I20260709 13:14:12.569168 29180 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:14:12.572818 28976 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:12.642298 22906 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:12.555900 29159 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.613795 29189 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:12.656136 29188 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.352s	user 0.001s	sys 0.001s
I20260709 13:14:12.666400 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.665879 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:12.674064 23458 raft_consensus.cc:2368] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15.
I20260709 13:14:12.669296 23464 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:12.676805 29002 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:12.677567 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.630517 28874 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.679579 28874 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:12.680550 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.681452 22763 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:12.681910 29197 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:14:12.683197 29196 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:12.681910 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.681210 22757 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:12.666030 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:12.620885 29141 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.688182 23466 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:12.657254 29139 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:12.626528 28923 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.614676 29161 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.697911 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:12.704466 22764 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:12.700069 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.696477 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.706194 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.695851 22805 leader_election.cc:400] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:12.707695 22805 leader_election.cc:403] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:12.708420 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.655825 29148 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:14:12.671532 29163 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:12.712277 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.711877 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:12.716584 22757 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:12.657289 29122 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:14:12.720376 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:12.723887 22764 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:12.701030 29079 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:12.711498 29038 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:12.719039 29172 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:12.737394 29154 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.882s	user 0.011s	sys 0.001s
I20260709 13:14:12.740149 29154 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:12.714745 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:12.727948 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.743322 22763 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:12.730033 29201 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:14:12.711849 29132 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:12.713174 29147 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:12.718182 29116 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:12.751206 22906 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:12.713537 22805 leader_election.cc:400] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:12.751875 22805 leader_election.cc:403] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:12.700649 29189 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:12.739439 29123 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.565s	user 0.002s	sys 0.010s
I20260709 13:14:12.769040 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.803144 28874 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:12.803853 28874 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:12.806413 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:12.806798 28874 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:12.809543 29149 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:12.811262 29149 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:12.812305 29133 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:12.814167 29164 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:12.814803 29164 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:12.816291 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45345.9us
W20260709 13:14:12.761166 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47859.4us
W20260709 13:14:12.852573 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 82895.2us
I20260709 13:14:12.854316 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.857096 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:12.745779 29082 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
W20260709 13:14:12.853010 29199 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:14:12.882421 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:12.767824 28947 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:12.768045 29172 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:12.914527 29172 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:12.879608 29194 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:14:12.896555 22763 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:12.897955 29168 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:12.899993 29206 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.900557 29180 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:12.902432 29164 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:12.902434 29117 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:12.903986 22805 leader_election.cc:400] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15.
I20260709 13:14:12.904877 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.949440 22805 leader_election.cc:403] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:14:12.906476 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:12.949797 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:12.950196 22760 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:12.908090 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:12.951300 22759 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:12.949849 22766 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:12.930189 22806 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.152s (SENT)
I20260709 13:14:12.940559 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.757058 29155 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:12.959899 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 56054.7us
I20260709 13:14:12.957758 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:12.856256 29123 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:12.973152 22759 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:12.974042 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.982987 22763 raft_consensus.cc:2376] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:14:12.960769 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:12.976447 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:12.984311 22761 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:12.921456 29141 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:12.922902 29150 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:12.995198 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:12.998544 22907 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:12.927196 29168 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:13.022861 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.023872 22759 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:13.028530 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67870.1us
I20260709 13:14:12.977391 28940 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:13.028920 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:12.991231 29079 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:13.039002 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.138617 22761 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:13.138868 22763 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:13.139789 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.139914 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.141055 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.040541 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:13.141891 22761 raft_consensus.cc:2368] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13.
I20260709 13:14:13.141139 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.142755 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.143505 22761 raft_consensus.cc:2376] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
W20260709 13:14:13.120255 29194 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:14:12.945384 29206 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.095580 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:13.033422 29164 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.028362 29127 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:13.039937 29146 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:13.100803 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:13.110801 22757 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:14:13.126329 28816 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.002s
I20260709 13:14:13.133791 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.156991 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.157966 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.158316 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.158818 22759 raft_consensus.cc:2410] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:13.159196 22757 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:13.159561 29195 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.556s	user 0.003s	sys 0.003s
I20260709 13:14:13.159685 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.160182 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.044905 29179 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:13.161443 29123 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:13.137594 22760 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:13.162240 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.032336 28976 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:13.143855 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.146881 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.147710 23461 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12.
I20260709 13:14:13.155037 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:13.155736 23463 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:14:13.170403 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.171136 22757 raft_consensus.cc:2376] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18. Re-sending same reply.
I20260709 13:14:13.172129 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.172746 22757 raft_consensus.cc:2376] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12. Re-sending same reply.
W20260709 13:14:13.173914 29198 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.520s	user 0.002s	sys 0.000s
I20260709 13:14:13.174692 22766 raft_consensus.cc:2376] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 12. Re-sending same reply.
I20260709 13:14:13.181988 22764 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:13.156437 29038 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.195243 22766 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:13.160327 22759 raft_consensus.cc:2376] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14. Re-sending same reply.
I20260709 13:14:13.161057 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 761) took 1081 ms. Trace:
I20260709 13:14:13.163120 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.095512 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:13.182972 22762 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:13.183739 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.206987 22759 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:13.184417 22763 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:13.180569 29189 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:13.152004 29127 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:13.167698 29179 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.218475 29179 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:13.218609 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 176692us
W20260709 13:14:13.218920 29211 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:14:13.190038 29210 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:14:13.191381 29218 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:14:13.221982 29206 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:13.222710 29054 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.143s	user 0.012s	sys 0.002s
W20260709 13:14:13.193742 29174 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.002s
W20260709 13:14:13.212225 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 282082us
I20260709 13:14:13.223181 29054 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:13.224480 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.223331 29127 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:13.225627 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.000s (SENT)
W20260709 13:14:13.188140 29212 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 13:14:13.229135 29144 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.003s
I20260709 13:14:13.188443 28843 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:13.208356 29038 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.229435 29146 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:13.203866 23457 rpcz_store.cc:276] 0709 13:14:12.079541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.079701 (+   160us) service_pool.cc:224] Handling call
0709 13:14:12.118677 (+ 38976us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:12.144899 (+ 26222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:12.144933 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:12.164070 (+ 19137us) log.cc:844] Serialized 52 byte log entry
0709 13:14:12.442404 (+278334us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:12.442446 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.442524 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:13.158748 (+716224us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:13.160702 (+  1954us) raft_consensus.cc:1685] finished
0709 13:14:13.160724 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:13.160974 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25370,"raft.queue_time_us":26164,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":15415,"spinlock_wait_cycles":8849792,"thread_start_us":218511,"threads_started":1,"wal-append.queue_time_us":364342}
I20260709 13:14:13.250623 29155 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:13.235404 29204 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:13.157024 28816 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.255462 22909 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:13.248623 29208 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:14:13.254398 22760 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:13.254700 29172 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.275406 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:13.257405 29082 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:14:13.257424 29177 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.976s	user 0.013s	sys 0.001s
I20260709 13:14:13.180617 29079 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15.
I20260709 13:14:13.276733 22757 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:13.212891 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.282353 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:13.302207 23461 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:13.220288 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:13.225358 29054 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:13.287211 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.304127 22761 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:13.271636 28843 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:13.222761 29206 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:13.291157 29158 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.127s	user 0.011s	sys 0.003s
I20260709 13:14:13.291651 29172 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.271684 29150 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:13.325821 29150 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.294526 28816 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.328352 29150 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.258462 29168 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.299400 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2450) took 1121 ms. Trace:
I20260709 13:14:13.299619 23462 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:13.240383 29141 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:14:13.236637 29143 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
W20260709 13:14:13.309571 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 89360.5us
I20260709 13:14:13.337917 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.338681 22766 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:13.338266 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:13.313750 29155 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:13.314283 29221 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.340940 29221 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.330044 22836 rpcz_store.cc:276] 0709 13:14:12.178180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.379876 (+201696us) service_pool.cc:224] Handling call
0709 13:14:12.441442 (+ 61566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:12.611539 (+170097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:12.611574 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:12.612033 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:14:12.853249 (+241216us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:12.853291 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.161925 (+308634us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:13.161971 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:13.297035 (+135064us) raft_consensus.cc:1685] finished
0709 13:14:13.297057 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:13.299311 (+  2254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41209,"raft.queue_time_us":53670,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7414,"spinlock_wait_cycles":161527168,"thread_start_us":269338,"threads_started":2,"wal-append.queue_time_us":360359}
I20260709 13:14:13.314083 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:13.314973 29226 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:14:13.298885 29177 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.340873 29141 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:14:13.350706 29177 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.274638 29144 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.352849 29144 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.319309 29038 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:13.297361 29082 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:13.356225 29082 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.357717 29082 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:13.322341 29206 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.359874 29206 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.284153 29146 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.363446 29146 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:13.333040 28843 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.366686 29206 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:13.371547 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:13.374143 29127 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:13.384995 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.385880 22757 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:13.386507 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:13.387190 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.387885 22765 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:13.389088 28940 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.391624 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.391888 22757 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:13.392261 22762 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:13.335974 29172 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.320883 28384 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.409746 28940 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.410008 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.410604 29117 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:13.411141 29117 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.412062 28940 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.398856 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.413823 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.415606 29038 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:13.415869 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.416188 29038 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:13.416812 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.417470 22761 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.409965 28384 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.418501 22762 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:13.417205 29038 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.430083 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.431142 22836 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:13.435297 22765 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:13.436522 28384 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.437305 29117 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:13.441326 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:13.402829 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.807s (SENT)
I20260709 13:14:13.387244 28923 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:13.405977 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.407604 29144 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:13.408833 29223 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:14:13.347227 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.385072 29155 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:13.481331 29155 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:13.482025 23463 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:13.442214 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.448571 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.482707 29155 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:13.483702 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80939.5us
I20260709 13:14:13.486754 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.487466 22765 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:13.488534 28923 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:13.453454 22807 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:13.464210 23461 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:13.465122 22834 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:13.465831 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:13.491338 22837 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:14:13.469610 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.492643 22764 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:13.470914 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.473448 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.473824 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.494935 22757 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:13.474226 22833 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:13.495975 22835 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:13.475111 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.475304 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.496995 22907 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:13.494292 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.499560 22761 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:13.501345 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.475865 22838 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:13.501793 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.476581 29144 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.477955 29132 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:13.479965 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.503171 29113 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:13.503641 29206 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:13.504405 22839 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:13.512333 22908 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:13.513587 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.505753 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.507064 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.518810 23431 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.520113 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.522715 22837 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:13.523710 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.524343 22757 raft_consensus.cc:2376] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:14:13.526080 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.526743 22806 leader_election.cc:400] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13.
I20260709 13:14:13.526872 22757 raft_consensus.cc:2376] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:14:13.526949 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.529982 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.392212 29221 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.541256 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:13.567683 22763 raft_consensus.cc:2376] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18. Re-sending same reply.
I20260709 13:14:13.540752 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.570407 22766 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:13.570773 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.542032 22806 leader_election.cc:403] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:13.542519 22762 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:13.542474 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.568267 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.572436 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.505847 29132 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:13.604908 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:13.601907 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.527483 29182 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:13.593109 22764 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:13.584635 22763 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:13.571599 22834 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:13.623971 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:13.593779 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.625077 22759 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:13.528196 29129 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:13.585793 22766 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:14:13.573901 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.521453 29113 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:13.567168 29147 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:13.527274 29146 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:13.630311 22761 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:13.647548 28843 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.648324 29144 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:13.648674 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.649149 22761 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:13.650722 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.651575 22906 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:14:13.652392 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.644068 29164 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:13.653527 29129 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:13.653945 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.654007 29232 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:14:13.654812 22906 raft_consensus.cc:2368] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:14:13.654644 29129 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.655689 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:13.656234 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.539831 28940 raft_consensus.cc:2764] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:14:13.657078 29171 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:13.657388 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.658037 29215 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.658610 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.659832 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:13.635855 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.401644 29179 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.622591 29159 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:13.637217 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.661880 22759 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:13.643415 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:13.645895 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72120.8us
I20260709 13:14:13.574364 28816 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:14:13.634990 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 118953us
I20260709 13:14:13.657588 29171 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.644141 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.539865 29127 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:14:13.664008 22914 raft_consensus.cc:2410] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:14:13.665683 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.664994 22805 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:13.666445 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:13.669238 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.668212 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:13.542311 28384 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:14:13.567603 29154 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:14:13.566891 29226 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:14:13.729693 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.707855 22761 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:13.642661 29113 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.707957 22915 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 12: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:13.742667 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.744100 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.740697 22914 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:13.670297 29143 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:13.732368 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:13.663911 29098 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13.
I20260709 13:14:13.646544 29147 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:13.644999 29182 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:13.672145 29164 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:13.757020 22906 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:13.644321 29062 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:13.757414 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89335.9us
I20260709 13:14:13.745395 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.675693 29171 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.666874 29163 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:13.765489 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:13.781834 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 116928us
W20260709 13:14:13.782538 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 116963us
I20260709 13:14:13.854792 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.763512 29130 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:13.864061 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 197693us
I20260709 13:14:13.758329 29159 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:13.866474 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.867725 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:13.812373 29147 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:13.886245 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.855993 29180 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:13.705739 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:13.892542 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.893884 22910 raft_consensus.cc:606] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.894340 22910 raft_consensus.cc:614] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:13.900839 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47150.1us
I20260709 13:14:13.855660 29146 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:13.901804 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.909938 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:13.911412 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:13.916401 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:13.919775 22906 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:13.884686 22907 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:13.875025 29204 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:13.880751 29163 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:13.932462 22915 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:13.934260 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:13.900530 28816 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:13.856580 29132 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:13.874724 29130 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:13.919833 29226 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:13.905669 29143 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:14.013983 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:13.930809 29204 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:13.897667 29205 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.422s
W20260709 13:14:14.034125 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 364292us
I20260709 13:14:14.034677 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:13.919719 28923 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:14:13.968981 29180 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:13.966334 29233 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:13.911206 28384 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 13:14:13.991688 29135 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:14.036092 29116 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:14.092976 22912 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:14.052295 29204 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:14.094249 29135 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.130215 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:14.134204 22912 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:14:14.085848 29233 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.099391 29179 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:14.107630 22906 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
W20260709 13:14:14.082691 29235 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:14:14.043977 29164 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.060494 29159 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.140514 29135 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:14.050308 29182 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:14.224628 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 313264us
I20260709 13:14:14.148671 29233 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:14.236783 22915 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:14.253405 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:14.175977 29116 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:14.203014 29130 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:14.215776 29099 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:14.179657 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:14.236724 29231 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.497s	user 0.001s	sys 0.001s
I20260709 13:14:14.290006 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:14.290736 22912 raft_consensus.cc:2368] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
W20260709 13:14:14.323519 29239 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 13:14:14.325637 29241 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.283s	user 0.004s	sys 0.000s
I20260709 13:14:14.326071 28384 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.330775 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:14.331265 29215 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.000s
I20260709 13:14:14.334125 28923 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:14.335044 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63705.8us
W20260709 13:14:14.358512 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 442177us
I20260709 13:14:14.358951 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:14.359809 22834 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:14.343524 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 422014us
I20260709 13:14:14.144656 29163 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.368650 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:14.368839 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:14.369357 22832 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:14.363168 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.364228 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:14.373982 22833 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:14.366820 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:14.367066 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:14.384328 22841 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:14.388674 22763 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:14:14.391634 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.392752 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:14.394249 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:14.363791 29215 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:14.289114 29116 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.342796 29099 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.159752 28816 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.395768 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:14.401330 22839 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:14.382846 29229 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:14.344527 29143 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:14:14.414748 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:14.386340 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:14.425837 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 66346.9us
I20260709 13:14:14.411749 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:14.383641 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 322361us
W20260709 13:14:14.445395 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 59177.1us
I20260709 13:14:14.360737 28384 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.456287 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:14.439028 23461 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:14.454347 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:14.373942 29146 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:14:14.461535 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:14.463497 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:14.466284 22732 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.671s (SENT)
I20260709 13:14:14.468431 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:14.469919 22913 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:14.470001 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:14.454445 29236 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.434s	user 0.000s	sys 0.002s
I20260709 13:14:14.390004 29141 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:14:14.470908 29079 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:14:14.465792 29249 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.383503 29252 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
W20260709 13:14:14.467801 29237 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.407s	user 0.000s	sys 0.001s
I20260709 13:14:14.455335 29133 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:14.489111 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2521) took 1795 ms. Trace:
I20260709 13:14:14.472312 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:14.498652 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:14.495690 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55169.2us
I20260709 13:14:14.467638 29215 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:14.503607 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:14.475958 29226 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:14.487006 29238 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.400s	user 0.001s	sys 0.000s
I20260709 13:14:14.516726 23466 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:14.476197 29147 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.004s
W20260709 13:14:14.518051 29234 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.531s	user 0.003s	sys 0.000s
I20260709 13:14:14.520255 29168 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:14.522032 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:14.570449 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:14.522495 28874 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:14.535326 29116 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:14.536716 29099 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:14.496490 29113 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.005s
I20260709 13:14:14.544816 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:14.546849 22833 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:14.513123 22758 rpcz_store.cc:276] 0709 13:14:12.678908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.686108 (+  7200us) service_pool.cc:224] Handling call
0709 13:14:12.686563 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:12.710636 (+ 24073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:12.710671 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:12.720701 (+ 10030us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.306702 (+586001us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:13.306749 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.306823 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.474006 (+1167183us) raft_consensus.cc:1685] finished
0709 13:14:14.474030 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.474187 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31127,"raft.queue_time_us":30809,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":22506,"thread_start_us":438560,"threads_started":1,"wal-append.queue_time_us":874226}
I20260709 13:14:14.476630 29132 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:14.487593 29251 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:14.554590 29241 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:14:14.595376 22757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.025s	sys 0.014s
W20260709 13:14:14.597539 22757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.026s	sys 0.014s
I20260709 13:14:14.633319 22912 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:14.576536 28874 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:14.648159 22906 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:14.632046 29237 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:14:14.630039 29129 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:14.657406 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3155) took 1065 ms. Trace:
I20260709 13:14:14.573961 29147 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.644897 29130 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:14.661067 22765 rpcz_store.cc:276] 0709 13:14:13.592438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.596636 (+  4198us) service_pool.cc:224] Handling call
0709 13:14:13.597040 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.619290 (+ 22250us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:13.640537 (+ 21247us) log.cc:844] Serialized 14 byte log entry
0709 13:14:13.777557 (+137020us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.786501 (+  8944us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.847474 (+ 60973us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:13.852550 (+  5076us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.324967 (+472417us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:14.344761 (+ 19794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.344863 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.657116 (+312253us) raft_consensus.cc:1685] finished
0709 13:14:14.657145 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.657329 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26231,"prepare.queue_time_us":272124,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":9170,"raft.queue_time_us":40915,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":31467,"spinlock_wait_cycles":17036032,"thread_start_us":625558,"threads_started":2,"wal-append.queue_time_us":285653}
I20260709 13:14:14.645979 28384 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:14.674234 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2538) took 1289 ms. Trace:
W20260709 13:14:14.675325 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 201624us
I20260709 13:14:14.586287 29113 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.674894 22760 rpcz_store.cc:276] 0709 13:14:13.384932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.385081 (+   149us) service_pool.cc:224] Handling call
0709 13:14:13.385410 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.385794 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.385825 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.386369 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.773877 (+387508us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:13.849934 (+ 76057us) log.cc:844] Serialized 14 byte log entry
0709 13:14:13.850173 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.850260 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.673910 (+823650us) raft_consensus.cc:1685] finished
0709 13:14:14.673945 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.674147 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":77873,"prepare.run_wall_time_us":50,"raft.queue_time_us":17981,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":9921,"spinlock_wait_cycles":11990272,"thread_start_us":387012,"threads_started":1,"wal-append.queue_time_us":539137}
W20260709 13:14:14.683661 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 214634us
W20260709 13:14:14.676613 29204 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
W20260709 13:14:14.704311 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 286294us
I20260709 13:14:14.711375 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:14.713908 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 257698us
I20260709 13:14:14.497809 29253 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.226s
I20260709 13:14:14.586768 29262 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.167s
I20260709 13:14:14.724149 29253 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:14.724670 29253 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:14.516515 29249 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.728235 29249 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:14.671530 29130 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.738626 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3346) took 1658 ms. Trace:
I20260709 13:14:14.741295 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:14.766394 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:14.729481 29180 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:14.743389 22911 rpcz_store.cc:276] 0709 13:14:12.915683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.917642 (+  1959us) service_pool.cc:224] Handling call
0709 13:14:12.918061 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.257942 (+339881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.257973 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.372031 (+114058us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b2c00051d28. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:13.372451 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.775812 (+403361us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:13.775882 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.775988 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.573178 (+797190us) raft_consensus.cc:1685] finished
0709 13:14:14.573212 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.573555 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350640,"raft.queue_time_us":341387,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":30133120,"thread_start_us":402949,"threads_started":1,"wal-append.queue_time_us":533470}
I20260709 13:14:14.700120 29229 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:14:14.703087 29204 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.778421 29264 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:14.789258 22834 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:14.770895 22832 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:14.802686 29264 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:14.806301 29264 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:14.806299 29062 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:14.808864 29062 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:14.809000 22840 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: [14.9-17.10]
I20260709 13:14:14.776744 29253 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.814606 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3361) took 1304 ms. Trace:
I20260709 13:14:14.815059 22913 rpcz_store.cc:276] 0709 13:14:13.510233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.510435 (+   202us) service_pool.cc:224] Handling call
0709 13:14:13.510726 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:14.475610 (+964884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.814312 (+338702us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b540010fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:14.814533 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205500672}
W20260709 13:14:14.816883 29260 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:14:14.817564 29244 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.393s	user 0.002s	sys 0.000s
I20260709 13:14:14.818807 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3159) took 1214 ms. Trace:
W20260709 13:14:14.819660 29266 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:14:14.819187 22764 rpcz_store.cc:276] 0709 13:14:13.604862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.628842 (+ 23980us) service_pool.cc:224] Handling call
0709 13:14:13.637775 (+  8933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.652085 (+ 14310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.652118 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.652675 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.915826 (+263151us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:14.342055 (+426229us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.344535 (+  2480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.344638 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.818545 (+473907us) raft_consensus.cc:1685] finished
0709 13:14:14.818570 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.818737 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3695,"prepare.queue_time_us":243919,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":6146,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":10652,"spinlock_wait_cycles":11825792,"thread_start_us":671976,"threads_started":2,"wal-append.queue_time_us":495471}
W20260709 13:14:14.822467 29265 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 13:14:14.822734 29286 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:14:14.743158 29132 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.080s
I20260709 13:14:14.823834 29132 consensus_queue.cc:1243] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:14:14.735327 29158 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
W20260709 13:14:14.827823 29233 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.563s	user 0.005s	sys 0.007s
I20260709 13:14:14.774966 29261 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:14.833869 29261 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:14.834347 29261 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:14.838245 29269 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:14.838654 29282 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.839193 29282 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.839999 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3153) took 1248 ms. Trace:
W20260709 13:14:14.840761 29268 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:14:14.840427 22757 rpcz_store.cc:276] 0709 13:14:13.591682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.595855 (+  4173us) service_pool.cc:224] Handling call
0709 13:14:13.596312 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.621754 (+ 25442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.621805 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.634651 (+ 12846us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.659964 (+ 25313us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b180032e120. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:14:14.599517 (+939553us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:14.644053 (+ 44536us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.737473 (+ 93420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.737572 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.839662 (+102090us) raft_consensus.cc:1685] finished
0709 13:14:14.839789 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.839935 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44766,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":37345,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":22878,"spinlock_wait_cycles":32811904,"thread_start_us":1032346,"threads_started":2,"wal-append.queue_time_us":1057858}
I20260709 13:14:14.842162 29282 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:14.688961 28874 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.851539 22906 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:14.869764 28923 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:14.872417 28923 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:14.858089 29233 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.876741 29233 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:14.878336 29252 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:14.880570 22835 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:14.804870 29130 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:14.954525 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:14.960350 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.959904 29264 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:14.961948 29135 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.723s	user 0.008s	sys 0.010s
I20260709 13:14:14.962453 29135 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.964888 29135 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:14.966945 29117 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.967583 29117 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:14.967955 29130 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:14.968580 29130 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.971455 29130 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:14.974525 29261 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.974946 29283 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:14.975709 29283 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:14.976868 29261 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:14.978096 29113 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:14.981055 23466 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:14.988590 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:14.989934 22765 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:14:14.992463 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:14.993310 22757 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:14.940939 29233 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:14.942204 29248 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:15.006187 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:14.993345 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 249451us
I20260709 13:14:15.028177 29132 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:15.029774 28384 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:15.030285 28384 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:15.031360 28384 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.033282 28384 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:15.043093 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:15.046232 22841 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.006s	sys 0.014s
W20260709 13:14:15.048231 22841 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.006s	sys 0.014s
W20260709 13:14:15.049023 22838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.006s	sys 0.014s
I20260709 13:14:15.049402 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:15.051525 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 507063us
I20260709 13:14:15.052127 23461 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:15.050582 29171 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.182s	user 0.015s	sys 0.000s
I20260709 13:14:15.062263 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:15.083787 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 539097us
I20260709 13:14:15.063002 22912 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:15.059290 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:15.066092 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:15.094888 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.081207 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:15.078352 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:15.096056 22766 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:15.064218 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:15.063313 22838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.006s	sys 0.015s
I20260709 13:14:15.018747 29248 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:15.169293 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74511.5us
I20260709 13:14:15.170259 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.170346 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:15.170992 22762 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:15.017091 29233 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:15.105868 29168 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 13:14:15.063792 29147 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:15.189666 22909 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
W20260709 13:14:15.196759 29281 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:14:15.197739 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.199076 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:15.201121 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 122232us
I20260709 13:14:15.171262 29283 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:15.180589 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 135633us
I20260709 13:14:15.173672 29155 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.174642 29274 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.180357 29251 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:15.213439 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.186761 29290 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:15.190387 29164 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.193531 29259 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:15.106178 29171 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.204985 29174 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:15.224238 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:15.192075 29062 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.238618 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.247814 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.220749 29259 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.259701 29283 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:15.250948 29154 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:15.222626 29290 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.238890 29251 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.235358 29274 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:15.216918 29155 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.286945 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:15.288805 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.215013 22762 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:15.283303 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 217316us
W20260709 13:14:15.347314 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58624.6us
I20260709 13:14:15.215663 29129 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:14:15.230132 29164 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.353108 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.355803 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2082) took 1714 ms. Trace:
W20260709 13:14:15.355018 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 295826us
I20260709 13:14:15.344758 22766 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:15.278237 29259 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:15.356300 22909 rpcz_store.cc:276] 0709 13:14:13.634739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.635593 (+   854us) service_pool.cc:224] Handling call
0709 13:14:15.348837 (+1713244us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540020add4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:15.348951 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3515813632}
I20260709 13:14:15.361536 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:15.363925 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.362419 22836 raft_consensus.cc:2376] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15. Re-sending same reply.
I20260709 13:14:15.387912 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:15.390224 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:15.348688 29205 raft_consensus.cc:2764] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:14:15.390268 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:15.393455 23466 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:15.386794 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:15.356784 29283 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:15.388389 29226 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.000s
I20260709 13:14:15.383870 29171 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:15.437425 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:15.389828 29079 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:15.468086 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:15.404327 29251 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:14:15.433490 29289 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:14:15.445722 29279 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
I20260709 13:14:15.480938 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:15.414670 29154 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:15.469688 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 116816us
I20260709 13:14:15.435693 29226 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:15.468791 29283 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:15.496131 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:15.469120 29294 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.485196 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:15.494541 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 391723us
I20260709 13:14:15.491346 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.496883 22836 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:15.474758 29251 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:15.529773 29251 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.530735 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:15.531153 29251 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:15.531229 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:15.502581 29301 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:14:15.534049 29296 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:15.534644 29296 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.536966 29296 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:15.505064 29277 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.339s	user 0.001s	sys 0.001s
I20260709 13:14:15.505460 29276 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.506000 29306 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.538100 29306 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.539209 29294 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.541065 29306 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:15.507637 29292 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 13:14:15.558581 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:15.508983 29233 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:15.509979 29249 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:15.514846 29290 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:15.517517 29295 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:15.562469 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2075) took 1154 ms. Trace:
I20260709 13:14:15.562857 22841 rpcz_store.cc:276] 0709 13:14:14.407910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.408942 (+  1032us) service_pool.cc:224] Handling call
0709 13:14:14.409407 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.409908 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.409948 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.416925 (+  6977us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.048742 (+631817us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:15.048793 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.048885 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.562165 (+513280us) raft_consensus.cc:1685] finished
0709 13:14:15.562264 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.562408 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3090,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":11539,"thread_start_us":572276,"threads_started":1,"wal-append.queue_time_us":904999}
I20260709 13:14:15.476845 29079 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:15.523869 29215 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.972s	user 0.012s	sys 0.000s
I20260709 13:14:15.525290 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2078) took 1109 ms. Trace:
I20260709 13:14:15.526548 29155 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:15.527796 29147 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:15.528154 29283 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.532861 23466 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:15.611629 29296 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:15.503434 29062 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:15.480614 29248 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:15.629359 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:15.471326 29285 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:15.543267 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:15.632215 22834 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:15.482935 29206 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:15.545452 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 54271.2us
I20260709 13:14:15.638877 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.638671 22836 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:15.546720 28874 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:15.506637 29297 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 13:14:15.550076 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ad66b5c9842437d9320507b92b78fd5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:14:15.550343 29302 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:15.550264 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:15.537139 29275 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:15.554924 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:15.555496 23462 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:15.648883 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 98713us
I20260709 13:14:15.557329 23463 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:15.559532 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2554) took 1015 ms. Trace:
I20260709 13:14:15.566008 22833 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:15.567807 22839 rpcz_store.cc:276] 0709 13:14:14.415768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.454394 (+ 38626us) service_pool.cc:224] Handling call
0709 13:14:14.454823 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.457127 (+  2304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.457159 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.470778 (+ 13619us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.731549 (+260771us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:14.731599 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.731675 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.525040 (+793365us) raft_consensus.cc:1685] finished
0709 13:14:15.525064 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.525206 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1880,"raft.queue_time_us":6163,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":7372,"thread_start_us":200976,"threads_started":1,"wal-append.queue_time_us":472281}
I20260709 13:14:15.603621 29215 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:15.614297 29294 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:15.615219 29079 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:15.618937 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:15.621031 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:15.631081 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:15.630585 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:15.635286 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:15.660277 22839 pending_rounds.cc:85] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 16
I20260709 13:14:15.641588 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:15.660750 22839 pending_rounds.cc:107] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:14:15.661124 22839 raft_consensus.cc:2889] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:15.643448 22649 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 93439.3us
I20260709 13:14:15.643808 22732 leader_election.cc:400] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:14:15.662387 22732 leader_election.cc:403] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:14:15.649811 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:15.613008 29259 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.650275 22763 rpcz_store.cc:276] 0709 13:14:14.544409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.545801 (+  1392us) service_pool.cc:224] Handling call
0709 13:14:14.546156 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.556090 (+  9934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.556129 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.591082 (+ 34953us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.018211 (+427129us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:15.018265 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.018364 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.559187 (+540823us) raft_consensus.cc:1685] finished
0709 13:14:15.559218 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.559456 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8651,"raft.queue_time_us":9951,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":11000,"spinlock_wait_cycles":2142720,"thread_start_us":366475,"threads_started":1,"wal-append.queue_time_us":597321}
I20260709 13:14:15.658723 23458 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:15.656674 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:15.666067 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.641336 22836 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:15.663712 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:15.668975 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.658648 22909 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:15.659328 22913 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
W20260709 13:14:15.670002 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.676662 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:15.648000 29155 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:15.679847 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:15.669170 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:15.683894 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.654644 29248 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:15.630220 29147 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.684329 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:15.698371 22834 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:15.690536 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.644435 29296 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:14:15.664434 29062 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:15.650332 29206 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.678807 29174 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:14:15.726506 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:15.654839 29274 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:15.675128 28843 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.644629 29129 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:14:15.732029 29155 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:15.682750 29249 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:15.682914 29002 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.683120 29294 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:15.697548 29189 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:15.713364 29113 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:15.730347 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:15.689415 29285 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:15.697721 29306 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:15.688037 29275 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:15.732641 29155 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.718659 29312 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:15.681197 28947 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:14:15.697304 29079 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:14:15.772250 22841 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:15.740381 29221 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:15.769698 29113 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:15.772330 29062 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:15.777575 29312 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:15.740783 29249 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:15.838224 29221 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.816480 29002 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.847690 29113 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:15.864605 29249 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:15.776702 29294 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.858819 29306 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:15.875665 29252 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:14:15.903190 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2467) took 1536 ms (client timeout 2036 ms). Trace:
W20260709 13:14:15.901861 29317 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 13:14:15.903751 22835 rpcz_store.cc:269] 0709 13:14:14.366475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.366734 (+   259us) service_pool.cc:224] Handling call
0709 13:14:15.902990 (+1536256us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54000c80d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:15.903123 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147928448}
I20260709 13:14:15.890116 29221 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:15.904599 29324 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:14:15.847533 29274 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.903313 29252 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:15.962294 29327 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:15.964003 29147 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:15.966257 29325 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:14:15.908604 29275 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:15.968007 29259 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:15.970570 29294 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:15.973182 29306 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.946012 29312 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.941752 28384 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.986495 28384 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:15.990474 29253 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:15.949630 29328 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:14:15.995959 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.010495 22835 raft_consensus.cc:2410] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:14:16.040833 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.010347 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:16.011971 22763 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:15.994458 28384 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:16.016749 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.023707 22762 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:14:16.027112 22766 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:16.026160 29252 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.046116 29252 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:16.051609 29113 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.053661 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.054348 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:16.055445 29204 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.057353 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.058490 22758 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:14:16.059921 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.060573 22762 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:16.078737 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:16.081642 29206 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.088068 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.023736 29155 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.093840 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:16.097595 29054 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:16.097424 29182 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:16.105451 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:16.111152 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 420718us
I20260709 13:14:16.093933 29206 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.122689 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:16.124109 29135 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:16.144970 29135 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.145493 29135 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:16.109339 29294 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:16.164350 29294 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:16.165485 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44440us
I20260709 13:14:16.102846 28384 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:14:16.178104 22762 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:14:16.183751 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61177.8us
I20260709 13:14:16.164987 29294 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:16.181384 29282 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.188504 29054 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:14:16.213600 22914 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:16.214829 22914 raft_consensus.cc:606] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:16.215205 22914 raft_consensus.cc:614] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:16.184060 29147 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.126138 29206 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.186735 29113 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.190081 29336 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:16.208572 29296 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.219333 29345 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:16.207363 22732 connection.cc:256] RPC call timeout handler was delayed by -1.5545552680000583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24940000000000004s for a response
W20260709 13:14:16.213563 29338 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:14:16.222131 29337 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:14:16.221628 29215 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:16.248952 29345 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.210278 29251 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:14:16.286687 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:16.298022 28874 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.005s
I20260709 13:14:16.222750 29282 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.266590 29215 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:16.238227 29147 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:16.315929 29335 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:16.317540 29206 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.332020 29221 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.247808 29296 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.311388 29251 consensus_queue.cc:1243] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:16.299973 29249 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.338002 29079 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.351047 29221 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:16.356767 29282 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.343317 29306 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:16.364898 29345 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.366730 29147 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.368522 29296 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.369894 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:16.379730 23459 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:16.385818 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 433604us
I20260709 13:14:16.387080 29117 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.393358 29264 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:16.394519 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2076) took 1983 ms. Trace:
I20260709 13:14:16.375613 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:16.377808 23430 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.571s (SENT) [suppressed 44 similar messages]
I20260709 13:14:16.396953 22763 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:16.394889 22838 rpcz_store.cc:276] 0709 13:14:14.410702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.411257 (+   555us) service_pool.cc:224] Handling call
0709 13:14:14.411679 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.413584 (+  1905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.413631 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.468061 (+ 54430us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.822429 (+354368us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b4800130158. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:14:15.097171 (+274742us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:15.097234 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.097333 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.392099 (+1294766us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b240029f198. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea42fb0 00007f68a16b9488 00007f68a16b9be8 00007f68a1691398 00007f68a168c76a 00007f68a168bbb6
0709 13:14:16.394205 (+  2106us) raft_consensus.cc:1685] finished
0709 13:14:16.394228 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.394445 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1386,"raft.queue_time_us":4555,"raft.run_cpu_time_us":2825,"raft.run_wall_time_us":395639,"spinlock_wait_cycles":43877504,"thread_start_us":595097,"threads_started":1,"wal-append.queue_time_us":709295}
I20260709 13:14:16.403033 29249 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:16.404317 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 198115us
I20260709 13:14:16.355019 29079 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.399209 28947 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.410557 28947 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.412789 28947 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.399009 29148 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.357695 29135 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.420029 29002 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:16.420519 29002 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:16.422107 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:16.422962 22762 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:16.355031 29215 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.420193 29135 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:16.447942 22763 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:16.460892 22762 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:16.465288 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:16.372686 29306 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.467393 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:16.473232 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:16.473536 22907 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 12.
I20260709 13:14:16.471504 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:16.476174 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:16.477028 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:16.478113 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:16.426520 29345 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.475733 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:16.487959 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:16.488710 23457 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:16.427466 29254 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.489900 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:16.494311 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:16.413937 29148 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.495177 22910 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:16.430572 29180 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.494648 22906 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:16.487344 29158 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:14:16.501905 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:16.503288 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.504235 22841 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:14:16.505249 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.505553 29254 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.507175 29148 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.507799 29254 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.505941 22838 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:16.497116 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:14:16.510923 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 133163us
I20260709 13:14:16.519029 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.421356 29002 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.492051 29215 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:16.524115 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2077) took 2105 ms. Trace:
I20260709 13:14:16.547617 22840 rpcz_store.cc:276] 0709 13:14:14.412308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.412460 (+   152us) service_pool.cc:224] Handling call
0709 13:14:14.808645 (+396185us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:14.809473 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.513698 (+1704225us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54001ff4d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:16.517147 (+  3449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910196992}
I20260709 13:14:16.525128 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:16.526252 29355 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:14:16.528860 29283 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.529312 29001 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:16.510394 29180 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.530900 29129 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:16.530830 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:16.537657 22764 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:16.539394 29206 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:16.515285 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:16.539924 29336 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:16.544025 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47022.5us
I20260709 13:14:16.509155 29253 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.498754 29155 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.496286 29275 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.503400 29054 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:16.572175 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.551131 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:16.539691 22759 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:14:16.556772 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.573964 22762 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:16.574540 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:16.575146 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:16.575713 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.576462 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.577380 22840 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:16.577945 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.558377 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.558274 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:16.553181 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:16.552680 29350 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:14:16.574833 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.503767 29345 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.611793 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:16.615684 22764 raft_consensus.cc:2368] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18.
I20260709 13:14:16.552480 29282 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:16.619133 29282 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.619611 29282 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:16.553261 29001 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.570348 29206 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.576722 29204 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.630225 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:16.633689 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:16.635398 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1093) took 1277 ms (client timeout 1571 ms). Trace:
I20260709 13:14:16.625469 29001 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:16.637140 22806 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.890s (SENT)
W20260709 13:14:16.635926 22833 rpcz_store.cc:269] 0709 13:14:15.358544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.358738 (+   194us) service_pool.cc:224] Handling call
0709 13:14:16.635212 (+1276474us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:16.635319 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402517504}
I20260709 13:14:16.568468 29002 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:16.644356 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.644214 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:16.647087 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:16.640525 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:16.651546 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:16.552937 29283 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:16.654943 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.901s (SENT)
I20260709 13:14:16.656777 22906 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:16.654889 22910 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:16.626026 29054 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:16.554248 29129 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:16.568792 29294 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.646472 29206 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.577028 29253 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.643209 29345 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:16.659852 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:16.714668 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54897.7us
I20260709 13:14:16.611666 29275 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.648488 29296 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:16.625734 29204 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.576645 29155 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.711854 22759 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:16.570626 29336 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:16.796530 29324 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:14:16.797407 29180 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:16.890365 29358 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:14:16.882172 29348 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.718662 29249 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:16.715570 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:16.894860 29141 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:16.925781 23457 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:16.930506 29204 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.932400 29155 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:16.934085 29356 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:14:16.934921 29253 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.936302 29129 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.892464 29054 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:16.971089 29262 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:16.927246 29296 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:16.977945 29296 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:16.983767 29164 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:16.914477 29294 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:16.947525 29336 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:16.966516 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 251035us
W20260709 13:14:16.971522 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 70558.2us
I20260709 13:14:16.973577 29365 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:17.004501 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2080) took 2008 ms. Trace:
I20260709 13:14:16.912909 29349 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:16.999054 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.001642 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.011459 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.019780 22836 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:17.028851 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.029738 22836 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:17.033085 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 137550us
I20260709 13:14:17.005041 22832 rpcz_store.cc:276] 0709 13:14:14.996150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.003178 (+  7028us) service_pool.cc:224] Handling call
0709 13:14:15.003535 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.700669 (+697134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.700703 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:15.700745 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:15.700788 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.004237 (+1303449us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540017d9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:17.004294 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.004437 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21427,"raft.queue_time_us":18028,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":17333,"spinlock_wait_cycles":1561577216}
I20260709 13:14:17.044831 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.045836 23461 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:17.046833 23458 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:17.047685 23463 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:14:17.049016 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.050179 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.050953 23458 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:17.051396 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:17.057121 22732 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.010s
W20260709 13:14:17.058674 22732 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.007s	sys 0.010s
W20260709 13:14:17.060000 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 514510us
I20260709 13:14:16.989480 29283 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:16.966513 29348 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.003798 29336 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:16.972049 29141 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.034214 29262 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:17.034179 29365 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.098052 23461 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:17.036938 29147 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:17.086212 22913 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:17.076968 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:17.103329 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.104773 22806 leader_election.cc:400] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18.
I20260709 13:14:17.105134 22806 leader_election.cc:403] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:14:17.086495 29148 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:14:17.108451 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.071880 29361 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:17.039446 29349 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.081869 29179 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:14:17.100334 29251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.007s	sys 0.019s
I20260709 13:14:17.109333 22759 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:17.130316 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.143218 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:17.144291 22759 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.143389 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.148996 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.151175 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.101046 29365 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:17.106014 29229 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:17.142405 29282 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.004s
W20260709 13:14:17.140942 29251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.010s	sys 0.022s
I20260709 13:14:17.124759 28923 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:17.110759 28843 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:17.171142 29204 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:17.095882 29336 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.171277 28923 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:17.174716 28923 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.176365 28923 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:17.181861 29054 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:17.182559 29054 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:17.183609 29339 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.920s	user 0.002s	sys 0.005s
I20260709 13:14:17.163103 29377 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:17.155992 29148 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:17.194602 29054 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.195465 29275 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:17.168678 29369 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.253s	user 0.000s	sys 0.002s
I20260709 13:14:17.197734 29348 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:17.198037 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.155241 29253 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.199928 29306 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.201458 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.202198 22835 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:17.141148 29361 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.203387 22832 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:17.204979 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2575) took 1025 ms. Trace:
I20260709 13:14:17.205552 29174 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:17.206105 29174 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:17.205432 22758 rpcz_store.cc:276] 0709 13:14:16.179136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.179690 (+   554us) service_pool.cc:224] Handling call
0709 13:14:16.192508 (+ 12818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.262739 (+ 70231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.262794 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:16.263523 (+   729us) log.cc:844] Serialized 26 byte log entry
0709 13:14:16.336972 (+ 73449us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:16.337016 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.204553 (+867537us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b54003408d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:17.204610 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.204661 (+    51us) raft_consensus.cc:1685] finished
0709 13:14:17.204684 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.204909 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14330,"raft.queue_time_us":3207,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":30237,"spinlock_wait_cycles":157018240,"thread_start_us":73041,"threads_started":1,"wal-append.queue_time_us":84567}
I20260709 13:14:17.207279 29174 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.208412 29174 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.208559 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.212749 22833 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:14:17.226305 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.226960 29079 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.227665 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:17.219009 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.220286 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.228660 22766 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:17.228972 22839 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:14:17.221891 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.229805 29361 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:17.230996 22835 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:17.232316 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.234088 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.234298 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:17.234628 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.234786 22832 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:17.235091 22910 raft_consensus.cc:2368] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20.
I20260709 13:14:17.235738 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:17.236310 22906 raft_consensus.cc:2376] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18. Re-sending same reply.
I20260709 13:14:17.237720 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.239516 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.240144 22906 raft_consensus.cc:2368] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:17.240995 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:17.241683 22910 raft_consensus.cc:2376] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16. Re-sending same reply.
W20260709 13:14:17.243471 29366 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.541s	user 0.008s	sys 0.000s
I20260709 13:14:17.243763 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.213176 29349 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:17.245432 22906 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:17.246670 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.248206 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.250255 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.256672 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.258809 22765 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:17.259423 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.259792 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:17.260453 22765 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:17.262092 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.263284 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.264029 22914 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:17.264941 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:17.265050 29363 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.565s	user 0.007s	sys 0.000s
I20260709 13:14:17.265627 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:17.266938 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.268316 29374 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:17.269235 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.269881 29283 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.270282 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.270548 29283 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.269922 22914 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:17.271961 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:17.272814 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:17.273546 29221 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:17.274354 29221 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:17.274472 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.275632 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.275362 29221 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.276599 28923 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.277161 28923 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.273615 29264 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:17.279904 22914 raft_consensus.cc:2368] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20.
W20260709 13:14:17.280107 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 112461us
I20260709 13:14:17.272790 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.268334 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.227756 29079 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:17.290544 22766 raft_consensus.cc:2410] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:14:17.312086 22766 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:17.291501 22765 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:17.279361 28923 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.307282 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:17.265144 29002 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 13:14:17.322234 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.320394 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.326784 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.327822 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.170082 22759 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:17.273250 29283 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.167778 29229 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:17.269074 29374 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.297712 29320 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.353542 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.353910 23457 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:17.320268 29179 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.350828 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.357183 23465 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:17.354351 22910 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:17.349570 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.355423 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.319615 29189 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.320698 29164 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:17.333966 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:17.361112 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.359923 22759 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:17.332363 29205 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:17.364835 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.367277 22838 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:17.365144 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.335961 29264 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:17.342948 29148 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.369578 22765 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:17.370501 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.349169 29283 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:17.333489 22758 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:17.285779 29282 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.376466 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.348331 29229 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.332978 29315 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:14:17.385942 29204 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:17.366637 29189 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.386484 29172 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:17.359791 29320 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.363539 29179 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.388581 29264 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.354167 29132 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:14:17.380088 29312 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:14:17.325986 29182 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:17.380077 29348 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:17.422618 29348 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.423244 29253 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18.
W20260709 13:14:17.425922 29215 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.875s	user 0.014s	sys 0.001s
I20260709 13:14:17.403072 29141 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.427433 29282 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.428231 23459 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
W20260709 13:14:17.429641 29367 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.325s	user 0.000s	sys 0.007s
I20260709 13:14:17.390234 29296 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.393613 29148 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.395675 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.400897 29144 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:17.359977 29365 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.332866 29373 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:17.331836 29079 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.454233 29079 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.411839 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.454846 29365 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.455086 22832 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:17.366103 29249 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:17.413539 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.417533 29315 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:17.420274 22760 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:17.435499 22834 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:17.408172 29283 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.467705 29283 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.468508 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.471261 29262 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:17.471830 29262 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:17.472463 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.461472 29144 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.473961 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.475409 29144 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.438264 29320 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:17.405301 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.425272 29348 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.443053 22835 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:17.451771 29374 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:17.459380 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 45956.2us
I20260709 13:14:17.414647 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.439558 29204 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:17.492062 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.418509 29233 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:14:17.461589 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.499580 22832 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:17.463622 22763 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:17.465066 29189 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.478308 29373 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:17.505091 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:17.507261 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.443676 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.496074 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.492627 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.500983 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:17.501549 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 86989.2us
I20260709 13:14:17.510655 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:17.509825 22910 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:17.516340 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.522030 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.527073 22836 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:14:17.528411 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84849us
I20260709 13:14:17.528604 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:17.504729 29283 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:17.539242 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.510972 29373 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.515370 29082 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:17.513579 29232 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:17.543081 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:17.547276 22836 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:17.543828 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.526497 29189 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:17.532106 29320 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:17.551687 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.555218 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.569631 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.570871 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.555253 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:17.573756 22764 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.006s	sys 0.016s
W20260709 13:14:17.574108 22764 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.006s	sys 0.016s
I20260709 13:14:17.574154 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.546288 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.543341 29082 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.564724 29179 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.541990 29373 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:17.580571 22833 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:17.538669 28923 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:17.527777 29365 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:14:17.494835 29262 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.537464 29283 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.588151 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3168) took 1047 ms. Trace:
I20260709 13:14:17.590484 22836 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:17.595027 22732 leader_election.cc:400] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20.
I20260709 13:14:17.661933 22732 leader_election.cc:403] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:14:17.660671 22764 rpcz_store.cc:276] 0709 13:14:16.541169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.639494 (+ 98325us) service_pool.cc:224] Handling call
0709 13:14:16.639869 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.640216 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.640246 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:16.641827 (+  1581us) log.cc:844] Serialized 26 byte log entry
0709 13:14:16.701431 (+ 59604us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:17.234981 (+533550us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b18000835c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:17.241988 (+  7007us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:17.574564 (+332576us) log.cc:844] Serialized 14 byte log entry
0709 13:14:17.587772 (+ 13208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.587871 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.587918 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:17.587930 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.588072 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32657,"prepare.queue_time_us":250243,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":3249,"raft.queue_time_us":335021,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":34603,"spinlock_wait_cycles":61467776,"thread_start_us":738566,"threads_started":2,"wal-append.queue_time_us":59640}
I20260709 13:14:17.629722 22840 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:17.547106 29232 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:17.624256 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:17.587275 29311 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.005s	sys 0.020s
I20260709 13:14:17.616858 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.673300 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:17.689196 22765 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:17.564837 29148 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:17.685201 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:17.620112 28923 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.674888 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:17.564904 29204 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.619940 29174 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:17.590692 29082 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:17.622032 29320 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:17.687644 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:17.587641 29189 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.694911 22760 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:14:17.702976 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 108087us
I20260709 13:14:17.722062 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:17.704174 22837 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:17.709539 29366 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:14:17.691413 29374 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:17.711683 28923 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.596014 29315 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.122s
W20260709 13:14:17.625111 29154 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.891s	user 0.020s	sys 0.019s
I20260709 13:14:17.620981 29249 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:17.688577 29311 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.005s	sys 0.020s
I20260709 13:14:17.670063 29283 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:17.682222 29232 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.733799 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.661011 29259 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:14:17.694522 29312 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.769961 29312 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.772975 29312 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.659054 29155 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:17.751936 29154 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.022s	user 0.021s	sys 0.019s
I20260709 13:14:17.742002 29374 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.694221 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.696635 29253 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.682549 29323 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:14:17.718654 29336 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:17.737888 29287 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.010s	sys 0.009s
I20260709 13:14:17.717890 29348 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:14:17.717370 29054 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:14:17.749275 29147 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:17.759461 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.765340 29002 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20.
I20260709 13:14:17.773818 29155 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.776175 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:17.808339 29287 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.010s	sys 0.009s
I20260709 13:14:17.730795 29189 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.789111 22837 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:17.756616 29132 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:17.791071 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.797425 22763 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:14:17.806691 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.009s	sys 0.050s
W20260709 13:14:17.830261 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.009s	sys 0.051s
W20260709 13:14:17.830931 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 679851us
I20260709 13:14:17.831233 22833 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:17.756879 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.736572 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.780921 29365 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:17.833284 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 76525us
I20260709 13:14:17.754102 29249 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:17.777432 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55499.6us
I20260709 13:14:17.834548 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:17.834690 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.837566 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.842150 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.844491 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.845052 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.836005 22841 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:17.754419 29320 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:17.825821 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:17.821326 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.854672 22837 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:17.819835 29306 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:17.846210 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:17.807060 29253 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:17.819468 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 485630us
I20260709 13:14:17.834626 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:17.848050 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:17.857627 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.859414 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.859908 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:17.861577 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:17.879551 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:17.849802 29174 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:17.844194 28816 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:17.848004 29275 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:17.850795 29365 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:17.822424 29229 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:17.829641 29054 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:14:17.903220 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:17.907313 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63500.3us
I20260709 13:14:17.869611 29306 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:17.855058 28843 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:17.852795 29132 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:17.872077 29274 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:17.873451 29321 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:17.851219 29312 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 13:14:17.883706 29383 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:17.938944 22834 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1101) took 1416 ms (client timeout 1883 ms). Trace:
I20260709 13:14:17.941655 22837 raft_consensus.cc:1217] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Deduplicated request from leader. Original: 10.6->[11.7-15.9]   Dedup: 11.7->[13.8-15.9]
I20260709 13:14:17.942464 22837 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: [11.7-15.9]
I20260709 13:14:17.955809 29262 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:17.939604 22834 rpcz_store.cc:269] 0709 13:14:16.522506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.522656 (+   150us) service_pool.cc:224] Handling call
0709 13:14:17.938650 (+1415994us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400321ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:17.938830 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018344704}
I20260709 13:14:17.900820 29275 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:17.924922 29174 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:17.926350 29321 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.939816 29215 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:17.959110 29251 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:17.975360 29321 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:17.971719 29275 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:17.986472 29382 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:14:17.987246 29229 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.988951 29253 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:17.928696 29141 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:14:18.019604 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:18.020558 23459 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:18.008025 29381 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:14:17.862607 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:18.042188 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175681us
I20260709 13:14:18.053027 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:18.055797 29249 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:18.063522 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3165) took 1526 ms. Trace:
I20260709 13:14:18.064110 22761 rpcz_store.cc:276] 0709 13:14:16.537020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.537256 (+   236us) service_pool.cc:224] Handling call
0709 13:14:16.537596 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.190076 (+652480us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b18001fe220. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:14:17.190236 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.190262 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.190933 (+   671us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.594975 (+404042us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:17.595021 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.595104 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.063132 (+468028us) raft_consensus.cc:1685] finished
0709 13:14:18.063161 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.063440 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87900,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9530,"spinlock_wait_cycles":18653056,"thread_start_us":503565,"threads_started":2,"wal-append.queue_time_us":627385}
I20260709 13:14:18.040660 29383 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:18.040002 29154 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:18.084106 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 393032us
I20260709 13:14:18.092691 29406 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:14:18.094448 29296 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:18.094906 29321 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.097615 29054 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:14:18.098479 29386 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:14:18.108520 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:18.109598 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:18.052994 29374 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.131309 22911 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:14:18.131891 22911 pending_rounds.cc:85] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 14
I20260709 13:14:18.132146 22911 pending_rounds.cc:107] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:14:18.132403 22911 raft_consensus.cc:2889] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:18.138804 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:18.123960 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 499793us
I20260709 13:14:18.005915 29306 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.042552 29365 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.104658 29155 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.094623 29349 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:18.157346 22910 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:18.123095 29189 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.149310 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:18.160368 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 325836us
I20260709 13:14:18.162544 22832 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:18.125223 29054 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:18.185611 22732 leader_election.cc:400] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:18.186935 29132 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:18.187170 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.188216 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.188000 29132 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.188438 29155 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:18.189468 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:18.145681 29296 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:18.113623 29154 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.195065 22841 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:18.195758 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:18.146488 29002 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:18.086858 29174 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:18.210263 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 362325us
I20260709 13:14:18.105383 29383 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:18.242470 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46649.5us
I20260709 13:14:18.254726 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:18.257849 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 349478us
I20260709 13:14:18.260128 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.267304 22762 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:18.267597 29127 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:18.166813 28923 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.196413 29390 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:18.202004 29306 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:18.280285 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 417801us
I20260709 13:14:18.241467 29155 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:18.296168 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.262629 29374 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:18.348031 22835 raft_consensus.cc:2368] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:18.300164 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.301667 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.203182 29189 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
W20260709 13:14:18.327099 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54488.2us
I20260709 13:14:18.350667 22840 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
W20260709 13:14:18.354503 29395 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:14:18.298931 29390 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:18.355264 29390 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:18.356166 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2591) took 1099 ms. Trace:
I20260709 13:14:18.356607 22766 rpcz_store.cc:276] 0709 13:14:17.257006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.361821 (+104815us) service_pool.cc:224] Handling call
0709 13:14:17.362233 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.457105 (+ 94872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.457154 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.462891 (+  5737us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.929488 (+466597us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:17.929557 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.102045 (+172488us) spinlock_profiling.cc:243] Waited 94.4 ms on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:18.102112 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.355808 (+253696us) raft_consensus.cc:1685] finished
0709 13:14:18.355844 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.356097 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1409,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":18939,"spinlock_wait_cycles":217101184,"thread_start_us":424958,"threads_started":1,"wal-append.queue_time_us":657915}
W20260709 13:14:18.358997 29402 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:14:18.360545 29154 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:18.363920 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56504.2us
I20260709 13:14:18.256088 29249 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
W20260709 13:14:18.366207 29398 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:18.366426 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.298086 29054 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.368901 29054 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:18.298905 29141 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
W20260709 13:14:18.351569 29082 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.004s
I20260709 13:14:18.373935 29155 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:18.374552 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73047.4us
I20260709 13:14:18.375471 29400 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:18.376032 29400 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:18.376652 29155 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:18.377672 29389 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:14:18.377025 29403 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:14:18.377008 29400 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.234025 29002 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.374142 29082 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.215335 29296 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.287681 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.354374 29321 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:18.393635 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 106332us
I20260709 13:14:18.394763 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.410375 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:18.414216 22839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.007s	sys 0.036s
W20260709 13:14:18.416427 22839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.008s	sys 0.036s
W20260709 13:14:18.417989 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 152855us
I20260709 13:14:18.395290 29189 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.400158 29374 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:18.429095 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 270085us
I20260709 13:14:18.455878 22915 raft_consensus.cc:606] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.408007 29135 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:18.407907 29154 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:18.455293 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.415761 29233 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:18.394918 29287 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:18.473379 29416 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:18.476951 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.467985 29158 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:14:18.478202 29164 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:18.479315 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:18.479146 29164 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.482921 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:18.484529 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.487931 22764 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:18.479422 29158 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:18.493013 29396 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:14:18.494469 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2101) took 1007 ms. Trace:
I20260709 13:14:18.494865 22838 rpcz_store.cc:276] 0709 13:14:17.486836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.486996 (+   160us) service_pool.cc:224] Handling call
0709 13:14:17.487439 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.487746 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.487779 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.494648 (+  6869us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.884943 (+390295us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:17.884995 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.885098 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.494214 (+609116us) raft_consensus.cc:1685] finished
0709 13:14:18.494237 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.494401 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":355024,"threads_started":1,"wal-append.queue_time_us":618007}
I20260709 13:14:18.498171 29296 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:18.500268 29232 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.004s
I20260709 13:14:18.500769 29232 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.502022 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.502967 29232 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:18.503799 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:18.504884 29002 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:18.505105 22758 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:18.424199 29411 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:18.522826 29383 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.508697 28923 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.526199 23459 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:14:18.531126 29283 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.756s	user 0.006s	sys 0.007s
W20260709 13:14:18.531687 29391 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.442s	user 0.000s	sys 0.002s
I20260709 13:14:18.533103 29253 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:18.533571 29253 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:18.534509 29253 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.536043 29253 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:18.490247 29251 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:18.539319 29251 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:18.508690 29135 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:18.454092 29189 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.466780 29233 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:18.515259 29082 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:18.518575 29287 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:14:18.572553 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 147231us
I20260709 13:14:18.578240 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:18.567982 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 371833us
W20260709 13:14:18.571810 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 196495us
I20260709 13:14:18.579650 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.541102 29132 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:18.544101 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:18.603008 29410 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:18.553257 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:18.604307 23459 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 23: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:18.603811 29410 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.554378 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:18.583367 22732 leader_election.cc:400] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20.
I20260709 13:14:18.607841 23459 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:14:18.609778 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.623713 29410 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:18.632030 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52493.2us
I20260709 13:14:18.633312 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.636339 23465 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:18.641973 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.642300 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.644829 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:18.645208 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.552520 29135 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:18.640394 29374 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.656795 29374 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.659291 29403 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:18.659448 29374 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:18.659878 29403 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:18.660977 29403 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.661975 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.662971 29189 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.640080 29154 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:18.665992 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.663641 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.668336 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.504928 29275 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.639871 29287 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:18.672616 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.531690 29283 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.620959 29251 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.677199 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.680929 29251 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:18.677683 22908 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.011s	sys 0.015s
W20260709 13:14:18.681864 22908 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.012s	sys 0.015s
I20260709 13:14:18.682265 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.643031 29164 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:18.663679 29408 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:18.689380 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.666235 29135 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:18.694811 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.655984 29412 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.701709 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:18.702562 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.708073 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:18.684376 29373 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.058s	user 0.013s	sys 0.002s
I20260709 13:14:18.697408 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:18.720110 22761 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:18.687546 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 926) took 1274 ms. Trace:
I20260709 13:14:18.716661 22765 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:18.728272 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.693773 29345 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:18.727898 22908 rpcz_store.cc:276] 0709 13:14:17.413572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.413878 (+   306us) service_pool.cc:224] Handling call
0709 13:14:17.414166 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:17.414423 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:14:17.782408 (+367985us) log.cc:844] Serialized 14 byte log entry
0709 13:14:18.682626 (+900218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.683058 (+   432us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:18.683130 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.683245 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.687457 (+  4212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25804,"prepare.queue_time_us":590856,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":80169,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":20776,"thread_start_us":1247937,"threads_started":3,"wal-append.queue_time_us":686733}
I20260709 13:14:18.698537 29172 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:14:18.758531 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 177419us
I20260709 13:14:18.737723 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.760227 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:18.736625 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:18.762669 22764 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:18.764510 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.767313 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:18.771624 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:18.774493 22911 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:14:18.779382 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 146209us
I20260709 13:14:18.781396 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:18.783957 22838 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:18.736429 29400 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:18.736625 29421 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:18.799817 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:18.768797 28843 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:18.769994 29365 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.774062 29296 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:18.731310 29373 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.772424 29127 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:18.834111 22911 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:18.769368 29388 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:14:18.835722 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 198596us
I20260709 13:14:18.837387 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:18.843343 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:18.782899 29413 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:18.850587 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75156.7us
I20260709 13:14:18.879946 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.839030 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:18.721396 29412 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:18.701810 29408 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.867635 22838 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:18.871598 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:18.883191 23465 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:18.896175 22908 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:18.900952 22911 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:14:18.902895 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63976.5us
I20260709 13:14:18.924726 22834 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:14:18.937794 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 66956.5us
I20260709 13:14:18.884677 29406 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:18.887178 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:18.833154 29400 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:18.907140 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.031577 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:18.967607 29428 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:14:18.971190 29427 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:14:18.972580 29418 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.476s	user 0.003s	sys 0.000s
I20260709 13:14:18.976526 29412 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.968755 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:18.988953 29275 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:18.991582 29373 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:18.857115 29365 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:18.995026 29408 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:18.960369 29306 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.000s
I20260709 13:14:18.860409 29390 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.000844 29282 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:18.968341 29253 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:19.008919 29439 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:14:18.929526 29249 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:18.961066 28923 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
W20260709 13:14:19.030380 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 143278us
I20260709 13:14:19.033290 22764 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:19.073151 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2506) took 1348 ms. Trace:
I20260709 13:14:19.107290 22759 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:19.121423 22839 rpcz_store.cc:276] 0709 13:14:17.724720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.846543 (+121823us) service_pool.cc:224] Handling call
0709 13:14:17.870961 (+ 24418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.457237 (+586276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.457379 (+   142us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:18.664904 (+207525us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:18.665341 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.834541 (+169200us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:19.032194 (+197653us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.032625 (+   431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.032783 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.032847 (+    64us) raft_consensus.cc:1685] finished
0709 13:14:19.072882 (+ 40035us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.073077 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19759,"raft.queue_time_us":95209,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":70559,"spinlock_wait_cycles":11265280,"thread_start_us":805719,"threads_started":3,"wal-append.queue_time_us":243114}
W20260709 13:14:19.111753 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 207763us
I20260709 13:14:19.111817 22909 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:14:19.066071 29141 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:19.116088 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 147494us
I20260709 13:14:19.130410 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.130681 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.131731 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.132215 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.132946 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.067387 29406 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.117511 29283 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.134061 22908 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:19.135326 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.137971 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.138561 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.139187 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:19.122462 29412 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:19.118386 29249 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.139657 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:19.118090 29259 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:19.145061 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.145815 22913 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:19.127302 29435 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.115622 29282 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.140595 29253 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:19.160766 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2610) took 1086 ms. Trace:
I20260709 13:14:19.170032 22763 rpcz_store.cc:276] 0709 13:14:18.074344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.074950 (+   606us) service_pool.cc:224] Handling call
0709 13:14:18.075715 (+   765us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:18.076577 (+   862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.076620 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:18.087252 (+ 10632us) log.cc:844] Serialized 52 byte log entry
0709 13:14:18.270556 (+183304us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:18.270619 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.270715 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.971824 (+701109us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:19.157494 (+185670us) raft_consensus.cc:1685] finished
0709 13:14:19.157539 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.160680 (+  3141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5874,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":10219,"spinlock_wait_cycles":62441856,"thread_start_us":127498,"threads_started":1,"wal-append.queue_time_us":382695}
I20260709 13:14:19.152251 29249 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:19.202068 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:19.203085 29321 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:19.203588 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:19.204416 29321 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:19.204900 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:19.205541 22838 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:19.206732 22840 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:19.208072 22838 raft_consensus.cc:1217] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Deduplicated request from leader. Original: 10.6->[11.7-15.9]   Dedup: 11.7->[13.8-15.9]
I20260709 13:14:19.140987 29113 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:14:19.208623 22838 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: [11.7-15.9]
I20260709 13:14:19.210866 29420 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:19.211436 29420 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:19.212437 29420 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:19.218333 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2495) took 1676 ms (client timeout 1540 ms). Trace:
W20260709 13:14:19.226637 29426 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:14:19.227317 29424 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:19.218829 22835 rpcz_store.cc:269] 0709 13:14:17.542133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.542336 (+   203us) service_pool.cc:224] Handling call
0709 13:14:19.218081 (+1675745us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:19.218247 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1837971840}
I20260709 13:14:19.227934 29424 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.231920 29274 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:19.233649 29082 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:19.234088 29082 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:19.235033 29082 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.242110 29082 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:19.243098 29406 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:19.195324 29253 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.244092 29424 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.246045 29253 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.235960 29434 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:19.246472 29437 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:19.247068 29434 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.247488 29434 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:19.248286 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.249154 29182 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:19.249984 29182 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:19.249387 22879 leader_election.cc:400] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:19.250726 29400 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.251021 22879 leader_election.cc:403] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:19.156291 29259 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.252997 29390 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.159395 29435 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.254715 29282 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.176456 29365 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.256423 29345 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.257001 29345 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:19.256544 29435 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:19.262085 22761 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:19.262146 29450 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:19.254705 29259 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:19.268747 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46252 (request call id 52) took 1403 ms. Trace:
I20260709 13:14:19.269205 22915 rpcz_store.cc:276] 0709 13:14:17.861201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.861373 (+   172us) service_pool.cc:224] Handling call
0709 13:14:19.264786 (+1403413us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54001257d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aeb853 00007f68a6ab960a 00007f68a003f7d4 00007f68a003f762 00007f68a003f691 00007f68a003f60c 00007f68a003e8c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:19.264870 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341291392}
I20260709 13:14:19.270236 22837 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:19.283838 29182 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:19.286590 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.287753 22837 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:19.289294 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.294073 22835 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:14:19.257205 29412 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:19.299114 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:19.158676 29135 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.303908 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:19.317963 29321 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:19.301713 29259 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:19.348172 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 220441us
I20260709 13:14:19.348599 22910 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:19.320915 29433 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:19.344194 29264 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:19.359014 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.366218 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.343889 29321 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:19.365024 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.377529 23461 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:19.376459 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:19.371953 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.379076 23459 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:19.376602 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:19.363950 29348 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:19.385818 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.383864 22908 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:19.353248 29400 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:14:19.353111 29397 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:14:19.389531 22838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
W20260709 13:14:19.391561 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.363948 29433 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.379979 29321 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:19.393646 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.398006 23461 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:19.394886 22838 raft_consensus.cc:606] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:19.384930 29348 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:19.350394 29259 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.396106 29433 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:19.424822 29457 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 13:14:19.428206 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.433441 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:19.436733 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 958) took 1169 ms. Trace:
I20260709 13:14:19.437160 22758 rpcz_store.cc:276] 0709 13:14:18.257544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.259730 (+  2186us) service_pool.cc:224] Handling call
0709 13:14:19.407258 (+1147528us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:19.427180 (+ 19922us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560871808}
I20260709 13:14:19.426617 29435 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.444247 29403 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:19.452884 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:19.460548 29467 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:19.461102 29467 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.462019 29467 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:19.439486 29132 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:14:19.471336 23459 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:19.425055 29348 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.494618 29233 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.500432 29348 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:19.500283 29433 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.475788 29135 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.450941 29435 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.519613 29435 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.542562 29433 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.543030 29434 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.545341 29434 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.546809 29467 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.615938 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:19.622406 22915 raft_consensus.cc:606] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:19.628907 22915 raft_consensus.cc:614] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:19.635293 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:19.638430 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:19.643491 29135 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.647529 29135 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:19.654075 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.655179 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.660885 29135 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.663779 29180 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.665894 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:19.667826 29221 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.670259 29249 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.673254 22836 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 13.
I20260709 13:14:19.690186 29221 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.695495 29345 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.702095 29434 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.703595 29434 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.706060 29449 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.707428 29345 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.707358 29449 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.694028 22915 raft_consensus.cc:606] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:19.702082 23465 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21.
I20260709 13:14:19.715022 29180 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.724915 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.726135 29249 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:19.727238 29467 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.730297 29127 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.731406 29434 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.731379 29127 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.735221 29251 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.735520 29221 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.736105 29449 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.736589 29283 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:19.737699 29283 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.736114 29251 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.780052 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.781232 23459 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:19.737151 29449 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.741235 29283 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.797125 22909 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:19.749840 29180 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.824648 29127 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.802362 29434 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.830502 29434 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.833794 29189 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.835062 29189 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.839282 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:19.840984 22911 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:14:19.842155 29449 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.850188 23459 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:14:19.850234 22760 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:19.853281 29189 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.853762 22765 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:19.854805 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.855142 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:19.855975 23459 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:19.858560 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:19.859791 23461 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 13: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:19.861656 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:19.862536 23457 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:19.862686 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:19.863984 22765 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:19.866344 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:19.867501 23458 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:19.850486 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:19.878288 29414 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.448s	user 0.000s	sys 0.012s
I20260709 13:14:19.881320 29434 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:19.879354 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 86533.6us
I20260709 13:14:19.907006 29251 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.927706 22758 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:19.932103 22758 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:19.940023 29481 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.941222 29481 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.942512 29449 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.944139 29481 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:19.944927 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:19.944927 29481 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:19.947697 29416 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:19.947471 29481 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.948073 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:19.948344 29416 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:19.949625 29141 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:19.950922 29141 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.955075 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:19.957089 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.962554 29141 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:19.966506 22758 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.968281 22758 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:19.940448 29321 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.018s
I20260709 13:14:19.971216 29321 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.977662 29412 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.979080 29412 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:19.981887 29458 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.816s	user 0.004s	sys 0.000s
I20260709 13:14:19.981323 29480 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:19.984732 29480 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:19.985200 29480 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:19.986533 29480 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:19.987809 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.988209 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.989346 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:19.990306 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.990337 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:19.992525 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:19.985064 29180 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:19.983870 29416 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:19.993769 22835 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:19.966583 29481 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:20.026122 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.033094 29251 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:20.034173 29251 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.036856 29251 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.033221 29321 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:20.045898 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2516) took 1263 ms. Trace:
I20260709 13:14:20.046666 22841 rpcz_store.cc:276] 0709 13:14:18.782367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.782552 (+   185us) service_pool.cc:224] Handling call
0709 13:14:18.782896 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.896465 (+113569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.896500 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:18.921256 (+ 24756us) log.cc:844] Serialized 26 byte log entry
0709 13:14:19.116206 (+194950us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:19.362818 (+246612us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.363220 (+   402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.363335 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.972700 (+609365us) spinlock_profiling.cc:243] Waited 76.2 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:19.995063 (+ 22363us) raft_consensus.cc:1685] finished
0709 13:14:20.005130 (+ 10067us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.045623 (+ 40493us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78822,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":74386,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":26752,"spinlock_wait_cycles":175330688,"thread_start_us":447323,"threads_started":3,"wal-append.queue_time_us":198721}
I20260709 13:14:20.051082 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 934) took 1168 ms. Trace:
I20260709 13:14:20.051676 22910 rpcz_store.cc:276] 0709 13:14:18.882290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.882473 (+   183us) service_pool.cc:224] Handling call
0709 13:14:18.882767 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.349042 (+466275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.015092 (+666050us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54000cfb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:20.050827 (+ 35735us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063937408}
W20260709 13:14:20.054589 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.055290 29412 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:20.056871 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51720.4us
W20260709 13:14:20.057174 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66295.4us
I20260709 13:14:20.058370 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:20.060859 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.062256 22836 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:20.062637 29315 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:20.064009 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.066360 29483 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:20.066965 29483 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.067365 29483 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:20.068545 29321 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:20.069921 29321 consensus_queue.cc:1243] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:14:20.073973 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:20.074690 29274 raft_consensus.cc:993] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:14:20.075003 29480 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:20.075024 22834 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:20.084735 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.086086 23457 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:20.078336 22837 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:20.087177 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.075315 29274 raft_consensus.cc:1081] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:14:20.101982 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2630) took 1092 ms. Trace:
I20260709 13:14:20.103005 22766 rpcz_store.cc:276] 0709 13:14:18.842140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.151666 (+309526us) service_pool.cc:224] Handling call
0709 13:14:19.152004 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.152246 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:19.152521 (+   275us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.909910 (+757389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:19.924448 (+ 14538us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:19.929190 (+  4742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.929343 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.934879 (+  5536us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":563109,"prepare.queue_time_us":119,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1403,"raft.queue_time_us":591283,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":162,"thread_start_us":163983,"threads_started":1}
I20260709 13:14:20.004638 29180 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.095103 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:20.108419 22835 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:14:20.108903 23458 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:14:20.110103 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 84494.9us
I20260709 13:14:20.112715 22765 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:20.117354 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2613) took 2004 ms. Trace:
I20260709 13:14:20.117941 22760 rpcz_store.cc:276] 0709 13:14:18.112601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.112789 (+   188us) service_pool.cc:224] Handling call
0709 13:14:18.113141 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.851219 (+1738078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.116915 (+265696us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54003408d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:20.117249 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3991815552}
I20260709 13:14:20.121266 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:20.122570 29440 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.889s	user 0.003s	sys 0.000s
I20260709 13:14:20.122730 22759 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:20.123651 29460 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.124426 29460 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.130777 29460 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:20.111824 29249 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.134368 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:20.135489 29180 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.137297 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.105196 29158 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.148250 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3185) took 1496 ms. Trace:
I20260709 13:14:20.148453 29215 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:14:20.121906 29430 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.261s	user 0.000s	sys 0.003s
I20260709 13:14:20.151162 22762 rpcz_store.cc:276] 0709 13:14:18.644732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.645971 (+  1239us) service_pool.cc:224] Handling call
0709 13:14:18.646415 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.646887 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.646932 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:18.667344 (+ 20412us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.800818 (+133474us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:18.842594 (+ 41776us) log.cc:844] Serialized 14 byte log entry
0709 13:14:18.852290 (+  9696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.852381 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.232652 (+380271us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b1800370000. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:19.703946 (+471294us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b1800370000. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f68a59c558f 00007f689ea04571 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd
0709 13:14:20.121527 (+417581us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:20.141153 (+ 19626us) raft_consensus.cc:1685] finished
0709 13:14:20.141179 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.141472 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17714,"prepare.queue_time_us":41556,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":2122,"raft.queue_time_us":13849,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":10478,"spinlock_wait_cycles":117338752,"thread_start_us":96930,"threads_started":1,"wal-append.queue_time_us":178795}
I20260709 13:14:20.165757 22757 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:20.165925 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.167292 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.167680 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.168285 22757 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:20.168736 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:20.165261 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.172135 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.176856 22833 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:20.174232 22766 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:20.170620 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.169466 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:20.186625 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:20.188817 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:20.190707 29180 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:20.196175 29460 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.197664 29229 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:20.169734 29158 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.158106 29215 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:20.134418 29249 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.208647 22835 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:20.216209 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:20.222812 29420 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.010s	sys 0.018s
I20260709 13:14:20.236155 22761 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:14:20.238547 22765 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:20.232227 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2629) took 1253 ms. Trace:
I20260709 13:14:20.230553 29460 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.242691 29433 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.245913 29180 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.230926 22762 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:20.249837 29135 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:20.258041 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:20.259862 29158 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:20.260263 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:20.260972 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.262110 22841 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:14:20.263068 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.263326 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:20.263864 22765 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:20.263994 22841 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:20.262480 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 100023us
I20260709 13:14:20.274516 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.265275 22765 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:14:20.277184 22765 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:20.277778 22765 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:20.280642 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.281905 22766 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:20.276077 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:20.284323 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:20.239910 29420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.011s	sys 0.018s
I20260709 13:14:20.287431 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.288218 22837 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
W20260709 13:14:20.292977 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 122501us
I20260709 13:14:20.305629 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.240792 22764 rpcz_store.cc:276] 0709 13:14:18.834963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.146100 (+311137us) service_pool.cc:224] Handling call
0709 13:14:19.146510 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.146762 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:19.147383 (+   621us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.929492 (+782109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:19.935914 (+  6422us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:20.016934 (+ 81020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.023035 (+  6101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.088689 (+ 65654us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":567034,"prepare.queue_time_us":582,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":595935,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":254,"thread_start_us":196410,"threads_started":1}
I20260709 13:14:20.308969 29079 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:20.305758 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:20.290719 29460 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:20.292232 29232 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:20.298087 29135 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:14:20.311051 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:20.291658 29180 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.276232 29416 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:14:20.325691 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.330650 29390 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:20.333668 22833 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:20.332191 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:20.333737 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.338668 29179 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:20.296118 29229 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.339728 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:20.316903 29433 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:20.341589 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.354077 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:20.362495 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 102436us
I20260709 13:14:20.345819 22839 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:20.342286 29274 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:20.390668 29274 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.391183 29274 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:20.361867 29135 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.367261 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.347350 29390 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.397183 29189 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:20.341931 29229 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.341495 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.365706 29144 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:20.326560 29420 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.407011 29348 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:20.341413 29312 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.342630 29460 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:20.339747 29416 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:20.408442 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.399977 22765 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.361776 29306 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:20.421679 29132 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:20.421794 22758 raft_consensus.cc:1217] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 11.8->[14.9-18.11]   Dedup: 14.9->[16.10-18.11]
I20260709 13:14:20.423053 22758 pending_rounds.cc:85] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:20.411237 29433 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:20.429756 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109781us
I20260709 13:14:20.423377 22758 pending_rounds.cc:107] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:14:20.440879 22758 raft_consensus.cc:2889] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:20.414892 29416 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:20.417390 29306 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:20.414428 29460 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:20.342077 29215 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.452670 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.426323 29144 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:20.425568 29158 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:14:20.494585 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 127569us
I20260709 13:14:20.427662 29315 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:20.518983 29315 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.516839 22765 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:20.522805 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.523022 29416 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.525002 29416 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:20.486529 29312 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.458032 29483 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.556591 29312 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.559576 29483 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:20.535118 29459 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.466s	user 0.002s	sys 0.005s
I20260709 13:14:20.503836 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.459234 29460 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:20.562126 29433 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.563349 29433 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.439616 29135 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.568116 29135 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:20.568989 29135 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:20.418502 29141 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:14:20.527313 29144 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:20.530438 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 77913us
W20260709 13:14:20.531208 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 189888us
I20260709 13:14:20.535249 29259 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:20.499809 29400 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:20.536158 29232 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:20.539659 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2518) took 1740 ms. Trace:
I20260709 13:14:20.530462 29249 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:20.542416 29233 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:20.449481 29306 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:20.616273 23461 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:20.570628 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67020.5us
I20260709 13:14:20.571319 23458 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:14:20.616317 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.624791 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.587466 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:20.583128 22841 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
W20260709 13:14:20.588074 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.628785 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.587245 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 163494us
I20260709 13:14:20.588621 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:20.637437 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50120us
I20260709 13:14:20.654986 29249 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:20.655869 29249 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.571959 29135 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.614310 22832 rpcz_store.cc:276] 0709 13:14:18.798432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.798558 (+   126us) service_pool.cc:224] Handling call
0709 13:14:18.798861 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.414022 (+615161us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4800130158. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:14:20.185943 (+771921us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b1800324dc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:14:20.193689 (+  7746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:20.193725 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:20.195228 (+  1503us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.286672 (+ 91444us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:20.286754 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.291253 (+  4499us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.538911 (+247658us) raft_consensus.cc:1685] finished
0709 13:14:20.538941 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.539129 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97865,"raft.run_cpu_time_us":2615,"raft.run_wall_time_us":523265,"spinlock_wait_cycles":30068352,"thread_start_us":184244,"threads_started":2,"wal-append.queue_time_us":228583}
W20260709 13:14:20.659293 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70800.6us
I20260709 13:14:20.664546 29449 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:20.660954 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.585459 29390 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:20.593039 29483 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:20.589108 29315 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.656280 29249 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:20.487675 29132 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.668022 29135 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:20.665362 29449 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:20.684088 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 267574us
I20260709 13:14:20.457020 29420 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.696878 29002 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:20.688366 29480 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:20.697873 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:20.699599 29409 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.700405 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.669337 29434 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:20.702020 29490 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.252s	user 0.003s	sys 0.001s
I20260709 13:14:20.703387 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:20.693255 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68581.4us
I20260709 13:14:20.597574 29274 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.704363 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:20.489157 29348 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.709685 29312 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:20.713673 29420 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:20.697969 29189 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:20.723526 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:20.733587 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:20.713712 29483 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.746644 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:20.750533 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.750566 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.766165 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.741745 29232 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:20.742317 29420 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:20.809811 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.811058 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.750339 29467 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:14:20.816742 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 119000us
I20260709 13:14:20.750715 29390 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:20.825585 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:20.816059 22766 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:20.740154 29259 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:14:20.768146 29312 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:20.722995 29002 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.832473 23458 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:20.739494 29409 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:20.806540 29420 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.754370 29434 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:20.860947 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 157646us
I20260709 13:14:20.830435 29390 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:20.867868 29147 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:20.868491 29147 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.868938 29147 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:20.870260 29002 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.873935 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.866302 29390 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:20.847798 29467 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:20.840665 29400 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.853364 29420 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:20.878123 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:20.874616 29312 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.875929 29323 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:20.916090 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:20.912569 29127 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:20.949781 29461 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.951054 29483 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.930688 29400 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:20.954717 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.954996 29497 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:14:20.957100 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.957741 22915 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:20.958063 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.981942 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:20.967068 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57357.9us
I20260709 13:14:20.930577 29141 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:20.969521 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.906050 29002 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:20.930097 29180 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:20.994104 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 51567.8us
I20260709 13:14:20.938716 29144 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:21.007951 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:21.022625 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.012090 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.024466 22806 leader_election.cc:400] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:21.024791 22806 leader_election.cc:403] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:21.023954 22837 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:21.029347 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:21.033986 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 194302us
I20260709 13:14:21.024668 22910 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:21.028678 23458 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:20.994585 29433 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:20.988884 29215 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:20.978751 29127 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.009756 29274 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.058048 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:21.058998 29484 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.014s	sys 0.023s
I20260709 13:14:21.058676 29180 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.059422 29148 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
W20260709 13:14:21.098820 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53980.5us
I20260709 13:14:21.011566 29141 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:21.066691 29496 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:21.071996 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:21.074177 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.074369 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:20.981856 29461 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.075500 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.080967 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:21.084137 29445 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:21.086097 29509 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:21.050314 29449 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.093683 29321 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.094079 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.048538 29147 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:20.994350 29460 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.028698 29144 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:21.100927 29484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.015s	sys 0.024s
I20260709 13:14:21.122357 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:21.090696 22762 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 12.
I20260709 13:14:21.128939 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:20.991792 29323 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.124044 23459 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:21.130103 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.009577 29002 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.137290 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:21.143886 23458 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:21.148339 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.152231 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.101814 29467 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.157636 23461 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:21.160645 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 148077us
I20260709 13:14:21.161090 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.097812 29215 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.165621 22908 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:21.168479 29508 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:14:21.182839 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.183802 22910 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:21.170280 29410 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:21.170069 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.150408 29409 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.174628 29348 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:21.157291 29499 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:21.181834 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.171890 29504 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:21.185917 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.192013 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.177423 29323 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.197520 29410 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.163290 29449 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:21.145118 29321 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.212759 29410 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.207685 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.179320 29460 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.235555 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.244006 29517 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.232216 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:21.196455 29434 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.244856 29517 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.249624 22911 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.252360 22913 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:21.210062 29274 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:21.256705 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:21.261957 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46607.3us
I20260709 13:14:21.263978 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.266873 22766 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:21.267601 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:21.268877 22908 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:21.256460 22760 pending_rounds.cc:85] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:21.270766 22760 pending_rounds.cc:107] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:14:21.271096 22760 raft_consensus.cc:2889] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:21.208993 29449 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:21.272888 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.278628 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:21.269299 22908 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:21.280601 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:21.249398 29409 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:21.293254 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 170450us
I20260709 13:14:21.307435 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.205964 29499 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.320143 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.320672 22913 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:21.314100 22910 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:21.272631 29274 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:21.326287 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109313us
I20260709 13:14:21.284453 29449 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:21.324052 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.274310 29147 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.329764 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.330717 22758 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:21.332544 29321 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.333822 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.334092 22908 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.335094 22908 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:21.323611 29416 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.766s	user 0.007s	sys 0.009s
I20260709 13:14:21.329437 29274 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.343591 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.343479 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.344861 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.327175 29400 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.349345 22910 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:21.353801 22761 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:21.350657 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.351650 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.350476 22914 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:21.357795 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.361953 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.358507 22764 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:14:21.348362 29306 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.727s	user 0.015s	sys 0.000s
I20260709 13:14:21.367307 22761 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:21.367520 29141 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.368244 29127 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:21.369982 29233 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.733s	user 0.014s	sys 0.002s
I20260709 13:14:21.372874 29002 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.374384 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.341408 29409 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.361827 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.382197 22911 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:21.391788 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.393090 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:21.394376 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50935.9us
I20260709 13:14:21.406544 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:21.408390 22837 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:21.408695 22836 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:21.406970 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.409981 22759 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:21.406893 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.397622 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:21.396111 23431 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.663s (SENT) [suppressed 2 similar messages]
I20260709 13:14:21.415723 22913 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:21.378163 29509 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:21.418869 22910 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:21.411090 22838 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 13.
I20260709 13:14:21.375644 29461 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.422787 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:21.418704 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56976.1us
I20260709 13:14:21.422974 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.414143 29229 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:21.424669 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.427570 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.425815 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.429095 29144 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.430084 29215 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.394557 29484 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.430822 29484 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.431218 29320 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.431768 29320 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.429154 22839 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:21.428243 22910 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:21.428066 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.439457 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.436914 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:21.374105 29400 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
W20260709 13:14:21.442797 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.449266 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.450686 29215 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:21.433394 29484 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.486158 29400 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:14:21.456938 29420 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.002s
I20260709 13:14:21.463959 29132 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:21.468652 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:21.491777 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48006.6us
I20260709 13:14:21.383286 29306 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.434520 29517 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.484824 29215 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:21.488500 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 182062us
I20260709 13:14:21.488857 22908 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:21.426404 29434 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:21.471755 29499 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:21.486857 29400 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:21.493460 22836 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
W20260709 13:14:21.501987 22915 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.003s
I20260709 13:14:21.502553 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:21.514297 22915 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
W20260709 13:14:21.502907 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.515575 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.516073 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 137959us
W20260709 13:14:21.504128 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76154.2us
I20260709 13:14:21.494566 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.419184 29409 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.511027 22807 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.505156 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.522428 22911 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:21.491848 29420 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.522814 29400 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.524400 29215 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.524952 22907 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:21.525177 22912 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:21.526610 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.524257 29400 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.528436 22912 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:21.524854 29420 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:21.519019 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.529918 22906 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:21.515895 29523 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:14:21.531533 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.532083 29523 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:21.532665 29523 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:21.533171 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:21.533051 29523 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:21.515003 22910 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:21.532997 29434 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:21.543632 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.544282 22837 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:21.512151 29321 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:21.542897 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:21.586786 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72231.6us
W20260709 13:14:21.535426 29135 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.800s	user 0.004s	sys 0.012s
I20260709 13:14:21.537227 29517 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:21.579679 22910 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:21.536888 29412 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:21.533352 29148 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:21.587714 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.579134 22766 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:21.591949 22912 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:21.590687 29483 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:21.598173 29483 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:21.598644 29483 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:21.593242 29148 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:21.600601 29148 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:21.604786 29345 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:21.605358 29345 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:21.605891 29345 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:21.606212 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.606426 29229 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:21.607530 29484 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:21.607925 29180 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.608109 29484 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.608502 29484 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:21.607415 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.609877 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.612119 29336 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:21.613037 29336 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:21.615094 29306 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:21.615196 29336 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.613071 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:21.618754 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:21.619232 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.620184 29522 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:21.620637 29522 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.620769 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.621039 29522 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:21.624498 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:21.621767 29147 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:21.625741 29147 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.626129 29147 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:21.619714 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:21.622330 29514 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:21.622535 29002 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:21.625383 22764 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19.
I20260709 13:14:21.594067 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.333652 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:21.633599 22910 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:21.581151 29499 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:21.592944 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66437.5us
I20260709 13:14:21.651598 29510 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:21.635919 22911 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:21.639696 29521 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:21.657480 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.657790 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:21.659039 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:21.659351 29215 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:21.659855 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:21.660198 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.630330 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:21.637374 29002 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.667500 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.668241 29002 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.587329 29449 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.661567 22760 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:21.669410 29510 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:21.670598 29449 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:21.591002 29249 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.871s	user 0.011s	sys 0.007s
I20260709 13:14:21.606973 29229 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:21.670622 29510 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.671394 29249 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.651461 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:21.674242 29518 raft_consensus.cc:993] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: : Instructing follower acfda9b350734def9b6c621f66a5f247 to start an election
I20260709 13:14:21.676994 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.678387 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:21.680182 29461 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:21.681346 29002 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:21.682520 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.660336 29526 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:21.661437 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.641887 29412 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:21.648727 29499 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.632758 29320 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.586395 29321 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:21.709136 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:21.681071 29461 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:21.649820 29306 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:21.710163 22833 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:21.702136 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.709462 29321 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:21.723938 29412 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:21.635565 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 116610us
I20260709 13:14:21.726446 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.613245 29274 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.591617 29135 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.733556 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:21.713124 29461 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.713886 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.714907 29524 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:14:21.715821 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.715968 29249 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:21.716482 22805 leader_election.cc:400] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16.
I20260709 13:14:21.717608 29002 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:21.718479 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.719380 22837 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:21.719723 29509 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:21.754403 29532 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:21.755053 29532 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.757774 29416 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:21.758378 22807 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.693116 29526 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:21.703790 29499 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:21.792905 22915 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:14:21.740976 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 76502.3us
I20260709 13:14:21.749259 29409 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:21.749747 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 88393.3us
I20260709 13:14:21.737807 29449 raft_consensus.cc:1081] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Signalling peer acfda9b350734def9b6c621f66a5f247 to start an election
W20260709 13:14:21.758739 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 128500us
W20260709 13:14:21.802495 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.812386 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:21.758337 29416 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.819491 22805 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:21.821096 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.748204 29514 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:21.793143 29518 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:21.765738 29532 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.813740 29509 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.832181 29079 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:21.847090 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 133309us
I20260709 13:14:21.832393 29180 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:21.830225 29416 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:21.849591 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 91304.8us
I20260709 13:14:21.851910 22807 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:21.826730 29409 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:21.843116 29336 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:21.861567 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.863637 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.865463 29433 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:21.865896 29433 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:21.866290 29433 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:21.848173 29532 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:21.863916 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.874737 29514 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.876850 29274 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:21.885540 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:21.891234 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.857934 29449 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:21.902912 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:21.869589 29215 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:21.855616 29536 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:14:21.881110 29390 consensus_meta.cc:287] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.940s	user 0.014s	sys 0.001s
W20260709 13:14:21.905133 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68340.6us
I20260709 13:14:21.906351 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.873958 29079 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.868842 29180 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:21.907841 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.860759 29409 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.919227 29545 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:21.911105 29147 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:21.870491 29532 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:21.928511 29274 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:21.928313 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48470us
I20260709 13:14:21.920910 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:21.937557 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73667.2us
I20260709 13:14:21.911790 29148 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.998713 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:21.922988 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:21.984081 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 180869us
I20260709 13:14:21.986388 22837 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:21.873324 29336 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.990252 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:21.928073 29179 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:21.928308 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:21.993816 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:21.996037 29409 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:21.997052 22730 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21.
I20260709 13:14:21.926486 29510 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:21.998944 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:21.906224 29215 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:21.989897 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:22.003760 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 80688.3us
I20260709 13:14:21.932104 29536 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:22.025331 29536 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.026376 29510 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.997733 29135 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.029173 29510 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.029656 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:22.031145 22763 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:14:22.031983 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.032582 29215 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.033829 22758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7"
dest_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46950
I20260709 13:14:22.034092 29215 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:22.034567 22758 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:22.034947 22758 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:22.036718 29510 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:22.037122 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.037364 29510 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:22.037985 29510 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.040822 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.044862 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.045830 29148 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.045961 29002 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:22.046412 29002 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:22.046846 29002 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:22.049571 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.050285 22760 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:22.051141 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:21.998306 29532 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.929900 29390 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.055019 29147 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.058560 29135 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:22.058804 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.002184 29484 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.061471 29484 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.062194 29484 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:14:22.062312 29533 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:14:22.062628 29484 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:22.022395 22834 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:21.997304 29345 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:21.872475 29526 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.012528 23458 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:22.019450 22911 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:22.008610 22835 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:14:22.007587 29132 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:22.075151 22839 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:22.067889 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.071125 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47858.2us
I20260709 13:14:22.077908 29336 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:22.080775 29551 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:22.084085 29483 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:22.013257 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 218752us
W20260709 13:14:22.074966 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 78152.9us
W20260709 13:14:22.065680 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.063093 29484 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:22.084841 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:22.045977 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.093186 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:22.096596 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50721.2us
I20260709 13:14:22.098380 22759 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:22.098950 22914 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:22.097564 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.100566 22758 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.064230 29180 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.104702 29501 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:22.118701 29501 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.119092 29501 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:22.120134 29345 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:22.128834 29390 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.073740 29148 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:14:22.129626 22760 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:14:22.130105 22760 pending_rounds.cc:85] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 13
I20260709 13:14:22.130407 22760 pending_rounds.cc:107] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:14:22.130630 22760 raft_consensus.cc:2889] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:22.130940 29523 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.000s
I20260709 13:14:22.131598 29523 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.132229 29483 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.133165 22765 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:22.133780 22833 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:22.104701 29409 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:22.139942 29409 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:22.141084 29409 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.144672 29079 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:22.146337 22757 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:22.146772 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3445) took 1240 ms. Trace:
I20260709 13:14:22.130009 29148 consensus_queue.cc:1243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:14:22.148439 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1176) took 1192 ms. Trace:
I20260709 13:14:22.104907 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.149632 23463 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:22.104131 22880 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19.
I20260709 13:14:22.084924 29321 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.148855 23461 rpcz_store.cc:276] 0709 13:14:20.956575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.956744 (+   169us) service_pool.cc:224] Handling call
0709 13:14:22.148183 (+1191439us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002c6d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:22.148343 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394770304}
I20260709 13:14:22.150204 22880 leader_election.cc:403] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:14:22.149807 22839 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:22.147183 22907 rpcz_store.cc:276] 0709 13:14:20.906139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.953551 (+ 47412us) service_pool.cc:224] Handling call
0709 13:14:21.524611 (+571060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.525483 (+   872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.146536 (+621053us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b5400211bd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:22.146705 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1290064256}
I20260709 13:14:22.164467 22758 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:22.165190 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 63776.1us
I20260709 13:14:22.147817 29229 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:22.104754 29445 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.023s	sys 0.012s
I20260709 13:14:22.147699 29002 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.102550 29336 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:22.103173 29521 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:22.170174 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.170406 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.174307 22758 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:22.171810 22909 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:22.177359 29002 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.173177 29336 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:22.178910 29336 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:22.183660 29522 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.560s	user 0.000s	sys 0.012s
I20260709 13:14:22.184360 29522 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.165934 29127 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:14:22.185997 23462 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:22.186679 29127 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:22.187191 29127 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:22.188498 29127 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.190122 29127 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:22.191429 29336 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.188907 22841 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:22.171021 29445 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.024s	sys 0.012s
I20260709 13:14:22.141219 29510 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.168430 29320 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19.
W20260709 13:14:22.195428 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 193466us
I20260709 13:14:22.209224 23459 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:22.210342 22760 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:22.214846 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:22.220430 23463 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:14:22.222779 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.188509 29229 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.224248 22907 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:22.232077 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.233434 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:22.234872 29522 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.235141 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.236037 22757 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:14:22.234462 29499 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.237864 22912 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 24: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:22.168542 29483 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:14:22.241456 29499 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.242162 29002 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:22.242604 29002 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:22.243501 29002 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.255044 29336 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.264016 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3461) took 1182 ms. Trace:
I20260709 13:14:22.264746 29509 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:22.266893 29321 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:22.269155 29523 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.275035 29510 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.277240 29416 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.277827 29554 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:22.278581 29554 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:22.279049 29554 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:22.279129 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49142.1us
I20260709 13:14:22.279996 29416 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:22.193331 29433 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.281950 22908 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:22.283094 29433 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.283833 23465 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:22.286578 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.289222 23463 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 24: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:22.290287 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.286692 22765 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:22.291458 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62318.6us
W20260709 13:14:22.288326 29548 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.230s	user 0.003s	sys 0.001s
I20260709 13:14:22.296501 29531 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:22.298615 29519 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:22.299041 29540 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.502s	user 0.002s	sys 0.000s
I20260709 13:14:22.299261 29519 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.301226 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.308374 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:22.313851 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 158032us
I20260709 13:14:22.316177 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.307051 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2562) took 1239 ms. Trace:
I20260709 13:14:22.320416 22832 rpcz_store.cc:276] 0709 13:14:21.068044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.072647 (+  4603us) service_pool.cc:224] Handling call
0709 13:14:21.072994 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.142033 (+ 69039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.142075 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:21.150036 (+  7961us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:21.150082 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.306730 (+1156648us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:22.306799 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.306966 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146140928}
I20260709 13:14:22.315260 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.326961 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.264672 22915 rpcz_store.cc:276] 0709 13:14:21.082283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.516020 (+433737us) service_pool.cc:224] Handling call
0709 13:14:21.516357 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.793581 (+277224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.263532 (+469951us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54003ca0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:22.263889 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635334528}
I20260709 13:14:22.330307 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:22.338748 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:22.297238 29531 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:22.341812 22910 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:22.337296 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.342288 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:22.353688 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.354588 22837 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:22.359843 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.381860 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.382241 29555 consensus_queue.cc:1059] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:14:22.384392 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.384974 29526 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:22.385779 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.385641 29547 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:14:22.386586 23463 raft_consensus.cc:2368] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:22.387391 29515 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:14:22.388608 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48378.1us
I20260709 13:14:22.389816 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:22.336117 29336 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:22.401140 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:22.405032 23461 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:22.405866 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.406769 22757 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.406906 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.396724 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:22.414489 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:22.400621 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:22.419373 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43614.4us
I20260709 13:14:22.427228 29522 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:22.429194 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:22.436043 22839 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:22.445581 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.342689 29144 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:14:22.408177 29336 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:22.448843 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:22.395251 29547 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:22.463490 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.429832 29517 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:22.400390 29445 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:14:22.472019 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 84409.9us
I20260709 13:14:22.433595 29345 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:22.432845 29522 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:22.394829 29412 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:14:22.385252 29532 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.473109 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:22.414009 29509 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:14:22.404991 29323 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:14:22.479228 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:22.491654 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.471565 29517 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.494925 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.497288 22839 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:22.501752 29569 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:22.453297 29523 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:14:22.503872 29287 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:22.489806 29519 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.519783 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:22.522058 22911 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:22.499419 29517 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:22.473032 29547 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.540910 22839 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:22.543404 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.545169 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.548550 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.549281 22761 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:22.550515 22761 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:22.551532 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.552886 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.560297 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:22.560976 29546 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.542s	user 0.002s	sys 0.000s
I20260709 13:14:22.561311 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.562604 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.563119 22762 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:22.566464 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.567433 22765 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:22.567556 29410 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:22.567945 29410 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:22.568362 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:22.568775 29557 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:14:22.570433 29556 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.141s
W20260709 13:14:22.572134 29564 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:14:22.573946 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.538277 29484 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.578783 22766 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:22.579196 22766 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:22.580152 29484 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.579953 22766 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.568917 29410 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.581053 22766 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.582042 22915 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:22.585945 29537 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.787s	user 0.014s	sys 0.006s
I20260709 13:14:22.585978 29522 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.593425 22764 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:22.594089 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.595270 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.596110 22758 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:14:22.594630 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:22.598412 22765 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:22.511045 29336 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.605403 23457 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:22.627912 23457 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:22.628180 22766 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:14:22.628597 22766 pending_rounds.cc:85] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 10
I20260709 13:14:22.628806 23457 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.608009 29531 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.611402 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3463) took 1527 ms. Trace:
I20260709 13:14:22.634428 22909 rpcz_store.cc:276] 0709 13:14:21.084644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.526832 (+442188us) service_pool.cc:224] Handling call
0709 13:14:21.676214 (+149382us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:14:21.676487 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.183106 (+506619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.611091 (+427985us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b540010e5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:22.611331 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156196992}
I20260709 13:14:22.613144 29461 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:22.636441 29461 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:22.521369 29559 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:22.637844 29559 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.638315 29559 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:22.637924 29461 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.640043 29461 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:22.644891 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.645743 29531 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:22.650266 29553 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:22.650938 29553 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.653465 29553 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:22.657063 29553 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.658216 29536 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:14:22.658818 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.657689 29553 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.660423 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:22.619056 29514 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:22.662959 29514 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:22.663719 29553 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.663893 29514 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.620325 29002 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:22.662380 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:22.667464 29306 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:22.628832 22766 pending_rounds.cc:107] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:14:22.703192 22910 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:22.703137 22766 raft_consensus.cc:2889] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:22.691215 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:14:22.672935 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 331142us
I20260709 13:14:22.685062 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:22.630210 23457 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:22.712021 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:22.665369 29501 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.000s
W20260709 13:14:22.609696 29560 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:14:22.732724 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:22.733841 22906 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:22.729521 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:22.732136 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:22.736461 22915 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
W20260709 13:14:22.662446 29562 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:14:22.732676 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:22.747973 22805 leader_election.cc:400] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:22.748368 22805 leader_election.cc:403] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:14:22.748811 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63843.8us
I20260709 13:14:22.749958 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:22.727079 29449 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:22.747473 29523 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:22.627385 29420 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.128s
W20260709 13:14:22.756273 29127 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.000s
I20260709 13:14:22.618958 29532 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:22.761569 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:22.779757 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.665480 29514 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:22.781606 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49006us
I20260709 13:14:22.786949 22761 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:22.788965 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.789983 22841 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 24: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:22.790263 22839 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:22.787508 22761 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:22.796909 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47081.2us
I20260709 13:14:22.799562 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:22.800199 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.800908 22841 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:22.750502 29215 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:22.803732 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.627198 29144 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.147s
W20260709 13:14:22.811906 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 459823us
I20260709 13:14:22.812405 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.813999 22839 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:22.830061 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.822479 23465 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:22.830687 23461 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:22.766242 29523 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:22.798102 29416 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:22.829784 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:22.802557 29306 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:22.852818 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:22.746829 29501 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.847646 22761 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.858476 29522 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:22.870668 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:22.871840 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:22.873888 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.877595 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.877924 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:22.879485 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:22.881014 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:22.883235 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.796960 29412 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:14:22.818346 29519 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:14:22.886893 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.890384 22834 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:22.893976 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.894766 23458 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:22.840615 29259 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:14:22.899509 29306 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:22.900658 29306 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.901765 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:22.833002 29345 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:22.905040 29345 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.905607 29345 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:22.907682 23466 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:22.831389 29215 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.909785 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.910544 29215 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.821259 29449 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:14:22.846841 29079 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
I20260709 13:14:22.835420 29536 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:14:22.914696 29536 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:22.915166 29536 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:22.916317 29536 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:22.903367 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:22.918318 23465 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:22.918708 22912 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:22.919857 29566 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:14:22.920027 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48153.1us
I20260709 13:14:22.920305 29566 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:22.921061 29180 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:22.924603 29547 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:22.856598 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:22.863608 22761 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:22.865952 29554 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.006s
I20260709 13:14:22.867684 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:22.932849 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 76373.7us
I20260709 13:14:22.877970 23457 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:22.958282 23458 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:22.967329 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.969915 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:22.970948 23466 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:22.969321 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:22.928056 29542 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.018s	sys 0.035s
I20260709 13:14:22.978391 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:22.994128 22834 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:22.995426 23461 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:22.996305 23457 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:22.930903 29410 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:22.988047 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 205450us
I20260709 13:14:22.977648 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:22.976365 22840 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:22.993268 22762 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:22.986143 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:22.993032 29336 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.837281 29553 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 13:14:23.003230 23463 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:23.002810 22912 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:23.002457 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 180609us
I20260709 13:14:22.993739 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:23.013959 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:23.011317 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:23.019368 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:23.021138 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.976147 29542 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.018s	sys 0.035s
I20260709 13:14:22.957990 29556 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:23.042671 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:22.948979 29180 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:14:22.949862 29554 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:22.977560 29515 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.058840 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:23.064157 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.957633 29400 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:14:22.921290 29566 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.074224 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:23.075918 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56676.2us
W20260709 13:14:23.081238 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.090653 22832 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:23.024320 29409 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:23.094259 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.096331 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 85139.6us
I20260709 13:14:23.119237 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:23.120090 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45980.7us
I20260709 13:14:23.089859 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:23.134054 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.135716 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:23.072537 29135 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.077608 29274 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:23.073767 29127 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:23.174511 29127 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:23.175915 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47400us
I20260709 13:14:23.021297 29336 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.137423 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.147219 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:23.151363 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:23.169736 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 157920us
I20260709 13:14:23.077780 29483 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:23.088176 29484 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:23.182968 29523 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:23.186170 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.185833 29135 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:23.182802 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.188932 29523 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:23.191789 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.182448 29336 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:23.125146 29259 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:23.125844 29306 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:23.196162 22832 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:23.198787 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71791.8us
W20260709 13:14:23.202171 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50950.8us
W20260709 13:14:23.205681 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.208485 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.211753 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.090512 29132 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
W20260709 13:14:23.219321 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 100202us
I20260709 13:14:23.147457 29229 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:23.220284 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.085325 29515 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.123315 29519 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:14:23.230389 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.190918 29484 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:23.177460 29127 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.172300 29547 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:23.183727 29555 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:23.195927 29518 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:23.211088 29259 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.210619 29526 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:23.180097 29509 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.224037 29511 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:14:23.218964 29532 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:23.250340 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.254722 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.223680 29229 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:23.201215 29483 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:23.262243 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.266567 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.273147 22763 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.001s
I20260709 13:14:23.274247 22763 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:23.269364 29509 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.270457 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.271066 29229 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:23.271263 29574 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:14:23.291198 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:23.243289 29555 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:23.272063 29518 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:23.255474 29259 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:23.298458 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64554.9us
I20260709 13:14:23.304592 29515 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:23.304838 29576 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:14:23.271785 29135 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:23.309599 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.312635 22764 raft_consensus.cc:2368] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
W20260709 13:14:23.313697 29575 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.113s	user 0.000s	sys 0.002s
W20260709 13:14:23.316305 29577 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:14:23.318847 29483 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.319329 22841 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:23.327972 29559 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.000s
I20260709 13:14:23.328517 29559 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.331936 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.332839 29509 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.333561 29509 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.335446 29554 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.337240 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.338018 22766 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:23.339526 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.340308 22913 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:23.310456 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:23.343104 22762 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
W20260709 13:14:23.343631 29583 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:14:23.344745 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.345950 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:23.348421 29559 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.318922 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:23.337549 29531 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.003s
I20260709 13:14:23.249605 29127 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:14:23.302580 29555 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:23.364740 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 147230us
I20260709 13:14:23.334271 29509 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.368332 22761 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:23.396781 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:23.357149 22834 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:23.370487 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.372053 29127 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:23.372900 22837 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:23.380017 29483 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.380404 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:23.383044 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.427068 22757 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:23.383527 29345 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.384940 22765 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:23.393181 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.357684 22766 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
W20260709 13:14:23.412992 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 205286us
I20260709 13:14:23.401197 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:23.310340 29578 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.438982 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:23.438957 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.359364 29554 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.368227 29553 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:23.429368 29573 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
I20260709 13:14:23.445648 22762 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:23.447059 29390 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:23.447576 29524 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.461032 23466 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:23.461135 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:23.465968 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:23.467272 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.470418 22757 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:23.467176 22806 leader_election.cc:400] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:23.425459 29127 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:23.479753 22839 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:23.481101 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:23.481668 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.482865 29412 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:23.483435 29412 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:23.484064 29514 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.687s	user 0.013s	sys 0.000s
I20260709 13:14:23.485677 29588 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:14:23.438076 29483 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.484512 29412 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.489238 29412 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:23.342531 29518 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.496760 29141 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:23.497375 29141 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:23.498329 29141 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.499524 29141 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:23.499926 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.473099 29526 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:23.501932 29524 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.502452 29320 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:23.503074 29320 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:23.504027 29320 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.504345 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:23.502446 29557 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:14:23.504449 29345 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:23.477104 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:23.508138 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.508508 22759 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:23.508528 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.535137 22837 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:23.523451 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.516848 22765 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: [20.9-20.9]
W20260709 13:14:23.547631 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78770.8us
W20260709 13:14:23.548385 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 87336.9us
I20260709 13:14:23.468003 29553 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.467141 29554 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.488334 29483 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.551342 22836 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:23.502007 29306 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:23.507429 29127 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.477365 29390 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.554754 29312 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:23.553596 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 375319us
I20260709 13:14:23.505470 29320 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:23.565147 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:23.551049 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:23.557400 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.568331 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.559609 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.569130 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:23.569679 22836 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:23.554258 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:23.559898 29306 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:23.570596 29594 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:23.571789 29594 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:23.571971 29132 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:23.572167 29594 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:23.573014 29483 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:23.459805 29578 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.573163 29594 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.583006 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.556255 29553 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:23.586882 22837 raft_consensus.cc:2376] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16. Re-sending same reply.
I20260709 13:14:23.591456 29589 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:23.560039 29593 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:23.563591 29445 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:14:23.585215 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 106085us
I20260709 13:14:23.562085 29312 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:23.604707 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.607467 22841 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:23.607775 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.609164 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.608525 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:23.610143 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.587177 29345 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.608491 22837 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:23.604943 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:23.589776 29526 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:23.619227 22841 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:14:23.613336 29345 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.621300 29598 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:14:23.617839 29002 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.952s	user 0.010s	sys 0.008s
I20260709 13:14:23.629247 29591 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:23.617911 29559 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:14:23.621143 29509 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.633539 29509 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:23.634119 29509 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.636524 29509 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:23.623413 29554 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:23.638386 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.639861 22763 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:23.640101 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.642107 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.629698 29595 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.644876 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.646711 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.648470 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:23.648834 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.697927 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:23.699463 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.699854 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:23.702100 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.649333 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:23.650616 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.693225 29559 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:23.706851 22914 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:23.707238 29559 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.675626 22841 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:23.603654 29589 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:23.683584 22835 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:23.604472 29445 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:23.583238 29409 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:14:23.720314 29229 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.720649 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:23.650767 22911 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:14:23.721695 29554 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:23.722203 29554 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:23.614518 29410 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.010s
I20260709 13:14:23.723106 29554 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.685079 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.724521 22912 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:23.685372 29312 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.725864 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.726985 22914 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:23.571789 29306 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.687093 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:23.728283 23459 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:23.728453 22839 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:23.725955 22832 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:23.688645 29571 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:23.573227 29523 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.731033 29523 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:23.689591 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.732429 29592 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:23.733530 22913 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:23.690490 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:23.690728 29578 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:23.690088 22664 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 7 to 9, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.694681 22836 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:23.685139 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.737612 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.738294 29345 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:23.738323 22836 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:23.694339 29515 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:14:23.740840 22836 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:14:23.710155 29517 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.041s	user 0.018s	sys 0.002s
I20260709 13:14:23.741478 29229 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.711176 29509 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:23.717437 29002 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:23.744712 22907 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:23.745581 29002 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:23.712414 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.746762 22908 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:23.710455 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.746950 29002 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.715219 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.748483 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:23.742509 29517 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.749935 23466 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:23.752944 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.753824 22906 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:23.754815 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:23.711014 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99342.4us
I20260709 13:14:23.716758 22910 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:23.759716 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:23.617709 29542 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:23.680735 29584 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:14:23.719722 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.616263 29526 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:23.779732 22835 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:23.733983 29135 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.745738 29536 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:23.749164 29501 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:23.771682 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.772395 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.790422 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.774648 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:23.701243 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:23.791575 22841 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:23.759577 29461 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.118s	user 0.020s	sys 0.005s
W20260709 13:14:23.792055 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 90892.5us
I20260709 13:14:23.791083 22908 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:23.796300 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.793967 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:23.800269 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:23.716478 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 67302.5us
I20260709 13:14:23.780934 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.783664 22913 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:14:23.804463 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:23.783736 22911 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:23.697738 29524 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:23.782905 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.764955 29449 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:23.810911 22912 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:23.811719 22908 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 13:14:23.765998 29517 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:23.782908 29526 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:23.788878 29135 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.777482 29542 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:23.821765 23431 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.825489 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.710805 29259 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.711578 29595 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.850543 22841 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:23.852664 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:23.852646 22841 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:23.826992 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:23.806973 29593 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:23.866247 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:23.867863 22761 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:23.817454 29483 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:23.828524 29461 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:23.830451 29179 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:23.883600 29517 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:14:23.884589 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59210.3us
W20260709 13:14:23.884948 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52041.4us
I20260709 13:14:23.885926 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.888441 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:23.886605 23457 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 21. Ops: []
I20260709 13:14:23.886782 23463 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:23.796571 29400 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:14:23.840757 29511 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:23.827658 29547 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:23.894760 22835 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:23.896845 22914 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:23.822690 29524 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:14:23.821977 29215 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:14:23.878096 29483 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.865772 29593 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.907518 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:23.891381 29606 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:14:23.893414 29589 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:23.909771 29598 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:23.910745 29595 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.905702 29584 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:23.915179 29524 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:23.857880 29592 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:14:23.894281 29180 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:23.986886 29180 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:23.929687 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.991679 29572 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:23.992885 29135 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:23.943692 22879 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:23.959532 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71159.1us
W20260709 13:14:23.994946 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.950858 29483 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:23.976243 29595 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.972677 29517 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:23.992512 29572 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:23.953532 22911 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:23.993716 29608 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.000231 29135 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:24.013850 29608 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.014179 29511 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:23.902122 29531 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.015205 29531 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.016356 29608 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.017275 29531 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.967140 29559 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.018211 29559 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:24.019035 29132 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.019761 29132 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:23.996589 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53028us
I20260709 13:14:23.973809 29524 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:24.023566 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:24.024176 29420 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:24.024783 29420 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.027581 29420 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:23.973409 29606 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:23.904524 29547 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.000804 29595 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.030686 29595 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:23.954967 29593 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:24.003646 29517 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.013243 29572 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.014335 29135 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:24.023568 29524 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.028855 29445 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.030750 29547 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.010249 29180 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.034916 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:24.044461 22909 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
W20260709 13:14:24.052465 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.091250 22840 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 21 because replica has last-logged OpId of term: 20 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 8.
W20260709 13:14:24.080731 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.046712 29555 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.008s
W20260709 13:14:24.124169 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 113983us
W20260709 13:14:24.058780 29141 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.559s	user 0.015s	sys 0.010s
I20260709 13:14:24.129278 22838 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:24.131178 29524 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:24.133641 29595 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:24.055359 29135 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:24.110816 29412 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.001s
I20260709 13:14:24.144357 29509 raft_consensus.cc:2764] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:14:24.144766 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:24.153182 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:24.154754 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:24.155992 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:24.148947 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:24.138991 29141 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.159775 29141 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:24.180419 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 324302us
I20260709 13:14:24.136464 29555 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.267099 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:24.174161 29592 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.178711 29259 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.191661 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:24.193252 29483 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.193151 29306 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.183327 29571 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:24.205245 22764 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:24.218138 23465 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:24.230291 29127 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:24.233798 29132 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.235040 29542 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.237181 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:24.248128 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:24.250752 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:24.363519 22913 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
W20260709 13:14:24.265096 29609 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
W20260709 13:14:24.364107 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 113497us
I20260709 13:14:24.255075 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:24.305833 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:24.260810 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2140) took 1136 ms. Trace:
W20260709 13:14:24.318015 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.328562 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61575.7us
I20260709 13:14:24.367801 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:24.335135 22834 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:24.369532 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:24.366891 22840 rpcz_store.cc:276] 0709 13:14:23.124192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.202747 (+ 78555us) service_pool.cc:224] Handling call
0709 13:14:24.260452 (+1057705us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:24.260677 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976713984}
I20260709 13:14:24.370565 22834 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:14:24.276811 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:24.372879 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:24.346026 29554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.003s	sys 0.006s
I20260709 13:14:24.253682 29412 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.349439 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:24.168120 29445 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:24.350487 22833 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:24.388075 22761 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:14:24.383827 22835 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:14:24.352468 29526 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.511s	user 0.014s	sys 0.000s
I20260709 13:14:24.347209 29509 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:24.350504 29433 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.346836 29592 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.355522 29483 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.257758 29002 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.358422 29555 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:24.374717 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119668us
I20260709 13:14:24.415802 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:24.380739 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:24.382697 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3250) took 1302 ms. Trace:
I20260709 13:14:24.384505 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:24.390421 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 201413us
I20260709 13:14:24.420876 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:24.394474 22759 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:14:24.400956 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:24.399986 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:24.388935 29554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.003s	sys 0.009s
I20260709 13:14:24.381109 29571 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.387598 29542 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.434501 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:24.435530 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:24.428051 29616 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:24.427786 29607 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:24.442230 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:24.387101 29132 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:24.386999 29559 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:24.387254 29547 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:24.418617 22763 rpcz_store.cc:276] 0709 13:14:23.080649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.492502 (+411853us) service_pool.cc:224] Handling call
0709 13:14:23.492879 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:23.640287 (+147408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.382382 (+742095us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54002224d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:24.382626 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319057792}
I20260709 13:14:24.437740 29608 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:24.399708 29572 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:24.392104 29596 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:24.406759 29613 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:24.401180 29526 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.430693 29002 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.381034 29412 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.433866 29499 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:24.407074 29433 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.439786 29571 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.443524 29509 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:24.444324 29132 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:24.444077 29553 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.839s	user 0.015s	sys 0.000s
I20260709 13:14:24.446069 29547 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.440109 29581 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:24.448472 29483 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:24.429078 29449 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:14:24.466251 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.475205 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2142) took 1275 ms. Trace:
I20260709 13:14:24.520162 22832 rpcz_store.cc:276] 0709 13:14:23.199952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.232494 (+ 32542us) service_pool.cc:224] Handling call
0709 13:14:24.474939 (+1242445us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:24.475127 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072568064}
I20260709 13:14:24.522209 29613 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:24.523634 29569 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.523337 29613 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.524236 29569 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.527014 29524 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.528388 29524 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.475237 29499 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.477855 29433 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.478281 29509 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:24.532644 29509 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:24.533270 29524 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.478559 29132 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:24.478688 29571 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.479351 29553 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.543929 23458 raft_consensus.cc:2368] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18.
I20260709 13:14:24.484193 29566 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:24.484673 22841 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:24.486600 22836 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 22. Ops: []
I20260709 13:14:24.455719 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:24.556670 23464 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:24.557662 29524 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.545413 29553 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.482949 29581 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.558689 29524 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.561378 29524 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.472784 29002 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.561961 29581 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.473320 22761 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:14:24.484640 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2156) took 1051 ms. Trace:
I20260709 13:14:24.482584 29547 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:24.563284 22839 rpcz_store.cc:276] 0709 13:14:23.433365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.480785 (+ 47420us) service_pool.cc:224] Handling call
0709 13:14:24.484449 (+1003664us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b540028aad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:24.484572 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567165184}
I20260709 13:14:24.565452 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:24.567088 22839 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:24.483072 29141 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.569983 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:24.483968 29483 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:14:24.456734 29559 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:24.474830 29616 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:24.551478 29569 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.586452 29547 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.588762 23464 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:24.572670 22839 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:14:24.598384 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:24.606086 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:24.454880 29518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.011s	sys 0.035s
I20260709 13:14:24.617694 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:24.618440 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.631402 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:24.475368 29412 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.470014 29526 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.520519 29606 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:24.520781 29592 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:24.666874 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:24.703393 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:24.707670 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:24.604980 29547 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:24.678740 22839 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:24.720193 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:24.621694 29571 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.730100 22762 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
W20260709 13:14:24.637401 29518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.015s	sys 0.039s
I20260709 13:14:24.703393 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:24.741041 22759 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:24.683753 29569 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:24.734622 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:24.691097 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 236998us
I20260709 13:14:24.676404 29141 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.655586 29002 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:14:24.757001 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.757239 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:24.780735 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:24.456230 29596 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:24.784721 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 384841us
I20260709 13:14:24.795151 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:24.752329 29586 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:24.713351 29606 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.850147 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1196) took 1148 ms. Trace:
I20260709 13:14:24.858944 23458 rpcz_store.cc:276] 0709 13:14:23.702092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.702258 (+   166us) service_pool.cc:224] Handling call
0709 13:14:24.849903 (+1147645us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:24.850079 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891237248}
I20260709 13:14:24.798323 29559 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:24.877630 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:24.741281 29571 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.793148 29002 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:24.890561 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:24.891788 23460 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:24.895036 22805 leader_election.cc:400] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18.
I20260709 13:14:24.895583 22805 leader_election.cc:403] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:24.778254 29141 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.769243 29569 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:24.918205 29621 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:14:24.919973 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3486) took 1164 ms. Trace:
I20260709 13:14:24.920432 22906 rpcz_store.cc:276] 0709 13:14:23.755998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.760942 (+  4944us) service_pool.cc:224] Handling call
0709 13:14:23.780778 (+ 19836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:23.791836 (+ 11058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:23.791870 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:23.815360 (+ 23490us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.196220 (+380860us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b18002eae00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:14:24.346019 (+149799us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:24.346080 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.346173 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:24.919556 (+573383us) raft_consensus.cc:1685] finished
0709 13:14:24.919584 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:24.919881 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7107,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4153,"spinlock_wait_cycles":3788544,"thread_start_us":149505,"threads_started":1,"wal-append.queue_time_us":628750}
W20260709 13:14:24.924324 29622 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:14:24.876993 29627 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:24.900332 29518 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18.
I20260709 13:14:24.881026 29616 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:14:24.957055 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:24.941318 29445 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.000s
I20260709 13:14:24.954437 29180 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.959952 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
W20260709 13:14:24.958885 29536 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.163s	user 0.017s	sys 0.000s
W20260709 13:14:25.008618 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 213364us
I20260709 13:14:25.010531 23466 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:25.013716 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:24.987380 29180 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:24.946272 29002 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.014012 29517 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.034248 22839 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:25.035369 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:25.037125 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:24.956089 29627 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:14:25.022812 29624 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:14:25.032449 29593 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.001s	user 0.018s	sys 0.000s
W20260709 13:14:25.030920 29632 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:14:25.058950 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.048761 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:25.056219 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 261134us
I20260709 13:14:25.032191 29002 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:25.045091 29306 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.689s	user 0.002s	sys 0.014s
W20260709 13:14:25.041492 29509 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.508s	user 0.016s	sys 0.002s
I20260709 13:14:25.047719 29559 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.055375 29624 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:25.062371 29586 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:25.066066 29518 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:25.084097 22765 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:25.065194 29593 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:24.738564 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:25.073457 29637 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:25.067687 29571 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.113826 29637 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:25.114348 29637 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.103744 29518 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.114789 29637 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:25.040376 29517 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.118264 29586 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.118682 29569 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.067757 29509 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.122620 29509 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.123014 29593 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.123545 29509 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:25.124785 22762 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:14:25.129685 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.110806 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.088696 29606 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.086715 29636 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:25.074961 29559 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.144904 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.111579 29141 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.194600 22765 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:25.172994 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.174767 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2600) took 1541 ms. Trace:
I20260709 13:14:25.175740 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1191) took 1530 ms. Trace:
I20260709 13:14:25.208901 22841 rpcz_store.cc:276] 0709 13:14:23.633663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.854729 (+221066us) service_pool.cc:224] Handling call
0709 13:14:24.475109 (+620380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.554570 (+ 79461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.000402 (+445832us) spinlock_profiling.cc:243] Waited 9.33 ms on lock 0x7b54001bba54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:25.174403 (+174001us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001bba50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:25.174670 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1447342336}
I20260709 13:14:25.167891 29637 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.209465 23457 rpcz_store.cc:276] 0709 13:14:23.645423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.645600 (+   177us) service_pool.cc:224] Handling call
0709 13:14:23.683004 (+ 37404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:23.889307 (+206303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.175418 (+1286111us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b540039e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:25.175615 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467702144}
W20260709 13:14:25.170174 29628 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 13:14:25.177377 29596 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:25.180589 29132 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.642s	user 0.018s	sys 0.002s
I20260709 13:14:25.177369 29518 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:25.214484 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 150767us
W20260709 13:14:25.194733 29629 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.208s	user 0.002s	sys 0.000s
I20260709 13:14:25.220305 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:25.208089 29559 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.192489 29536 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.218312 22841 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:14:25.234920 22908 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:14:25.235637 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2606) took 1327 ms. Trace:
I20260709 13:14:25.235978 22841 rpcz_store.cc:276] 0709 13:14:23.908400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.217696 (+1309296us) service_pool.cc:224] Handling call
0709 13:14:25.218082 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:25.235353 (+ 17271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.235577 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:25.216454 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 167801us
W20260709 13:14:25.237469 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 930) took 2298 ms (client timeout 2220 ms). Trace:
W20260709 13:14:25.237979 23465 rpcz_store.cc:269] 0709 13:14:22.932220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.932412 (+   192us) service_pool.cc:224] Handling call
0709 13:14:25.229850 (+2297438us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54002c7754. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:25.230055 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920119040}
I20260709 13:14:25.150465 29624 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.238474 29287 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.239320 29624 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:25.239094 29287 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.209288 29390 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.241685 29433 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:25.242259 29433 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.225485 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1199) took 1029 ms. Trace:
I20260709 13:14:25.244084 29586 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:25.217228 29626 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:25.232389 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3246) took 2166 ms. Trace:
I20260709 13:14:25.245072 29433 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.245420 22764 rpcz_store.cc:276] 0709 13:14:23.066354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.437250 (+370896us) service_pool.cc:224] Handling call
0709 13:14:24.204856 (+767606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.362325 (+157469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.232140 (+869815us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b540035dc50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:25.232317 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764346496}
I20260709 13:14:25.246863 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1165) took 1484 ms. Trace:
I20260709 13:14:25.247252 22834 rpcz_store.cc:276] 0709 13:14:23.762761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.371292 (+608531us) service_pool.cc:224] Handling call
0709 13:14:24.371678 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.426940 (+ 55262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.426978 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.436594 (+  9616us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.685843 (+249249us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:24.685896 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.685971 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.246628 (+560657us) raft_consensus.cc:1685] finished
0709 13:14:25.246652 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.246804 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29687,"raft.queue_time_us":32937,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":9526,"thread_start_us":229559,"threads_started":1,"wal-append.queue_time_us":451912}
I20260709 13:14:25.244500 23459 rpcz_store.cc:276] 0709 13:14:24.195979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.196193 (+   214us) service_pool.cc:224] Handling call
0709 13:14:24.196612 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.972684 (+776072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.972730 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:24.972788 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:24.972831 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.225155 (+252324us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400650254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:25.225232 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.225408 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5346,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":4038,"spinlock_wait_cycles":1783642368}
I20260709 13:14:25.251375 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2607) took 1342 ms. Trace:
I20260709 13:14:25.251773 22841 rpcz_store.cc:276] 0709 13:14:23.908838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.250505 (+1341667us) service_pool.cc:224] Handling call
0709 13:14:25.250848 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.251041 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.251066 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.251118 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:25.251150 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.251210 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.251326 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:25.254259 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:25.254594 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2610) took 1318 ms. Trace:
I20260709 13:14:25.255004 22834 rpcz_store.cc:276] 0709 13:14:23.936638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.250533 (+1313895us) service_pool.cc:224] Handling call
0709 13:14:25.254094 (+  3561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.254286 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.254312 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.254351 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:25.254382 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.254434 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.254538 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:25.257308 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2612) took 1320 ms. Trace:
I20260709 13:14:25.257656 22834 rpcz_store.cc:276] 0709 13:14:23.937345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.256512 (+1319167us) service_pool.cc:224] Handling call
0709 13:14:25.256822 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.256991 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.257015 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.257055 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:25.257101 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.257152 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.257253 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:25.258522 29637 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.261353 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:25.263155 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:25.264439 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2143) took 2034 ms (client timeout 1926 ms). Trace:
I20260709 13:14:25.264786 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:25.265040 29559 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.259289 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2613) took 1316 ms. Trace:
I20260709 13:14:25.260274 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:25.265451 22908 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:25.267377 29559 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.267102 22834 rpcz_store.cc:276] 0709 13:14:23.942780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.258520 (+1315740us) service_pool.cc:224] Handling call
0709 13:14:25.258825 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.258995 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.259020 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.259058 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:25.259099 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.259150 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.259239 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:25.269186 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:25.270126 22907 raft_consensus.cc:2368] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20.
I20260709 13:14:25.270159 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:25.271257 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:55836 (request call id 23) took 1366 ms. Trace:
I20260709 13:14:25.271690 22832 rpcz_store.cc:276] 0709 13:14:23.904377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.521146 (+616769us) service_pool.cc:224] Handling call
0709 13:14:25.270942 (+749796us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54000a5d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af7735 00007f68a1aef9fa 00007f68a6ab74c9 00007f68a0033c54 00007f68a0033be2 00007f68a0033b11 00007f68a0033a8c 00007f68a0032d42 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:25.271165 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796486016}
I20260709 13:14:25.272460 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:25.273658 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:25.274302 22906 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:25.231966 29581 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:25.264930 22833 rpcz_store.cc:269] 0709 13:14:23.235401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.331640 (+ 96239us) service_pool.cc:224] Handling call
0709 13:14:25.264254 (+1932614us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002b3254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:25.264363 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340326784}
I20260709 13:14:25.277280 29581 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.278847 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1172) took 1071 ms. Trace:
W20260709 13:14:25.279191 29618 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.482s	user 0.005s	sys 0.000s
W20260709 13:14:25.230269 29631 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.205s	user 0.000s	sys 0.002s
I20260709 13:14:25.218894 29606 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:25.279206 22833 rpcz_store.cc:276] 0709 13:14:24.207975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.278026 (+1070051us) service_pool.cc:224] Handling call
0709 13:14:25.278340 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.278520 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.278543 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.278588 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:25.278618 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.278669 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.278784 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:25.282934 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62715.9us
W20260709 13:14:25.258764 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.286027 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:25.286234 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.291148 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:25.295099 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.295776 22764 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:25.295821 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.296492 22758 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:25.298588 29642 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:25.290972 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ad66b5c9842437d9320507b92b78fd5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:25.302129 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.303669 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.308497 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:25.310011 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:25.310741 29554 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:25.311214 29554 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:25.312341 29554 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.299889 29637 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:25.314364 29637 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.316838 29637 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.317026 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:25.317425 29433 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.305903 29409 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:25.318012 29433 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.318387 29312 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.318437 29409 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:25.319049 29312 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.320186 29433 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:25.319800 29409 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.323532 29517 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.326087 29596 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.326737 29596 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.330086 29642 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.330569 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:25.330719 29642 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.333034 29644 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:25.288682 29613 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:25.222875 29132 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.390117 22833 raft_consensus.cc:1217] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Deduplicated request from leader. Original: 10.6->[11.7-16.10]   Dedup: 11.7->[13.8-16.10]
I20260709 13:14:25.343344 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:25.344606 22757 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:25.419955 22833 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [11.7-16.10]
W20260709 13:14:25.305323 29127 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.943s	user 0.012s	sys 0.004s
I20260709 13:14:25.313409 29554 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:25.329031 29596 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.279844 29581 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:25.348069 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:25.288529 29390 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.300232 29634 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.226s	user 0.000s	sys 0.004s
I20260709 13:14:25.253269 29536 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.404872 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:25.333492 29642 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.317437 29637 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:25.408447 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 811) took 1005 ms. Trace:
I20260709 13:14:25.352325 29312 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:25.413147 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:25.333566 29644 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:25.357116 23466 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
W20260709 13:14:25.409163 29542 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.965s	user 0.014s	sys 0.002s
I20260709 13:14:25.427983 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1170) took 1221 ms. Trace:
I20260709 13:14:25.429598 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2186) took 1780 ms. Trace:
I20260709 13:14:25.431061 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.438751 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:25.338040 29613 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.504101 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.535202 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.535576 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:25.523492 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2611) took 1586 ms. Trace:
I20260709 13:14:25.503991 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:25.536232 22841 rpcz_store.cc:276] 0709 13:14:23.937003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.252888 (+1315885us) service_pool.cc:224] Handling call
0709 13:14:25.253140 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.253295 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:25.301361 (+ 48066us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.522936 (+221575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.523027 (+    91us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:25.523102 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.523213 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.523402 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45229,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":78155,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":141470,"threads_started":1,"wal-append.queue_time_us":132573}
I20260709 13:14:25.538763 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:25.407390 29287 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:25.291908 29606 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.540023 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:25.588299 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.588431 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.538614 23463 rpcz_store.cc:276] 0709 13:14:24.403624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.403797 (+   173us) service_pool.cc:224] Handling call
0709 13:14:24.404164 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:24.404861 (+   697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.404891 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:24.429312 (+ 24421us) log.cc:844] Serialized 52 byte log entry
0709 13:14:24.662225 (+232913us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:24.662315 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.662409 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.408035 (+745626us) raft_consensus.cc:1685] finished
0709 13:14:25.408058 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.408378 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9771,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":4117,"thread_start_us":193268,"threads_started":1,"wal-append.queue_time_us":490686}
I20260709 13:14:25.589588 22841 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:25.592367 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1168) took 1692 ms. Trace:
I20260709 13:14:25.593885 22841 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:14:25.505028 29647 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:14:25.592852 22835 rpcz_store.cc:276] 0709 13:14:23.900092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.440889 (+540797us) service_pool.cc:224] Handling call
0709 13:14:24.479900 (+ 39011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.524740 (+ 44840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.524775 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.525322 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.730855 (+205533us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:24.931336 (+200481us) log.cc:844] Serialized 14 byte log entry
0709 13:14:24.932421 (+  1085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.932532 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.592150 (+659618us) raft_consensus.cc:1685] finished
0709 13:14:25.592176 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.592307 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44026,"prepare.queue_time_us":323925,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":49,"raft.queue_time_us":47926,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":7932,"thread_start_us":376933,"threads_started":2,"wal-append.queue_time_us":462160}
I20260709 13:14:25.427415 29518 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:25.595690 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2163) took 1068 ms. Trace:
I20260709 13:14:25.541723 29644 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:25.603713 22835 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:25.604667 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 814) took 1122 ms. Trace:
I20260709 13:14:25.610848 22841 rpcz_store.cc:276] 0709 13:14:24.527103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.591996 (+1064893us) service_pool.cc:224] Handling call
0709 13:14:25.593651 (+  1655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:25.595371 (+  1720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.595628 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:25.620826 22841 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 9. (term mismatch)
I20260709 13:14:25.621393 22841 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 8
I20260709 13:14:25.621734 22841 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 20.9
I20260709 13:14:25.622030 22841 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:25.505534 29639 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
W20260709 13:14:25.505092 29645 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 13:14:25.623934 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 189009us
I20260709 13:14:25.670445 22838 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:25.624457 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.646983 22841 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:14:25.674099 22757 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:25.659909 22910 rpcz_store.cc:276] 0709 13:14:23.649129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.649365 (+   236us) service_pool.cc:224] Handling call
0709 13:14:23.649741 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:23.759102 (+109361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.417182 (+1658080us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b54000fb254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:25.429482 (+ 12300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153279872}
W20260709 13:14:25.660773 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.541923 29569 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.504359 29593 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.673702 22832 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:25.533627 29609 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:14:25.615581 23462 rpcz_store.cc:276] 0709 13:14:24.482047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.607569 (+125522us) service_pool.cc:224] Handling call
0709 13:14:24.608099 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.608676 (+   577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.608722 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.609389 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.992263 (+382874us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:25.050225 (+ 57962us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.051047 (+   822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.051201 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.276664 (+225463us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b24003974d8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea4309d 00007f68a167510e 00007f68a168c88c 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221
0709 13:14:25.604376 (+327712us) raft_consensus.cc:1685] finished
0709 13:14:25.604473 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.604605 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":75788,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":64,"raft.queue_time_us":4714,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":20877,"spinlock_wait_cycles":50923776,"thread_start_us":355262,"threads_started":2,"wal-append.queue_time_us":401073}
I20260709 13:14:25.714330 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:25.685263 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:25.716463 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.591428 29637 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:25.723206 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:25.724274 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.719455 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:25.596532 29141 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:25.588532 29519 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:14:25.730419 29637 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.741937 29642 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:25.753731 29642 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:25.755000 29642 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:25.757730 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.599570 29390 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.446673 29636 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:25.749995 29002 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.005s
I20260709 13:14:25.771739 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1202) took 1239 ms. Trace:
I20260709 13:14:25.772269 23461 rpcz_store.cc:276] 0709 13:14:24.532014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.721541 (+189527us) service_pool.cc:224] Handling call
0709 13:14:24.722151 (+   610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.722491 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.722530 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.733905 (+ 11375us) log.cc:844] Serialized 26 byte log entry
0709 13:14:25.055009 (+321104us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:25.055064 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.055150 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.771470 (+716320us) raft_consensus.cc:1685] finished
0709 13:14:25.771498 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.771667 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":272974,"threads_started":1,"wal-append.queue_time_us":330574}
I20260709 13:14:25.773861 29589 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:25.774811 29321 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:25.774488 29589 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.775468 29312 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.775373 29321 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.655685 29518 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.751889 29653 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.881732 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.892086 22833 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:25.730590 29141 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:25.895992 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.893394 29653 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.904982 29653 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:25.907274 29518 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.724558 29569 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.935113 29132 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:25.732386 29593 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.589779 29596 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.286s
I20260709 13:14:25.859440 29002 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:25.944034 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 270009us
I20260709 13:14:25.947767 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:25.908752 29536 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:25.909282 29141 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:25.900146 29636 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:25.936340 29321 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:25.953025 29002 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:25.944921 29132 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.892043 29312 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.953732 29653 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.996047 22838 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
W20260709 13:14:25.988692 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 449989us
I20260709 13:14:25.999379 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:26.001627 22761 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:26.002288 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:26.003388 22762 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:26.006194 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:25.977386 29613 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:26.017441 29646 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.081s	user 0.001s	sys 0.001s
I20260709 13:14:25.995226 29637 raft_consensus.cc:476] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Not starting pre-election -- already a leader
I20260709 13:14:25.986107 29553 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.025691 29553 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:25.987722 29662 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:26.006045 29542 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:25.975438 29636 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.012982 29518 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:26.043339 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77356.4us
I20260709 13:14:25.997339 29132 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.036461 29553 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.034055 29662 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:26.004518 29653 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:26.072691 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.078886 29312 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.080109 29312 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:26.080615 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.081298 29589 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.042593 29613 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:26.081684 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:26.082046 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.083220 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.083424 29606 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.083890 22906 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:26.084908 29180 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.085769 29522 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.086385 29522 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:26.087180 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.092878 29499 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.093042 29658 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.094697 29658 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.095319 29569 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.038076 29542 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.077674 29594 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:26.084849 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.111045 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.111922 23466 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:26.099551 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:26.075336 29623 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.647s	user 0.015s	sys 0.000s
I20260709 13:14:26.119234 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.120281 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.120635 22908 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:26.074244 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1171) took 1867 ms (client timeout 1712 ms). Trace:
I20260709 13:14:26.081310 22910 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:26.106385 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:26.107074 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.065068 29593 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.108546 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:26.121340 22838 rpcz_store.cc:269] 0709 13:14:24.207291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.236855 (+ 29564us) service_pool.cc:224] Handling call
0709 13:14:26.074024 (+1837169us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:26.074141 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3006201728}
I20260709 13:14:26.113718 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:26.109265 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93464.7us
I20260709 13:14:26.115614 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.117730 22832 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:26.119355 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.122457 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.044562 29596 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:14:26.124712 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.124984 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.109599 29519 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.110038 29542 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:26.137758 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.160418 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:26.085875 29132 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:26.134852 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:26.139701 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.167672 22910 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:26.137295 22913 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:26.128726 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.135186 23465 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:26.135576 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.141772 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.173696 22915 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:26.124243 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.146000 22909 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:26.108348 29522 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.149222 23459 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:14:26.127240 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.153012 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.154778 29180 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.158126 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.186707 22911 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 13: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:26.187347 22914 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:26.158815 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.112743 29569 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.098255 29662 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:26.128188 29287 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.110271 29606 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 13:14:26.208000 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.181061 23461 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:26.116963 29499 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:26.187994 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:26.170313 23462 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:26.127416 29660 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.192123 22912 raft_consensus.cc:2410] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:26.217821 22832 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:26.162290 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.195967 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.127873 29581 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:26.215135 29624 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.975s	user 0.011s	sys 0.003s
I20260709 13:14:26.197698 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:26.148504 29593 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:26.226325 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.127995 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.199563 29180 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:26.189311 29630 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.230s	user 0.010s	sys 0.006s
I20260709 13:14:26.235862 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2726) took 1592 ms. Trace:
I20260709 13:14:26.239804 22763 rpcz_store.cc:276] 0709 13:14:24.643838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.644017 (+   179us) service_pool.cc:224] Handling call
0709 13:14:25.262082 (+618065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.262313 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:14:25.262649 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.413918 (+151269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.413976 (+    58us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:25.414024 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.414106 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.234900 (+820794us) spinlock_profiling.cc:243] Waited 984 us on lock 0x7b1800433900. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4
0709 13:14:26.235670 (+   770us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400369050. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:26.235797 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38740,"prepare.queue_time_us":1276,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":89,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":25198,"spinlock_wait_cycles":1282314752,"thread_start_us":150827,"threads_started":1,"wal-append.queue_time_us":638765}
W20260709 13:14:26.206306 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 91604.8us
I20260709 13:14:26.215945 22908 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:26.229696 22757 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:14:26.253008 22757 pending_rounds.cc:85] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 11
I20260709 13:14:26.253298 22757 pending_rounds.cc:107] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:14:26.253557 22757 raft_consensus.cc:2889] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:26.230461 29581 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:26.257953 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 210550us
I20260709 13:14:26.221205 29499 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:26.234735 29554 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.793s	user 0.012s	sys 0.000s
I20260709 13:14:26.215660 29569 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.235850 29589 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:26.236617 29180 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.178300 29519 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:26.228926 29644 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.001s
I20260709 13:14:26.215536 29287 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.238411 29669 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:26.225978 29660 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.258157 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:26.260368 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 114400us
I20260709 13:14:26.259395 22841 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:26.217670 29132 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:14:26.239269 29002 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.266353 29624 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.239673 29593 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.306264 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3260) took 1011 ms. Trace:
I20260709 13:14:26.405201 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1167) took 2636 ms. Trace:
I20260709 13:14:25.536643 22834 rpcz_store.cc:276] 0709 13:14:24.206863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.268604 (+1061741us) service_pool.cc:224] Handling call
0709 13:14:25.269085 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.269309 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:25.415432 (+146123us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.427555 (+ 12123us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.427621 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:25.427676 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.427753 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.427906 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32794,"prepare.queue_time_us":29400,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":15757,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":90693,"thread_start_us":121452,"threads_started":2}
I20260709 13:14:26.414909 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.374365 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 999) took 1998 ms. Trace:
I20260709 13:14:26.309340 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1174) took 1785 ms. Trace:
I20260709 13:14:26.416865 22839 rpcz_store.cc:276] 0709 13:14:24.523793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.727233 (+203440us) service_pool.cc:224] Handling call
0709 13:14:26.309136 (+1581903us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:26.309267 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639878912}
I20260709 13:14:26.412510 22840 rpcz_store.cc:276] 0709 13:14:23.769074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.391104 (+622030us) service_pool.cc:224] Handling call
0709 13:14:24.405375 (+ 14271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.522209 (+116834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.522252 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.522936 (+   684us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.729905 (+206969us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:24.729956 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.263538 (+533582us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b5400411ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:25.263600 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:26.156931 (+893331us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:26.404848 (+247917us) raft_consensus.cc:1685] finished
0709 13:14:26.404884 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.405104 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60168,"raft.queue_time_us":60429,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":132856960,"thread_start_us":206551,"threads_started":1,"wal-append.queue_time_us":384667}
I20260709 13:14:26.422202 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:26.423031 22834 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:26.262229 29606 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:26.364221 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 120395us
W20260709 13:14:26.428483 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.859s (SENT) [suppressed 13 similar messages]
W20260709 13:14:26.423672 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 288878us
I20260709 13:14:26.416961 22759 rpcz_store.cc:276] 0709 13:14:25.295028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.295432 (+   404us) service_pool.cc:224] Handling call
0709 13:14:25.316375 (+ 20943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.494737 (+178362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.494789 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.494858 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:25.494914 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.305958 (+811044us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54002dcdd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:26.306016 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.306190 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52242,"raft.queue_time_us":63383,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":137,"spinlock_wait_cycles":288917376}
I20260709 13:14:26.429461 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:26.431360 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:26.434049 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 182055us
I20260709 13:14:26.437038 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.438967 23466 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:26.416065 22760 rpcz_store.cc:276] 0709 13:14:24.375665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.375793 (+   128us) service_pool.cc:224] Handling call
0709 13:14:24.376136 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.639392 (+263256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.639430 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.667143 (+ 27713us) log.cc:844] Serialized 26 byte log entry
0709 13:14:25.049092 (+381949us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:25.049984 (+   892us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.214170 (+164186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.214304 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:26.302063 (+1087759us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b18003ff100. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:26.323318 (+ 21255us) raft_consensus.cc:1685] finished
0709 13:14:26.374001 (+ 50683us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.374280 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3459,"prepare.queue_time_us":287,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":65959,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":13399,"spinlock_wait_cycles":12430208,"thread_start_us":694377,"threads_started":3,"wal-append.queue_time_us":378524}
I20260709 13:14:26.440883 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.444317 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:26.297344 29569 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.307081 29499 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.316272 29658 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.452524 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.258613 29664 raft_consensus.cc:2764] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:14:26.432231 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.454061 22837 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:26.308856 29581 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.418195 29636 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.458510 22840 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:26.465392 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.459331 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2152) took 3115 ms. Trace:
W20260709 13:14:26.459390 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 193615us
I20260709 13:14:26.311190 29390 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.314891 29141 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.466246 22840 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:26.475757 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.477141 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.478025 22840 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:26.479104 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.481101 22840 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:26.435417 29653 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.474071 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.419356 29624 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:26.483126 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 50979.7us
I20260709 13:14:26.486459 22839 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:26.484977 22838 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:26.466786 22836 rpcz_store.cc:276] 0709 13:14:23.339433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.742274 (+402841us) service_pool.cc:224] Handling call
0709 13:14:23.742729 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.555563 (+812834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.453677 (+1898114us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:26.455059 (+  1382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710288768}
I20260709 13:14:26.486970 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.416601 29644 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.495518 22834 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:26.496834 22833 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:26.497184 22836 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:14:26.482625 29658 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:26.499576 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70209.4us
I20260709 13:14:26.482879 29586 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:26.486441 29671 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:26.499250 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.502672 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.505875 22759 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:26.504222 22765 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:26.508807 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.469867 29606 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.496929 29518 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:26.511966 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.515129 22762 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:26.516109 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.516700 22762 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:26.501411 29658 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.525853 22910 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:26.523379 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.526742 22759 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:26.532313 22832 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:26.505368 29671 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:26.510829 29676 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:26.517486 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.528479 29658 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.514655 29518 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.514068 29673 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:26.546396 29668 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:14:26.547683 29644 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.550176 23464 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:26.550912 29588 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.551388 29588 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.558708 29519 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.560376 29141 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.565605 29287 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.538282 29637 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.568313 29637 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.570927 29637 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.573253 29660 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.575295 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1008) took 1268 ms. Trace:
I20260709 13:14:26.575744 22757 rpcz_store.cc:276] 0709 13:14:25.306824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.677256 (+370432us) service_pool.cc:224] Handling call
0709 13:14:25.762694 (+ 85438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:26.253981 (+491287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.575025 (+321044us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b540033fed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:26.575222 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072807680}
W20260709 13:14:26.580710 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.538838 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.584065 22759 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:26.534580 29554 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.506302 29586 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.542591 29676 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.540930 29679 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:26.616779 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96813.6us
I20260709 13:14:26.579279 29518 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:26.618695 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:26.625207 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.627079 22757 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:26.633476 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.634387 22759 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:26.604249 29637 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.652315 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:26.653252 22759 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:26.603430 29671 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.658749 29671 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.674751 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1164) took 2959 ms. Trace:
I20260709 13:14:26.675282 22837 rpcz_store.cc:276] 0709 13:14:23.715754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.926307 (+210553us) service_pool.cc:224] Handling call
0709 13:14:26.453858 (+2527551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:26.469720 (+ 15862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.674547 (+204827us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b54001bb550. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:26.674684 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5809459456}
I20260709 13:14:26.676589 29644 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:26.677765 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 179865us
I20260709 13:14:26.610858 29679 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.608237 29676 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.617446 29593 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:26.685773 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 250520us
I20260709 13:14:26.601985 29554 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.677160 29644 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.598758 29673 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:26.700078 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 81565.6us
I20260709 13:14:26.699193 22807 leader_election.cc:400] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20.
I20260709 13:14:26.701017 22807 leader_election.cc:403] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:14:26.703372 29542 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.516s	user 0.015s	sys 0.000s
I20260709 13:14:26.703706 29606 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.703900 29542 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.683259 29679 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:26.688306 29306 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:14:26.707398 29624 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.679419 29637 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.703253 29677 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:26.722019 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:26.713395 29637 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.715221 29644 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:26.731527 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:26.743072 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:26.753197 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 73384us
I20260709 13:14:26.756106 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.754873 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:26.763058 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:26.734587 29672 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.769053 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:26.770787 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.771481 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:26.772722 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77427.4us
I20260709 13:14:26.749104 29677 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:26.776331 29672 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.758170 29660 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:26.795357 22759 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:26.794394 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:26.797812 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:26.781399 29685 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:26.801702 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.785677 29677 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.802464 22910 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:26.806175 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.786530 29613 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:26.794255 29637 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.795279 29653 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:26.806839 22906 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:26.808007 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.808374 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.831055 22910 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:26.812422 29673 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.813262 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.806982 29677 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:26.815217 22907 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:26.793608 29644 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:26.841852 29688 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:14:26.842270 22835 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:26.782181 29678 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:14:26.844440 29542 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.838516 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.847007 29542 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:26.847123 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.847296 29433 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:26.847872 29433 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:26.849614 29673 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:26.850051 29673 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.850394 29673 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:26.797945 29684 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:26.853746 29684 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:26.854228 29684 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:26.802240 29390 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:26.818888 29588 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.800390 29519 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:26.791575 29670 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.025s
I20260709 13:14:26.799201 29321 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:14:26.860080 29321 consensus_queue.cc:1243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:26.853916 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.862331 29642 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.862425 22906 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:26.863162 29642 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.864883 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:26.821887 23430 connection.cc:256] RPC call timeout handler was delayed by -1.0776302629999464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 13:14:26.866858 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1006) took 1572 ms. Trace:
I20260709 13:14:26.824249 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.825371 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.867255 22764 rpcz_store.cc:276] 0709 13:14:25.296883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.296997 (+   114us) service_pool.cc:224] Handling call
0709 13:14:25.297270 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.623676 (+326406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.623718 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.623770 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:25.623804 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.866569 (+1242765us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400222ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:26.866634 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.866792 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750205824}
W20260709 13:14:26.811518 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.833658 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.837914 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:26.869402 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56221us
I20260709 13:14:26.869676 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.870185 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.799490 29681 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:26.878454 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.828874 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.814539 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:26.870317 22909 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:26.879673 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.879952 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.880378 22759 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:26.866485 29519 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:26.872613 29390 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:26.873020 29678 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:26.904588 29678 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.904994 29678 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:26.841243 29644 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.905951 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.906718 23462 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:26.907217 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.873605 29586 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:26.874203 29658 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:26.907770 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:26.909731 29658 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.909865 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.910511 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.907804 29644 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:26.886708 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.911882 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.910542 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:26.912518 22760 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:26.912592 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.913357 23457 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:26.913617 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.914341 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.882154 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.899729 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.825446 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.901315 29671 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:26.914979 22757 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
W20260709 13:14:26.901458 23464 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 818) took 2266 ms (client timeout 2160 ms). Trace:
I20260709 13:14:26.901795 29665 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:26.943341 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.848804 29433 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.902206 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.899307 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.949951 22838 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:26.903178 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.830461 29613 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.951380 22906 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:26.907882 22840 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:26.847441 29542 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.911147 22765 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:26.914836 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.878978 29681 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.900688 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.970441 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:26.970798 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70212.6us
I20260709 13:14:26.972684 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:26.923681 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:26.924345 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44852.4us
I20260709 13:14:26.974021 22764 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:26.917996 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:26.917996 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.926616 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.915280 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.976270 22907 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:26.931188 22766 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:14:26.931557 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.977214 22766 pending_rounds.cc:85] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:26.932452 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.977550 22766 pending_rounds.cc:107] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:14:26.977842 22766 raft_consensus.cc:2889] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:26.933209 23461 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:26.919205 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:26.940531 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:26.942116 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.944379 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:26.979835 22757 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:26.935664 23464 rpcz_store.cc:269] 0709 13:14:24.635032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.702899 (+ 67867us) service_pool.cc:224] Handling call
0709 13:14:26.901243 (+2198344us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54000b5c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:26.901377 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4239035136}
I20260709 13:14:26.888605 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:26.956516 22909 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:26.839210 29637 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:26.941708 29606 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.959352 29642 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:26.960340 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.962809 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.966006 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.968447 22835 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:26.979100 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.982612 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.974737 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:26.932317 29390 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.934619 29671 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:27.015381 22759 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:26.993988 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.016316 22839 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:26.996730 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.997390 22837 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:26.998186 22763 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:27.017266 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:27.000164 22766 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:27.000820 22909 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:27.019218 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.019806 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:26.995777 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76682us
I20260709 13:14:26.934199 29644 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.008293 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:26.992280 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:26.899307 29624 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.012817 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:27.012972 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.942756 29180 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:26.943017 29665 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:27.016459 22908 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:27.009253 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.023597 22758 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:27.035410 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:26.971480 29542 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:27.032059 22760 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:26.980521 29409 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.977012 29681 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:26.991904 29581 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:27.048949 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 74292.9us
I20260709 13:14:27.071514 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.014040 29390 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:27.049299 22762 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:27.052270 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:27.026033 29671 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:27.050438 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:26.966668 29640 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:14:26.994539 29433 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.004110 29589 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:27.002983 29608 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:27.043196 29483 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.052539 29180 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.053017 29624 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.065016 29519 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:27.065269 29637 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.011387 29518 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.046911 29644 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.133472 29644 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.143679 29180 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:27.156646 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.159735 29518 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.163628 29624 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.067286 29672 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.018842 29616 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.186509 29616 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.187027 29616 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.187454 29696 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:14:27.190259 29695 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:14:27.069186 29660 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20.
I20260709 13:14:27.172401 29693 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:27.194051 29693 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.194501 29693 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:27.069932 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:27.198251 22759 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:27.037113 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:27.198911 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 161881us
I20260709 13:14:27.199702 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.052423 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:26.966383 29677 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:27.076625 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 133620us
I20260709 13:14:27.077435 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:27.079082 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 140597us
I20260709 13:14:27.062722 29409 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.089731 22757 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:27.044418 29642 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.055501 29594 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:26.997100 29593 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:14:27.070739 29581 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.011809 29606 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.042788 29306 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:14:27.164640 22761 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:27.066306 29287 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:27.201346 22910 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:14:27.222303 29644 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:27.170648 29676 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:27.129545 29662 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.911s	user 0.010s	sys 0.005s
W20260709 13:14:27.210057 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.086597 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.211367 29637 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:27.213188 29554 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.208289 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:27.219929 22758 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: [21.9-21.9]
I20260709 13:14:27.221467 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.217587 29684 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.267623 29664 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:27.251778 29594 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:27.268196 29664 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:27.239584 29321 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:14:27.269622 29684 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.128101 29694 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.270706 29694 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.272380 22836 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:14:27.273406 29694 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.129904 29589 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:27.247452 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 183274us
I20260709 13:14:27.227255 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:27.093545 29673 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.274657 29589 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.276235 29673 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.216604 29616 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.259640 29670 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.278141 22766 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:14:27.277926 29670 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:27.279225 29679 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.573s	user 0.005s	sys 0.009s
I20260709 13:14:27.279353 29616 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:27.262296 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 175888us
I20260709 13:14:27.279959 29679 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.281816 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.283020 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.283317 29596 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:27.283782 29596 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.284207 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.285785 29569 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:27.263463 29662 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.286187 29669 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:27.286630 29669 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.286727 22915 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:27.287017 29669 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.287865 29662 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:27.267091 29637 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.288574 29637 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:27.289180 22834 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:27.262916 29703 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:27.284273 29596 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.290697 29644 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:27.321866 29703 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.322532 29703 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.322218 29644 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.286192 29569 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.325769 29569 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:27.326563 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.327275 22906 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:27.327941 29606 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.329463 29679 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.331351 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.293481 29677 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.294665 29613 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.269138 29664 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.249377 29581 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.267781 29586 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.202329 22832 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:27.268950 29594 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.189253 29696 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:27.333715 29616 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:27.310988 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 83846.5us
I20260709 13:14:27.397727 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:27.399403 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:27.276671 29693 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.403421 29693 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.304975 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:27.405184 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 385456us
I20260709 13:14:27.313643 29321 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:27.313993 29518 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:27.315419 22833 raft_consensus.cc:2368] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22.
I20260709 13:14:27.316412 29676 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.240427 29624 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:14:27.352474 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 280025us
W20260709 13:14:27.305892 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 84553.6us
I20260709 13:14:27.382094 29581 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.388293 29665 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.366444 29586 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.390626 22909 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:14:27.394322 29664 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:27.406891 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 102034us
I20260709 13:14:27.436278 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:27.409008 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.409272 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:27.428368 22908 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
W20260709 13:14:27.428835 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 115989us
W20260709 13:14:27.458727 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49820.4us
I20260709 13:14:27.376739 29671 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.428881 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.463517 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.468000 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:27.463730 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57725.4us
I20260709 13:14:27.472533 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:27.464682 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.474824 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:27.396204 29696 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.476121 29696 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:27.476068 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 267846us
I20260709 13:14:27.410179 29433 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:27.466181 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:27.462809 29586 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.484437 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.425308 29693 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:27.478041 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 216810us
I20260709 13:14:27.411631 29677 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:27.483978 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.446693 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:27.467518 29653 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.001s
I20260709 13:14:27.486594 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.482774 22730 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.493829 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.489007 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.426154 29616 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:27.471827 29312 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:27.480813 29593 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:27.489007 22913 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:27.474340 29662 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:27.479116 29180 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:27.500213 23464 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:14:27.490865 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44294.6us
I20260709 13:14:27.485695 29287 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:27.519304 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.490460 29665 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.393442 29681 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.485213 29483 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:27.485064 29642 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:27.516283 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.518241 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:27.496136 29499 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:27.493891 29640 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.526667 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:27.526384 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 43686.6us
I20260709 13:14:27.495488 29679 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:27.538225 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.531549 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:27.499483 29653 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:27.480722 29637 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.518139 29287 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.555138 29287 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.557761 29616 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.481048 29306 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:14:27.560379 29581 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.517956 29409 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.485086 29606 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:14:27.538478 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.511538 29662 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.498898 29693 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:27.487924 29588 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:27.550848 23462 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 14: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:27.563782 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47644.2us
W20260709 13:14:27.543715 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54772.4us
W20260709 13:14:27.551211 22907 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.003s
I20260709 13:14:27.520758 29180 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.551899 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:27.552788 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:27.553891 22912 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:27.565794 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.566162 22907 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:27.526418 29593 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:27.569993 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.525424 29644 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:27.570027 22915 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:27.573972 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.577155 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:27.577860 23466 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:27.578068 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:27.578850 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:27.579453 23458 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.579722 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.580540 22907 raft_consensus.cc:2376] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21. Re-sending same reply.
I20260709 13:14:27.581405 22806 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.583993 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:27.550601 29679 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.583662 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.587193 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:27.588572 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:27.590068 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:27.542842 29692 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:14:27.564291 29662 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:27.564685 29624 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.600157 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.607009 22907 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:27.568024 29685 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:27.603920 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:27.533721 29569 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.571399 29180 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:27.629791 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:27.633992 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:27.635769 22757 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:27.623236 23458 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:27.608096 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.638167 22907 raft_consensus.cc:2376] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18. Re-sending same reply.
I20260709 13:14:27.617924 29685 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.638996 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.639694 29685 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.640858 29671 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:27.642560 23466 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:27.643958 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.644722 22838 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:27.577574 29644 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.573633 29594 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:14:27.596452 29519 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:27.618655 22906 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:27.648419 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.649060 22906 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:27.601962 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:27.632761 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:27.650295 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48442.4us
I20260709 13:14:27.645864 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.656415 22907 raft_consensus.cc:2368] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
I20260709 13:14:27.656852 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.657467 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.653947 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.658754 22841 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:27.598836 29679 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:27.600288 29409 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:27.590526 29613 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.615675 29608 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:27.662735 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:27.613934 29708 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:27.663389 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.562985 29640 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.663862 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.664785 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.665812 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.654175 22807 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:27.667073 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:27.648787 29678 consensus_meta.cc:287] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.000s
I20260709 13:14:27.670910 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:27.614774 29606 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.614545 29624 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.699281 22837 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:27.702378 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.648784 29644 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.597036 29499 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:14:27.647681 29642 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:27.596717 29433 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:14:27.706502 22840 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:27.722134 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:27.706842 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52766.5us
I20260709 13:14:27.734932 22906 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:27.614102 29287 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.734126 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:27.780891 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:27.781586 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.782521 23460 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:27.666625 29613 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.711392 22760 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:27.737404 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:27.780861 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:27.784832 22757 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:27.785077 22841 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:27.780956 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.660981 29677 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:27.787958 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52300.4us
I20260709 13:14:27.642112 29593 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.789147 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:27.787667 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53696.1us
I20260709 13:14:27.790426 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.793118 22840 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:27.791447 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:27.663823 29554 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.661480 29684 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.793903 22832 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:27.710777 29637 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:27.826030 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104125us
I20260709 13:14:27.826972 22879 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:27.796141 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.827950 22907 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.781112 29678 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.803330 29670 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.667127 29669 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.831612 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.794417 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:27.732407 29642 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.834846 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:27.837307 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.714866 29658 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:27.666622 29519 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:27.817780 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.842161 22906 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:27.787950 29613 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.740250 29606 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:27.824652 22832 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:14:27.830478 22835 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:27.699026 29708 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:27.730746 29693 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.849889 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.847770 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.812656 29672 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:27.839192 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:27.787580 29608 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:27.848795 29694 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:27.875026 29608 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:27.831423 29713 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:14:27.783110 29596 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.869347 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.736992 29624 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.872756 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.877440 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:27.878373 22911 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:27.816915 29554 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.862183 29499 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:27.865957 29606 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.862510 29616 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:27.848909 29642 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:27.846225 29409 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:14:27.877588 29681 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:27.866737 29658 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:27.826254 29684 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.847894 29653 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:14:27.970337 29671 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:27.970757 29671 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:27.970942 29684 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.975958 29684 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:27.874018 29672 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.842818 29132 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.910719 29287 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:27.886600 22907 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:27.911427 29669 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:27.914314 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:28.012842 22757 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:27.913930 29519 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.920895 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:27.935112 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65887.1us
I20260709 13:14:27.944274 29616 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.911015 29569 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:27.909466 29706 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:14:27.935137 22906 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:27.944526 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:27.946259 23460 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:27.872888 29637 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:14:27.960896 29708 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:27.967022 29306 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:27.968355 29658 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:27.981190 29678 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:27.982107 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 997) took 1064 ms. Trace:
I20260709 13:14:28.039871 22909 rpcz_store.cc:276] 0709 13:14:26.917184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.018859 (+101675us) service_pool.cc:224] Handling call
0709 13:14:27.981846 (+962987us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:27.982036 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851523456}
I20260709 13:14:27.880672 29594 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:27.971769 29671 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.048617 22910 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:28.049726 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 989) took 1134 ms. Trace:
I20260709 13:14:28.026871 29616 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:28.050130 22910 rpcz_store.cc:276] 0709 13:14:26.914865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.257277 (+342412us) service_pool.cc:224] Handling call
0709 13:14:28.047995 (+790718us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b5800016298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:14:28.048337 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.049308 (+   971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.049648 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8844032}
W20260709 13:14:27.993845 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 418974us
W20260709 13:14:27.997453 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 528129us
W20260709 13:14:27.944559 29390 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.852s	user 0.007s	sys 0.006s
W20260709 13:14:27.962219 29542 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.895s	user 0.011s	sys 0.003s
I20260709 13:14:27.999882 23431 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:27.997954 29672 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.964057 29483 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:28.029808 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:28.060259 22906 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 13.
I20260709 13:14:28.034634 29306 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:27.996003 29676 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:28.033901 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:28.034160 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 83193.4us
I20260709 13:14:27.998203 29670 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:28.041374 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.042440 22908 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "023c6d83f2864eedb2775b615045955e"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:28.144375 22908 raft_consensus.cc:606] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:28.145041 29409 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:27.977072 29180 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.106415 29606 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:28.067929 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.038558 29640 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.073401 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:28.076928 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:27.999387 29684 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.083529 22909 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:28.061256 29483 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.085222 29637 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.094578 29596 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.092233 29732 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:28.097801 29703 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.775s	user 0.006s	sys 0.009s
I20260709 13:14:28.101794 29671 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:28.054544 29594 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.059330 29390 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.067354 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:28.117826 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:28.122270 22835 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: [21.9-21.9]
W20260709 13:14:28.146278 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 90206.8us
I20260709 13:14:28.058631 29708 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:28.152086 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:28.152096 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:28.102743 29624 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.162175 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.207206 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:28.179980 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 112757us
I20260709 13:14:28.184881 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.187997 29672 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:28.191651 29664 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.003s
I20260709 13:14:28.195592 29306 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.197158 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:28.197626 29662 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.004s
I20260709 13:14:28.164506 29613 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.198765 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:28.154177 29703 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.153796 29732 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.154263 29676 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.149466 29180 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.201149 22832 raft_consensus.cc:1217] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Deduplicated request from leader. Original: 10.6->[11.7-17.10]   Dedup: 11.7->[13.8-17.10]
I20260709 13:14:28.207311 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.061291 29542 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:28.164995 29586 consensus_meta.cc:287] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.669s	user 0.007s	sys 0.006s
I20260709 13:14:28.179507 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:28.254670 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 75252.7us
I20260709 13:14:28.205122 29693 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:28.216936 29608 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.216163 29677 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.242585 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:28.243063 22908 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 13.
I20260709 13:14:28.266072 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.267110 22908 raft_consensus.cc:2368] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
I20260709 13:14:28.152320 29483 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:28.247654 22759 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:28.160188 29679 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:28.272742 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 273091us
I20260709 13:14:28.253373 29732 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:28.219336 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:28.253222 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:28.250418 22832 pending_rounds.cc:85] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 7
I20260709 13:14:28.278620 22832 pending_rounds.cc:107] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 12.8
W20260709 13:14:28.278756 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64355.2us
I20260709 13:14:28.256340 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:28.152614 29637 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.256404 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.232085 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:28.280769 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:28.281657 23466 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:28.280699 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.159271 29670 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.258206 29693 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.261004 22840 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:28.278968 22832 raft_consensus.cc:2889] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.160207 29554 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.283480 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.153568 29684 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.292672 29698 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:28.280004 29608 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.232479 29644 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:28.294588 29729 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:28.295989 22910 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:28.257893 29709 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:28.296633 22757 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:14:28.280615 29677 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.263821 29737 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:28.297689 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1179) took 1475 ms (client timeout 1908 ms). Trace:
I20260709 13:14:28.317296 29606 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.299891 29637 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.301580 29672 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.244702 29642 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.303586 22906 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:14:28.305873 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:28.281917 29180 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:28.310572 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.319087 22907 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:28.441574 22907 raft_consensus.cc:606] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:28.243650 29662 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.257740 29586 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.323035 29693 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:28.453389 22664 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 8 to 11, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 11 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.307186 29409 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.260996 29616 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.296737 29725 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:28.398581 22757 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 13:14:28.403111 22833 rpcz_store.cc:269] 0709 13:14:26.822712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.823865 (+  1153us) service_pool.cc:224] Handling call
0709 13:14:28.297418 (+1473553us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54003219d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:28.297623 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026093440}
I20260709 13:14:28.299212 29581 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:28.404445 22807 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:28.406970 22761 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:28.406935 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 289266us
I20260709 13:14:28.347074 29658 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:28.412464 22911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3534) took 1582 ms (client timeout 2022 ms). Trace:
W20260709 13:14:28.419294 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.398265 29287 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.318558 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:28.381455 29743 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:28.477331 23461 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:28.478364 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 159910us
W20260709 13:14:28.478981 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74605.1us
I20260709 13:14:28.479314 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:28.447777 22907 raft_consensus.cc:614] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:28.448916 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 138447us
I20260709 13:14:28.481485 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:14:28.481422 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.416684 29637 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.483495 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:28.476991 22911 rpcz_store.cc:269] 0709 13:14:26.823359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.823523 (+   164us) service_pool.cc:224] Handling call
0709 13:14:28.405239 (+1581716us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54003381d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:28.405395 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296050560}
W20260709 13:14:28.451740 29696 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.002s
I20260709 13:14:28.451120 29700 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:28.484122 22906 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:28.411653 29737 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.457623 29390 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.478575 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.487588 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:28.450788 29180 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:28.489899 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:28.573884 29738 log.cc:927] Time spent T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:14:28.474277 29581 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:28.486336 29542 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.483108 29743 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:28.454535 29606 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.484982 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.636161 22911 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:28.489749 29287 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.490131 29596 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:28.574761 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.577456 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:28.586030 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:28.578126 29685 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.938s	user 0.014s	sys 0.000s
I20260709 13:14:28.463009 29409 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.637874 22906 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:28.629113 29743 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:28.638788 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 160296us
I20260709 13:14:28.490939 29653 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:28.491750 29746 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:28.578688 29624 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.632901 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:28.646615 22833 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:28.633114 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47271.4us
W20260709 13:14:28.649617 29730 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:14:28.576879 29613 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.650624 22909 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:28.653208 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.492944 29637 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.497330 29594 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.575263 29741 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.658577 29287 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:28.655794 29653 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.630399 29664 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.675622 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2193) took 1022 ms. Trace:
I20260709 13:14:28.655061 22911 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
W20260709 13:14:28.679211 29747 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:14:28.641713 29596 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.578204 29581 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.682327 29743 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:28.684092 29671 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.004s
I20260709 13:14:28.734819 29616 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:28.708567 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.723816 29608 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.731474 29596 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:28.741654 29743 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.711757 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:28.744837 29670 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:28.745847 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:28.746536 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:28.747175 22841 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:28.747228 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:28.749117 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2645) took 1084 ms. Trace:
I20260709 13:14:28.750010 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1185) took 1174 ms. Trace:
I20260709 13:14:28.750420 22839 rpcz_store.cc:276] 0709 13:14:27.575939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.576093 (+   154us) service_pool.cc:224] Handling call
0709 13:14:27.576570 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.577127 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.577154 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.616508 (+ 39354us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.111030 (+494522us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:28.111230 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.111316 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.749766 (+638450us) raft_consensus.cc:1685] finished
0709 13:14:28.749790 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.749947 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1214,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":12658,"thread_start_us":438500,"threads_started":1,"wal-append.queue_time_us":662069}
I20260709 13:14:28.655611 29699 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.711169 22732 leader_election.cc:400] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 13 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14.
W20260709 13:14:28.753742 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 65313.7us
I20260709 13:14:28.754770 29586 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.755193 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2647) took 1089 ms. Trace:
I20260709 13:14:28.755298 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.755620 22837 rpcz_store.cc:276] 0709 13:14:27.665733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.754335 (+1088602us) service_pool.cc:224] Handling call
0709 13:14:28.754646 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.754883 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.754907 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.754948 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:28.754979 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.755041 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.755140 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:28.757112 29679 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.757529 29737 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.758742 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1014) took 1129 ms. Trace:
I20260709 13:14:28.759040 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.759123 22765 rpcz_store.cc:276] 0709 13:14:27.629810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.629963 (+   153us) service_pool.cc:224] Handling call
0709 13:14:27.921355 (+291392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.921807 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.921834 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:27.921882 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:27.921916 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.921978 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.758573 (+836595us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540035f050. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:28.758680 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600224128}
I20260709 13:14:28.760375 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.760636 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:28.761281 22911 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:28.761669 29644 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.762293 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:28.763224 29671 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:28.763507 29662 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:28.763909 29671 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:28.690251 29676 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:28.713780 22834 rpcz_store.cc:276] 0709 13:14:27.653527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.658508 (+  4981us) service_pool.cc:224] Handling call
0709 13:14:27.658828 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.700466 (+ 41638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.700516 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.923150 (+222634us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b2c00000e98. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:27.923712 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.245222 (+321510us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:28.245279 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.245362 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.662986 (+417624us) raft_consensus.cc:1685] finished
0709 13:14:28.663013 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.675531 (+ 12518us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54842,"raft.queue_time_us":41774,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12403,"spinlock_wait_cycles":4818560,"thread_start_us":321043,"threads_started":1,"wal-append.queue_time_us":532616}
I20260709 13:14:28.768436 22834 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:28.769001 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 991) took 1853 ms. Trace:
I20260709 13:14:28.769670 22834 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:28.726060 29180 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.770629 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2642) took 1106 ms. Trace:
I20260709 13:14:28.770962 22834 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:28.493079 29658 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.663939 29746 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:28.723471 29664 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:28.731462 29745 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:14:28.719691 29653 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:28.575830 29696 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.990767 29742 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.991391 29742 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.992848 29696 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:28.752933 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.770989 22835 rpcz_store.cc:276] 0709 13:14:27.664248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.864881 (+200633us) service_pool.cc:224] Handling call
0709 13:14:27.865732 (+   851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:28.178362 (+312630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.770385 (+592023us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:28.770573 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588777216}
I20260709 13:14:28.999028 22835 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:28.757189 22837 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:29.000379 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2648) took 1334 ms. Trace:
I20260709 13:14:28.769373 22915 rpcz_store.cc:276] 0709 13:14:26.915428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.288346 (+372918us) service_pool.cc:224] Handling call
0709 13:14:27.569693 (+281347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.572125 (+  2432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.768825 (+1196700us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54000cfb50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:28.768946 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646113152}
I20260709 13:14:28.774317 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:29.000748 22837 rpcz_store.cc:276] 0709 13:14:27.666080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.756653 (+1090573us) service_pool.cc:224] Handling call
0709 13:14:28.756976 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.000132 (+243156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.000322 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:29.002125 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 240667us
W20260709 13:14:28.898722 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 143488us
I20260709 13:14:29.004163 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:29.004426 22907 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:28.710071 29624 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:28.736464 29589 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:29.004163 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:29.005282 29589 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.007195 29662 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:29.007781 29662 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:29.010213 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:28.736887 29677 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:29.018661 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2760) took 2073 ms. Trace:
I20260709 13:14:29.035866 22763 rpcz_store.cc:276] 0709 13:14:26.945692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.034377 (+ 88685us) service_pool.cc:224] Handling call
0709 13:14:29.018310 (+1983933us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400311854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:29.018544 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671129984}
I20260709 13:14:28.906095 22841 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:29.038477 22836 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:28.908228 29519 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:28.908977 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:28.907269 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 831) took 1409 ms. Trace:
I20260709 13:14:28.910838 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:28.912221 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:28.913780 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:29.040746 23465 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:28.749488 22833 rpcz_store.cc:276] 0709 13:14:27.665012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.748235 (+1083223us) service_pool.cc:224] Handling call
0709 13:14:28.748581 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.748777 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.748803 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.748849 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:28.748888 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.748947 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.749059 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:28.642473 29606 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.044524 22732 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.915488 29746 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:28.917815 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:29.080096 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 417493us
I20260709 13:14:28.732245 29581 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:28.994024 29742 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.001029 29653 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.003383 29670 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:28.656742 29685 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:28.907397 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.056751 29676 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.039160 23460 rpcz_store.cc:276] 0709 13:14:27.497379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.497540 (+   161us) service_pool.cc:224] Handling call
0709 13:14:28.777132 (+1279592us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54006112d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:28.907123 (+129991us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569245440}
I20260709 13:14:29.048187 29616 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:29.060973 29752 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:29.072757 23465 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:29.076937 22909 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:29.046262 22732 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:28.753365 29699 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:28.902859 29737 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:29.086203 23462 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:14:29.087703 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 330216us
I20260709 13:14:28.721063 29741 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.030089 29743 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:28.813032 29596 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.035372 29754 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:29.007587 29589 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.050259 29624 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:29.096235 22906 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:29.012938 29664 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:29.012471 29671 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:29.048914 29644 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:29.109386 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:29.218161 22760 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:29.090251 22729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.000s	sys 0.018s
W20260709 13:14:29.219086 22729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.000s	sys 0.018s
I20260709 13:14:29.111937 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:29.112082 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:29.220496 22841 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:29.152601 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:29.222086 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69626us
I20260709 13:14:29.223289 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:29.224387 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:29.157169 29653 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.158314 23465 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:29.054185 29499 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:29.170120 22839 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:29.166956 29483 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.859s	user 0.004s	sys 0.010s
I20260709 13:14:29.066421 29519 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:29.180155 22914 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 988) took 2256 ms (client timeout 1810 ms). Trace:
I20260709 13:14:29.171936 29743 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.183408 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3311) took 1071 ms. Trace:
I20260709 13:14:29.208029 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:29.083041 29677 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:14:29.138983 29642 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.289448 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.294292 29519 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.139350 29751 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:14:29.297940 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:29.300926 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2641) took 1637 ms. Trace:
I20260709 13:14:29.301363 22840 rpcz_store.cc:276] 0709 13:14:27.663984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.816999 (+153015us) service_pool.cc:224] Handling call
0709 13:14:27.850261 (+ 33262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.309086 (+458825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.300688 (+991602us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54000c76d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:29.300867 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943942912}
I20260709 13:14:29.303370 22840 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:14:29.209106 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:29.185268 29712 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.612s	user 0.004s	sys 0.023s
I20260709 13:14:29.234639 22833 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:29.173712 29588 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:29.208192 29714 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:29.278904 22835 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:29.278393 22764 rpcz_store.cc:276] 0709 13:14:28.112151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.629046 (+516895us) service_pool.cc:224] Handling call
0709 13:14:28.630071 (+  1025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.091310 (+461239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.091359 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:29.091422 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:29.091479 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.183056 (+ 91577us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:29.183131 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.183327 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060151552}
I20260709 13:14:29.284869 29637 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.285393 29658 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:29.322379 29732 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.048s	user 0.005s	sys 0.012s
I20260709 13:14:29.287856 29677 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:29.267674 29608 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:29.266991 29742 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:29.323423 29742 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.322937 29732 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.323521 29608 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:29.323877 29742 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:29.324615 29608 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:29.208817 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.304924 29596 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.306237 29321 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:29.326123 29321 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:29.326855 23466 raft_consensus.cc:2368] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:14:29.327395 29664 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:29.307140 29685 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.327961 29664 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.141264 29676 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.263459 29671 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.330284 29676 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:29.278162 22914 rpcz_store.cc:269] 0709 13:14:26.914568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.962452 (+ 47884us) service_pool.cc:224] Handling call
0709 13:14:29.170639 (+2208187us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54002107d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:29.170765 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333425024}
I20260709 13:14:29.331301 29671 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.331892 29671 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.332517 29671 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.277971 29483 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:29.280532 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 234322us
I20260709 13:14:29.315662 29752 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:29.268085 29660 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:29.158593 29616 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.319298 29588 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.320215 29714 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:29.286876 29743 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.327080 29321 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.216856 29644 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:29.335428 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:29.270812 29613 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:29.112819 29662 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.313721 22765 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:29.322947 29677 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:29.162074 29762 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.198s
W20260709 13:14:29.368304 22913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3544) took 2059 ms (client timeout 1929 ms). Trace:
I20260709 13:14:29.342458 29660 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:29.404246 22913 rpcz_store.cc:269] 0709 13:14:27.308918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.405677 (+ 96759us) service_pool.cc:224] Handling call
0709 13:14:29.367981 (+1962304us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b5400041854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:29.368192 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40092416}
I20260709 13:14:29.318327 22841 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
W20260709 13:14:29.359463 29694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.483s	user 0.005s	sys 0.023s
I20260709 13:14:29.372843 22760 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:29.386304 29743 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:29.384904 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 298968us
I20260709 13:14:29.398957 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.399278 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.372807 29644 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:29.411901 22757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.001s	sys 0.022s
I20260709 13:14:29.412401 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:29.421736 22757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.001s	sys 0.022s
I20260709 13:14:29.413090 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.372638 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:29.414729 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 205006us
W20260709 13:14:29.423416 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50891.3us
I20260709 13:14:29.414794 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:29.425350 23460 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:29.415877 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.411711 23462 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:29.351397 29752 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.431432 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:29.361994 29732 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.386313 29613 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:29.420517 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:29.434379 22912 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3532) took 2625 ms (client timeout 1733 ms). Trace:
I20260709 13:14:29.427392 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:29.405781 29321 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:29.418653 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:29.306507 29624 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:29.414546 29694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.539s	user 0.005s	sys 0.023s
I20260709 13:14:29.441359 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:29.437750 29696 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:14:29.442834 29685 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:29.439043 29606 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.443401 29613 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:29.434896 22912 rpcz_store.cc:269] 0709 13:14:26.808690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.809126 (+   436us) service_pool.cc:224] Handling call
0709 13:14:29.434177 (+2625051us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54000cf654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:29.434310 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1573620096}
I20260709 13:14:29.439034 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:29.448546 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:29.450948 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:29.414390 29642 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:29.421099 29743 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:29.456717 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.420090 29676 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.534684 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:29.546438 23458 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:14:29.541646 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:29.502106 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:29.489346 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48164.8us
I20260709 13:14:29.535832 22833 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 23: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:29.548856 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:29.465124 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:29.550823 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:29.551828 22911 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:29.463302 22915 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:14:29.535948 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 97024.8us
W20260709 13:14:29.490530 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.462610 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:29.470031 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:29.483907 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:29.554302 22760 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:29.473213 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:29.435948 29699 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.471930 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:29.541976 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:29.453377 29760 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:14:29.561384 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:29.573365 22760 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:14:29.556490 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 59578.1us
I20260709 13:14:29.562565 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.553369 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:29.575336 22915 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:29.456588 29757 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:29.566099 22913 raft_consensus.cc:2368] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:29.480758 29596 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:29.570382 22764 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:29.579233 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:29.453748 29717 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:29.456627 29750 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:29.574625 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:29.555131 29581 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:29.534360 29758 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:14:29.577131 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:29.537132 29554 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:29.456915 29694 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:29.538076 29739 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:29.566985 29681 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:29.608714 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 135559us
I20260709 13:14:29.455796 29762 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 13:14:29.656234 22835 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:29.453846 29765 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.237s
W20260709 13:14:29.566784 29693 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.103s	user 0.009s	sys 0.004s
I20260709 13:14:29.613525 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:29.618342 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:29.613934 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:29.541145 29606 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:29.674151 29640 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.673771 29693 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.694612 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.694460 29640 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.674763 29670 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:29.696018 29670 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:29.696597 29693 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.696895 29670 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.534643 29660 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:29.613427 22716 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50808.5us
I20260709 13:14:29.580561 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:29.637812 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:29.581336 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:29.452392 29677 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.554661 29743 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:29.645808 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:29.706634 22913 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:14:29.620910 29700 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.094s	user 0.013s	sys 0.000s
W20260709 13:14:29.648279 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 381974us
I20260709 13:14:29.650134 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:29.560120 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:29.660161 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.642537 29750 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:29.661780 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80503.1us
W20260709 13:14:29.640230 29708 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.418s	user 0.013s	sys 0.002s
I20260709 13:14:29.579720 29180 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:29.665701 22764 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:29.700881 29606 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:29.702037 22911 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
W20260709 13:14:29.732510 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.732719 22911 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:29.692229 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:29.534325 29714 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.711603 29660 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:29.708927 23460 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:29.709753 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:29.710868 22914 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:29.567394 29676 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.720350 29670 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:29.704700 29741 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:29.700636 29681 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:29.706254 29765 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:29.652554 29581 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.618062 29685 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:29.698002 29483 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:29.697714 29640 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:29.657013 29739 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.700524 29596 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.705195 22915 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:29.666420 29756 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:14:29.718369 29700 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:29.832370 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 279722us
W20260709 13:14:29.833021 22761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2782) took 1919 ms (client timeout 1992 ms). Trace:
I20260709 13:14:29.733685 29673 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.854697 22913 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:29.845305 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:29.887135 23460 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:29.700665 29694 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.894641 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:29.752226 29693 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:29.898511 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:29.897078 22761 rpcz_store.cc:269] 0709 13:14:27.913741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.920527 (+  6786us) service_pool.cc:224] Handling call
0709 13:14:29.832775 (+1912248us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b540024b8d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:29.832957 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891981568}
I20260709 13:14:29.903350 29640 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:29.903697 29776 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:29.904109 29777 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:29.838692 29653 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:29.906634 23458 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:29.907436 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:29.915336 29581 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:29.919759 29673 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.921030 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2644) took 2256 ms. Trace:
I20260709 13:14:29.922545 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2209) took 2384 ms. Trace:
I20260709 13:14:29.924449 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3567) took 1165 ms. Trace:
I20260709 13:14:30.148502 22906 rpcz_store.cc:276] 0709 13:14:28.759214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.761592 (+  2378us) service_pool.cc:224] Handling call
0709 13:14:28.761880 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.210788 (+448908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.924114 (+713326us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b540010dbd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:29.924369 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768392960}
I20260709 13:14:29.834071 29765 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:29.947254 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.147979 22832 rpcz_store.cc:276] 0709 13:14:27.664748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.200410 (+535662us) service_pool.cc:224] Handling call
0709 13:14:28.200894 (+   484us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:28.341482 (+140588us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:28.453672 (+112190us) log.cc:844] Serialized 14 byte log entry
0709 13:14:28.674895 (+221223us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:28.719968 (+ 45073us) log.cc:844] Serialized 78 byte log entry
0709 13:14:28.720277 (+   309us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:28.720728 (+   451us) log.cc:844] Serialized 14 byte log entry
0709 13:14:28.742280 (+ 21552us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b18004a71c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:29.920160 (+1177880us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b18004a71c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:14:29.920383 (+   223us) log.cc:844] Serialized 14 byte log entry
0709 13:14:29.920636 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.920731 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:29.920782 (+    51us) raft_consensus.cc:1685] finished
0709 13:14:29.920794 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.920966 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1193083,"prepare.queue_time_us":1513175,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":6315,"raft.queue_time_us":61559,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":45746,"spinlock_wait_cycles":168896000,"thread_start_us":145515,"threads_started":2,"wal-append.queue_time_us":779065}
I20260709 13:14:29.832973 29714 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:29.839658 29677 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:29.740549 29664 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:29.904808 22840 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:14:30.163154 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.164203 22840 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:30.165576 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:29.935288 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:29.954804 29608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.000s	sys 0.011s
W20260709 13:14:29.986824 29773 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:14:29.991241 29770 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:14:29.952383 29769 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.391s	user 0.000s	sys 0.001s
I20260709 13:14:30.001609 29693 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:29.974751 29606 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.908799 29596 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:29.950821 29640 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:30.045552 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1219) took 2519 ms (client timeout 1929 ms). Trace:
W20260709 13:14:30.087356 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.021217 29752 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.148130 22910 rpcz_store.cc:276] 0709 13:14:27.538295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.067686 (+529391us) service_pool.cc:224] Handling call
0709 13:14:28.116244 (+ 48558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.344708 (+228464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.922197 (+1577489us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54003c87d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:29.922459 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412415744}
I20260709 13:14:30.033242 29681 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.052909 29748 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:29.959421 29777 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:29.927709 29588 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:30.039942 23466 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 836) took 1755 ms (client timeout 1812 ms). Trace:
I20260709 13:14:30.145103 29776 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.012639 29741 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:14:30.143991 22732 leader_election.cc:400] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
W20260709 13:14:30.036887 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 344758us
I20260709 13:14:30.044647 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1020) took 2410 ms. Trace:
I20260709 13:14:30.167688 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.292447 22761 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:14:30.287811 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 382173us
I20260709 13:14:30.304047 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:30.287948 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 144020us
I20260709 13:14:30.307667 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.308072 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:30.309049 22761 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 13:14:30.287711 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.168336 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.315166 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:30.315480 22832 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:30.317827 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:30.291263 22766 rpcz_store.cc:276] 0709 13:14:27.634275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.634626 (+   351us) service_pool.cc:224] Handling call
0709 13:14:27.634949 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.635231 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:28.014409 (+379178us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.044074 (+2029665us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b18002f4ee0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287
0709 13:14:30.044206 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.044267 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:30.044326 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.044408 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.044555 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26421,"prepare.queue_time_us":557100,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":164232,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7479,"spinlock_wait_cycles":47363840,"thread_start_us":578001,"threads_started":3,"wal-append.queue_time_us":268111,"wal-append.run_cpu_time_us":2592,"wal-append.run_wall_time_us":1270607}
I20260709 13:14:30.318557 22832 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:30.319841 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.320098 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.289602 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3310) took 2177 ms. Trace:
I20260709 13:14:30.289762 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2771) took 2702 ms. Trace:
I20260709 13:14:30.330466 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.329867 22758 rpcz_store.cc:276] 0709 13:14:27.586740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.589235 (+  2495us) service_pool.cc:224] Handling call
0709 13:14:27.589632 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.593015 (+  3383us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:28.771793 (+1178778us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b18003dbdc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:14:29.165109 (+393316us) log.cc:844] Serialized 14 byte log entry
0709 13:14:29.352602 (+187493us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:29.359677 (+  7075us) log.cc:844] Serialized 26 byte log entry
0709 13:14:29.359852 (+   175us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:29.359891 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.359995 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.288846 (+928851us) raft_consensus.cc:1685] finished
0709 13:14:30.289133 (+   287us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.289440 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3837,"prepare.queue_time_us":567282,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":4604,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":33581,"spinlock_wait_cycles":41218816,"thread_start_us":578331,"threads_started":2,"wal-append.queue_time_us":274194}
I20260709 13:14:30.326848 22664 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 5 to 8, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 8 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:30.329778 22757 rpcz_store.cc:276] 0709 13:14:28.111867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.584340 (+472473us) service_pool.cc:224] Handling call
0709 13:14:28.634687 (+ 50347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:28.634956 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.634982 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:28.636215 (+  1233us) log.cc:844] Serialized 26 byte log entry
0709 13:14:29.422271 (+786056us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:29.430216 (+  7945us) log.cc:844] Serialized 14 byte log entry
0709 13:14:29.430447 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.430543 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.288835 (+858292us) raft_consensus.cc:1685] finished
0709 13:14:30.288926 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.289279 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11524,"prepare.run_wall_time_us":46,"raft.queue_time_us":5967,"spinlock_wait_cycles":14670592,"thread_start_us":719092,"threads_started":1,"wal-append.queue_time_us":1034423}
I20260709 13:14:30.333420 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.333884 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.334358 22758 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.335602 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.336586 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.331516 22766 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:30.289459 22655 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 244) took 49776 ms (client timeout 15000 ms). Trace:
I20260709 13:14:30.337100 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:30.184602 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.347630 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.196291 22908 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:30.348971 22766 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:30.193856 23461 rpcz_store.cc:269] 0709 13:14:27.526075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.526367 (+   292us) service_pool.cc:224] Handling call
0709 13:14:30.045306 (+2518939us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b5400093dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.045456 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160002944}
I20260709 13:14:30.350419 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:30.198894 23466 rpcz_store.cc:269] 0709 13:14:28.284605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.305536 (+ 20931us) service_pool.cc:224] Handling call
0709 13:14:30.039612 (+1734076us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54000b5c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.039846 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093883648}
I20260709 13:14:30.358590 22761 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:30.358759 22838 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:30.219084 29692 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:30.359840 29692 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:30.220888 29596 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.360379 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.217784 29409 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:30.361302 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.364816 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.210778 29676 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.366132 22837 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:30.211091 29739 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:30.343222 22655 rpcz_store.cc:269] 0709 13:13:40.512933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.513321 (+   388us) service_pool.cc:224] Handling call
0709 13:14:30.289096 (+49775775us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:13.225557 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:13.226352 (+   795us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:13.226404 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:14:13.226422 (+    18us) tablet.cc:662] Decoding operations
0709 13:14:13.228395 (+  1973us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:13.228499 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:13.228524 (+    25us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:14:13.295367 (+ 66843us) tablet.cc:701] Row locks acquired
0709 13:14:13.295423 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:14:13.295660 (+   237us) write_op.cc:270] Start()
0709 13:14:13.295841 (+   181us) write_op.cc:276] Timestamp: P: 1783602853295581 usec, L: 0
0709 13:14:13.295892 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:14:13.317979 (+ 22087us) log.cc:844] Serialized 7468 byte log entry
0709 13:14:14.524798 (+1206819us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aece28 00007f68a5527581 00007f68a5526d58 00007f68a552b9f1 00007f68a552b9a9 00007f68a552b939 00007f68a552b901 00007f68a552ab2d 000000000038f354 000000000038f199
0709 13:14:15.198589 (+673791us) op_driver.cc:464] REPLICATION: finished
0709 13:14:15.508624 (+310035us) write_op.cc:301] APPLY: starting
0709 13:14:15.508749 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:16.029638 (+520889us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:16.029699 (+    61us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:16.624309 (+594610us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:16.625017 (+   708us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:14:16.625091 (+    74us) write_op.cc:312] APPLY: finished
0709 13:14:16.692339 (+ 67248us) log.cc:844] Serialized 172 byte log entry
0709 13:14:17.782576 (+1090237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:17.816209 (+ 33633us) write_op.cc:454] Released schema lock
0709 13:14:17.852343 (+ 36134us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:17.930235 (+ 77892us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:17.958982 (+ 28747us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1886037,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":309636,"apply.run_cpu_time_us":64428,"apply.run_wall_time_us":2386462,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":358847,"prepare.run_cpu_time_us":51372,"prepare.run_wall_time_us":1368804,"raft.queue_time_us":404959,"raft.run_cpu_time_us":40674,"raft.run_wall_time_us":227900,"replication_time_us":1895142,"spinlock_wait_cycles":7629056,"thread_start_us":2109585,"threads_started":5,"wal-append.queue_time_us":1660014,"wal-append.run_cpu_time_us":48681,"wal-append.run_wall_time_us":3429070}]]}
I20260709 13:14:30.216502 29743 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.368592 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.369728 22766 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.202566 29673 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.370280 22765 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:30.216748 29671 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:30.168370 29653 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.386731 22840 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:30.388119 29653 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:30.375165 29743 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.389475 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.390372 22914 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:30.375336 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:30.376621 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.210330 29693 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.392987 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.393851 22914 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:14:30.211640 29746 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.116s	user 0.011s	sys 0.002s
I20260709 13:14:30.397202 29746 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.398134 29756 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:30.399433 29752 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:30.399911 29757 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:30.400624 29757 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:30.219848 29644 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.006s
I20260709 13:14:30.402235 29644 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.402767 29777 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.402575 29757 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.403620 29777 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:30.404894 29644 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:30.215869 29742 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.002s
I20260709 13:14:30.379364 22906 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:30.406886 29742 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.409175 22915 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:30.211905 29776 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.397156 22664 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 7, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 7 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:30.408032 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2196) took 2194 ms. Trace:
I20260709 13:14:30.197685 29765 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.199272 29748 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.409308 29644 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:30.396065 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.401248 29746 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:30.410091 22761 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:30.413462 29742 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.415784 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.415298 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.418579 29781 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.420588 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.422314 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.422488 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:30.193114 29321 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.001s
W20260709 13:14:30.170254 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.385069 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:30.176589 29608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.001s	sys 0.017s
W20260709 13:14:30.454401 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 139564us
I20260709 13:14:30.400465 29752 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:14:30.450191 29644 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:30.459421 29781 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.460728 29677 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.460546 29644 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.461766 29677 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.455752 29692 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:30.469182 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.462929 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.463122 29681 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:30.470714 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.471268 29681 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.466651 29694 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:30.471447 22664 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 9, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 9 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:30.472631 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.472739 29694 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.473747 22763 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:30.474536 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:30.475190 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.475472 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2654) took 2244 ms. Trace:
I20260709 13:14:30.476161 22763 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:30.477640 29742 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:30.478852 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.479537 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.479669 22763 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:30.482457 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:30.483526 23459 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:30.209031 29640 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.485976 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3566) took 1726 ms. Trace:
I20260709 13:14:30.486727 22907 rpcz_store.cc:276] 0709 13:14:28.758912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.763838 (+  4926us) service_pool.cc:224] Handling call
0709 13:14:28.764109 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.004906 (+240797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.484954 (+1480048us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54000447d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:30.485686 (+   732us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551745280}
I20260709 13:14:30.488803 22915 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:30.465570 29708 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:30.431141 22834 rpcz_store.cc:276] 0709 13:14:28.213998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.771718 (+557720us) service_pool.cc:224] Handling call
0709 13:14:28.772024 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.064688 (+292664us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:29.064741 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:29.086673 (+ 21932us) log.cc:844] Serialized 26 byte log entry
0709 13:14:29.667363 (+580690us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:29.667688 (+   325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.667919 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.407494 (+739575us) raft_consensus.cc:1685] finished
0709 13:14:30.407536 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.407807 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31822,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":257,"thread_start_us":455807,"threads_started":1,"wal-append.queue_time_us":624147}
I20260709 13:14:30.465975 29676 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:30.466327 29671 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.466605 22914 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:30.475973 22836 rpcz_store.cc:276] 0709 13:14:28.231105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.919687 (+688582us) service_pool.cc:224] Handling call
0709 13:14:29.038289 (+118602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.038865 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.475047 (+1436182us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400320fd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.475249 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271543168}
I20260709 13:14:30.469018 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:30.463205 29781 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.477141 23466 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:30.465788 29653 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.470717 22757 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:30.480806 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.498203 22910 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:30.505028 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:30.506016 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.505131 22763 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:30.507848 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.508813 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:30.509958 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:30.510988 22909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3562) took 2007 ms (client timeout 2360 ms). Trace:
W20260709 13:14:30.511489 22909 rpcz_store.cc:269] 0709 13:14:28.481831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.481962 (+   131us) service_pool.cc:224] Handling call
0709 13:14:30.489075 (+2007113us) spinlock_profiling.cc:243] Waited 8.59 ms on lock 0x7b5400125cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.489322 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19762560}
W20260709 13:14:30.513067 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 90806.5us
I20260709 13:14:30.459482 29554 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:14:30.518925 29677 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.519923 29739 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:30.520993 23465 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:14:30.521359 29708 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.522172 29593 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:30.522634 29593 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:30.521018 29676 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.524326 29593 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.525687 29653 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.527172 29776 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.527849 29616 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:30.511891 22655 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 138) took 22252 ms (client timeout 15000 ms). Trace:
W20260709 13:14:30.528808 22655 rpcz_store.cc:269] 0709 13:14:08.259927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.511601 (+22251674us) service_pool.cc:211] Skipping call since client already timed out
0709 13:14:30.511791 (+   190us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:14:30.532512 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:30.533833 29321 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:30.535917 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:30.536506 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3305) took 2324 ms. Trace:
I20260709 13:14:30.510232 29681 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.538571 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:30.538959 29306 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:30.477320 29673 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:30.539955 29673 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.540474 29673 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:30.537014 22762 rpcz_store.cc:276] 0709 13:14:28.108617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.295931 (+187314us) service_pool.cc:224] Handling call
0709 13:14:28.296328 (+   397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:28.306429 (+ 10101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.306470 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:28.319881 (+ 13411us) log.cc:844] Serialized 52 byte log entry
0709 13:14:30.416595 (+2096714us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b18000b4760. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1
0709 13:14:30.417041 (+   446us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:30.417097 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.417201 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.417262 (+    61us) raft_consensus.cc:1685] finished
0709 13:14:30.417282 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.432213 (+ 14931us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9596,"raft.queue_time_us":180438,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":8506,"spinlock_wait_cycles":17683328,"thread_start_us":73652,"threads_started":1,"wal-append.queue_time_us":702564}
I20260709 13:14:30.511011 22911 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:30.541646 29703 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.541718 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.542512 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.518700 29554 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:30.543293 22911 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:30.543469 29554 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:30.547108 22763 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:30.547914 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.548511 29692 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.550218 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.551944 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.552599 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:30.553212 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44080.3us
I20260709 13:14:30.554246 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.554112 29793 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:30.558056 29671 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.559484 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.561075 29321 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:30.562515 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2669) took 1101 ms. Trace:
I20260709 13:14:30.565668 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:30.558077 29593 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:30.566407 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.513321 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.565336 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 1029) took 1134 ms. Trace:
I20260709 13:14:30.542949 22762 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:30.578953 22766 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:30.579161 22758 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:30.579970 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3330) took 1098 ms. Trace:
I20260709 13:14:30.580164 22832 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:30.576668 22765 rpcz_store.cc:276] 0709 13:14:29.431015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.431156 (+   141us) service_pool.cc:224] Handling call
0709 13:14:30.564852 (+1133696us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b540035eb54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.565074 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155810048}
I20260709 13:14:30.575713 22838 rpcz_store.cc:276] 0709 13:14:29.460734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.579029 (+118295us) service_pool.cc:224] Handling call
0709 13:14:30.561333 (+982304us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54003228d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.562193 (+   860us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502190848}
I20260709 13:14:30.496382 29694 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.580425 22758 rpcz_store.cc:276] 0709 13:14:29.481821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.332942 (+851121us) service_pool.cc:224] Handling call
0709 13:14:30.579881 (+246939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.585189 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:30.593144 22836 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:30.594738 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2661) took 1145 ms. Trace:
I20260709 13:14:30.595284 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.597954 22914 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:30.600363 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:30.601652 22912 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:30.603027 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.603856 22912 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:30.603699 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:30.604890 23459 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:30.604799 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:30.606155 29793 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:30.606588 29793 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:30.607843 29671 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:30.607748 29793 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.608949 22909 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.612502 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.613986 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:30.615198 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.615712 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3323) took 1143 ms. Trace:
I20260709 13:14:30.616142 22762 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:30.616216 22765 rpcz_store.cc:276] 0709 13:14:29.472200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.589979 (+1117779us) service_pool.cc:224] Handling call
0709 13:14:30.590510 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.590769 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.590802 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.590843 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:30.615414 (+ 24571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.615499 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.615630 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.617815 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:30.618757 22832 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:30.622602 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:30.595320 22835 rpcz_store.cc:276] 0709 13:14:29.449638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.449781 (+   143us) service_pool.cc:224] Handling call
0709 13:14:29.537326 (+ 87545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.656855 (+119529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.594097 (+937242us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540028a5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:30.594511 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272744192}
I20260709 13:14:30.623631 22765 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:30.618512 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.626181 22912 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:14:30.626601 22762 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 22 index: 9. (index mismatch)
I20260709 13:14:30.627772 29671 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:30.630077 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.632064 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.633121 22835 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:30.645643 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3332) took 1084 ms. Trace:
I20260709 13:14:30.646433 29717 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:30.646633 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.648725 22765 rpcz_store.cc:276] 0709 13:14:29.540471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.617430 (+1076959us) service_pool.cc:224] Handling call
0709 13:14:30.617843 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.624043 (+  6200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.625113 (+  1070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.647092 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.648377 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2668) took 1189 ms. Trace:
I20260709 13:14:30.651486 22838 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:30.651058 22835 rpcz_store.cc:276] 0709 13:14:29.458602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.631554 (+1172952us) service_pool.cc:224] Handling call
0709 13:14:30.648304 (+ 16750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.651705 22911 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:30.650188 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3331) took 1145 ms. Trace:
I20260709 13:14:30.651556 22832 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:30.652668 22762 rpcz_store.cc:276] 0709 13:14:29.482213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.617162 (+1134949us) service_pool.cc:224] Handling call
0709 13:14:30.621040 (+  3878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.626901 (+  5861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.627061 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.654006 22757 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:30.612071 29653 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:30.656605 22761 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:30.653884 29776 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:30.658016 22762 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:30.658375 29732 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:30.658793 29732 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:30.659785 22762 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:30.661921 22909 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:30.662667 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3326) took 1184 ms. Trace:
W20260709 13:14:30.663625 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60155.8us
I20260709 13:14:30.664902 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3317) took 1191 ms. Trace:
I20260709 13:14:30.663267 22761 rpcz_store.cc:276] 0709 13:14:29.473118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.376118 (+903000us) service_pool.cc:224] Handling call
0709 13:14:30.657017 (+280899us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.665357 22766 rpcz_store.cc:276] 0709 13:14:29.470288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.584766 (+1114478us) service_pool.cc:224] Handling call
0709 13:14:30.649800 (+ 65034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.654723 (+  4923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.654749 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.654786 (+    37us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:30.661482 (+  6696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.661593 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.661713 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.669432 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.673017 29750 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.651780 29676 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.655607 29717 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:30.678907 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.683318 29708 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:30.681787 22836 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:30.684788 22914 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
W20260709 13:14:30.689443 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 123954us
I20260709 13:14:30.689648 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:30.690610 23463 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:30.690941 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:30.693637 22906 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:30.694285 22911 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:30.698812 22757 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:30.702297 22757 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:14:30.702847 22757 pending_rounds.cc:85] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:30.702679 22838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "6545453e74344bb2957dd3729c5b3870"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:14:30.703236 22757 pending_rounds.cc:107] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:14:30.703299 29751 raft_consensus.cc:2764] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:14:30.703537 22838 raft_consensus.cc:606] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Received request to transfer leadership to TS 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:30.703856 22757 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.704020 22838 raft_consensus.cc:614] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:30.704293 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:30.705160 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:30.701326 22840 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:30.707542 22766 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:30.710774 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.714882 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 834) took 2439 ms. Trace:
I20260709 13:14:30.715025 29751 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:30.715520 23464 rpcz_store.cc:276] 0709 13:14:28.275352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.276474 (+  1122us) service_pool.cc:224] Handling call
0709 13:14:28.276805 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.399383 (+122578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.399413 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.399457 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:28.399486 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.714339 (+2314853us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400236254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:30.714434 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.714650 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281376128}
I20260709 13:14:30.714802 22833 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:14:30.659847 29732 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.728582 22765 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:30.680053 29673 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.724207 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2667) took 1254 ms. Trace:
I20260709 13:14:30.742440 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3333) took 1169 ms. Trace:
I20260709 13:14:30.745711 22765 rpcz_store.cc:276] 0709 13:14:29.573618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.650184 (+1076566us) service_pool.cc:224] Handling call
0709 13:14:30.694811 (+ 44627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.729029 (+ 34218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.742371 (+ 13342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.737426 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:30.742645 22840 rpcz_store.cc:276] 0709 13:14:29.458270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.700447 (+1242177us) service_pool.cc:224] Handling call
0709 13:14:30.700991 (+   544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.712152 (+ 11161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.712300 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.755434 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.755831 22838 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:30.754779 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2777) took 3019 ms. Trace:
I20260709 13:14:30.757579 22760 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.757598 22759 rpcz_store.cc:276] 0709 13:14:27.733634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.733835 (+   201us) service_pool.cc:224] Handling call
0709 13:14:27.737010 (+  3175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.271619 (+534609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.749031 (+2477412us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b540002dd54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:30.752773 (+  3742us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173686656}
I20260709 13:14:30.754021 22732 leader_election.cc:400] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16.
I20260709 13:14:30.752969 29554 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:30.763947 29554 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:30.748582 29673 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.770098 22832 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:14:30.773654 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.775282 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.777143 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.778143 22838 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:30.778911 29777 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:30.780565 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2646) took 3115 ms. Trace:
W20260709 13:14:30.780819 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.783027 22764 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.027s	user 0.010s	sys 0.016s
W20260709 13:14:30.780604 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75734.8us
I20260709 13:14:30.790908 22839 rpcz_store.cc:276] 0709 13:14:27.665263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.752156 (+1086893us) service_pool.cc:224] Handling call
0709 13:14:28.752449 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.268908 (+516459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.780088 (+1511180us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54000ec4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:30.780354 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958697856}
I20260709 13:14:30.793013 22760 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:14:30.795707 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 58646.7us
I20260709 13:14:30.797559 22764 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:30.798224 22839 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:30.790668 29777 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:30.799752 29777 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:30.800727 22839 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:30.802680 29768 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.803815 29768 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:30.805852 29775 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.045s	user 0.006s	sys 0.011s
I20260709 13:14:30.805783 22915 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 22 index: 9. (term mismatch)
I20260709 13:14:30.806574 22915 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 8
I20260709 13:14:30.806923 22915 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 21.9
I20260709 13:14:30.807231 22915 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.810343 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2666) took 1352 ms. Trace:
I20260709 13:14:30.798190 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3329) took 1316 ms. Trace:
I20260709 13:14:30.805883 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:30.813526 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.813912 29732 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:30.811985 22764 rpcz_store.cc:276] 0709 13:14:29.481538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.613677 (+132139us) service_pool.cc:224] Handling call
0709 13:14:30.798104 (+1184427us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:30.818292 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 130424us
I20260709 13:14:30.818799 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:30.819455 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:30.820595 29593 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:30.821141 29593 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:30.821761 29593 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:30.823814 29777 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:30.811061 22835 rpcz_store.cc:276] 0709 13:14:29.457944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.712642 (+1254698us) service_pool.cc:224] Handling call
0709 13:14:30.768621 (+ 55979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.809857 (+ 41236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.809891 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.809985 (+    94us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:30.810020 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.810096 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.810241 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.826987 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.807111 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:30.829051 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.821199 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:30.842290 29748 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:30.858223 29798 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:14:30.858701 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:30.870158 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.862032 29673 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:30.851925 29554 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:14:30.883221 29714 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.004s
I20260709 13:14:30.884662 23462 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:30.889547 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2239) took 1423 ms. Trace:
I20260709 13:14:30.890040 22908 rpcz_store.cc:276] 0709 13:14:29.466226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.559771 (+ 93545us) service_pool.cc:224] Handling call
0709 13:14:30.888902 (+1329131us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54003c9bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:30.889297 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080743168}
I20260709 13:14:30.894591 22838 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:30.856161 29743 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:30.891192 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.862154 29692 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:30.881071 29765 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:30.907917 22835 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:30.910522 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57414.7us
W20260709 13:14:30.910851 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 103873us
I20260709 13:14:30.914000 23464 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:14:30.914578 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:30.917768 29774 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:30.918211 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.919436 22764 raft_consensus.cc:2410] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:14:30.909379 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2664) took 1458 ms. Trace:
W20260709 13:14:30.925462 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104345us
I20260709 13:14:30.934046 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:30.908252 29692 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.933598 22838 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:30.920866 29774 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
W20260709 13:14:30.939370 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.942463 29593 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:30.944684 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43987.2us
W20260709 13:14:30.945799 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.967077 22835 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:30.945695 29711 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:30.937767 29692 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:30.973246 22763 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:30.975313 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.977654 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:30.980007 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:30.937925 29644 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:14:30.988143 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:30.977864 29711 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:30.997483 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3318) took 1523 ms. Trace:
I20260709 13:14:30.998993 22758 rpcz_store.cc:276] 0709 13:14:29.470602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.592571 (+1121969us) service_pool.cc:224] Handling call
0709 13:14:30.626129 (+ 33558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.665980 (+ 39851us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:30.865194 (+199214us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.993248 (+128054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.993315 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:30.993364 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.993444 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.993622 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35798,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":3595,"raft.queue_time_us":3452,"thread_start_us":193981,"threads_started":2}
I20260709 13:14:31.002764 29686 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.003684 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.003523 29686 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.004534 22764 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.996922 29711 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:30.996508 29306 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:31.018124 29306 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.018736 29653 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:31.019270 29653 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.019627 29653 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:31.020265 22759 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:30.993484 29768 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:31.009941 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45908.8us
I20260709 13:14:31.033622 29132 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.034488 29132 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.033360 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.038434 29671 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:31.039723 29306 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:30.989614 29717 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.047456 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.025231 29751 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.037261 29132 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:31.056398 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3320) took 1585 ms (client timeout 1933 ms). Trace:
I20260709 13:14:31.058907 29751 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:31.059809 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.060755 29704 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:31.061157 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 965) took 1167 ms (client timeout 1540 ms). Trace:
W20260709 13:14:31.064739 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.066253 22761 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:31.067494 29802 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:14:31.068697 29702 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:31.061681 23462 rpcz_store.cc:269] 0709 13:14:29.893343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.893558 (+   215us) service_pool.cc:224] Handling call
0709 13:14:31.060926 (+1167368us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54002c8154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:31.061068 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2251172224}
I20260709 13:14:31.061486 29704 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:31.057096 22760 rpcz_store.cc:269] 0709 13:14:29.471248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.574272 (+103024us) service_pool.cc:224] Handling call
0709 13:14:31.056150 (+1481878us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001f95d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:31.056323 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2578415360}
I20260709 13:14:31.069950 29702 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.084831 29754 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:31.085440 29704 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:31.062521 29686 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:31.086598 29702 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:30.922190 22836 rpcz_store.cc:276] 0709 13:14:29.450562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.724983 (+1274421us) service_pool.cc:224] Handling call
0709 13:14:30.781840 (+ 56857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.782146 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:14:30.832749 (+ 50603us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.908954 (+ 76205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.909037 (+    83us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:30.909093 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.909173 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.909312 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11953,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":1563,"thread_start_us":119950,"threads_started":2}
W20260709 13:14:31.099781 23432 leader_election.cc:336] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.565s (SENT)
I20260709 13:14:31.080154 29593 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.085489 29754 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:31.105595 29751 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.100399 29717 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:31.127424 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64984.9us
I20260709 13:14:31.130296 29776 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:14:31.136080 29776 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:31.136518 29776 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:31.123301 29754 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:31.156453 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:31.162362 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68374.2us
I20260709 13:14:31.161739 22764 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:31.165832 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:31.166581 29795 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:31.164091 29774 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.168263 29774 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.173099 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2792) took 1200 ms. Trace:
I20260709 13:14:31.173921 22761 rpcz_store.cc:276] 0709 13:14:29.973178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.666728 (+693550us) service_pool.cc:224] Handling call
0709 13:14:31.062782 (+396054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.066820 (+  4038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.172871 (+106051us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001f95d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:31.173033 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879109888}
I20260709 13:14:31.150380 29751 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:14:31.139529 29776 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.183266 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:31.184046 22841 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:31.162292 29686 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:14:31.187744 29132 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:31.188903 29795 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:31.195932 29132 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:31.196538 29132 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:31.197974 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:31.206523 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2202) took 2094 ms. Trace:
I20260709 13:14:31.207180 22837 rpcz_store.cc:276] 0709 13:14:29.112095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.112598 (+   503us) service_pool.cc:224] Handling call
0709 13:14:29.156170 (+ 43572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.366706 (+1210536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.202552 (+835846us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000a5854. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:31.206370 (+  3818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2781148544}
W20260709 13:14:31.210726 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53057.4us
I20260709 13:14:31.214082 29774 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:31.221177 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.222514 29795 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:31.223060 29795 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:31.224105 29795 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.226053 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:31.187208 29751 consensus_queue.cc:1243] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:31.234318 29671 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:31.237166 29306 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:31.253769 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.240422 29708 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.241575 29793 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:31.260260 29708 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.262909 29708 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:31.251361 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.231622 22759 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:31.211095 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.240351 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:31.255715 29671 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:31.265292 29708 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:31.277566 29306 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:31.279994 29696 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:31.281208 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.282671 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:31.282846 29754 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.285724 29708 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:31.285912 29306 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:31.286265 29708 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:31.286458 29306 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:31.287019 29306 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:31.275928 29774 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:14:31.255887 29686 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:31.289321 29686 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.282671 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.273867 29777 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:31.298287 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:31.269645 29776 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:31.278568 29653 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.307157 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:31.312247 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 102881us
I20260709 13:14:31.210853 29132 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.327440 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.327719 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:31.322026 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.310678 29748 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:31.322403 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45120.8us
I20260709 13:14:31.330821 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:31.333832 29774 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:31.334424 29670 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:31.335054 29670 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:31.335158 29692 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.336135 29670 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.291723 29711 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.337477 23460 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:31.338461 29725 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:31.339345 29711 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:31.332459 23463 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:31.334470 29774 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:31.331465 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:31.343626 23462 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:31.343927 22833 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:31.348212 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:31.340097 29711 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:31.356191 22836 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:31.331156 29593 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:31.331023 29743 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:31.358635 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:31.340945 29748 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:31.320537 29754 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:31.321815 29717 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:31.296665 29671 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.355252 29711 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.369094 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.356999 29692 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:31.379220 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:31.380836 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.362421 29725 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:31.382899 22912 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:31.381625 22841 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:31.366254 29748 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:31.369067 29670 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:31.392156 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:31.393366 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:31.392939 29774 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.394160 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.396140 22910 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:31.396845 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:31.415120 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.415586 22911 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:31.368726 29673 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:14:31.397689 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:31.398389 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.399583 22913 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:14:31.401650 29810 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:14:31.402135 29751 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:31.412282 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.379628 29711 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:31.425864 22914 raft_consensus.cc:2376] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23. Re-sending same reply.
I20260709 13:14:31.426985 29732 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:31.364136 29676 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:14:31.371446 29704 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:14:31.430946 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:31.417220 29673 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:31.438844 29734 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:31.439692 29719 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:14:31.417567 29743 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:14:31.384459 29725 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:31.446491 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.446859 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.493747 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:31.494483 22760 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:31.447474 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.448307 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.496368 22761 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:31.496893 22759 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:31.449584 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.498404 22763 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:31.453827 22911 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:31.457365 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:31.420068 29765 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.009s
I20260709 13:14:31.513993 22759 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:31.388207 29686 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:31.516978 22765 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:14:31.517453 22765 pending_rounds.cc:85] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 14
I20260709 13:14:31.517738 22765 pending_rounds.cc:107] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:14:31.517977 22765 raft_consensus.cc:2889] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:31.390532 29593 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:31.473845 22907 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:14:31.475435 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.477929 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.524560 22766 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:31.525117 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.525928 22910 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17. Re-sending same reply.
I20260709 13:14:31.478847 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.479604 22762 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:31.500367 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.519510 23460 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:31.519217 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.522557 22912 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:31.529935 22757 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:31.529732 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.489966 29676 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:31.485209 29734 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:31.502136 29671 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:31.538319 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.523079 22906 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:31.542536 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.499426 29725 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:31.543829 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 260108us
I20260709 13:14:31.545222 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:31.572614 22911 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17. Re-sending same reply.
W20260709 13:14:31.582846 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.587523 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.570761 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.592972 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.594317 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.531795 29743 consensus_queue.cc:1243] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:31.597797 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:31.598699 22906 raft_consensus.cc:2393] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:31.599543 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.542294 29734 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:31.606525 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 228213us
I20260709 13:14:31.602818 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:31.534420 29676 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:31.605187 22763 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:14:31.624874 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54235.2us
I20260709 13:14:31.623294 22910 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:31.625929 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.626595 22765 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:31.625253 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 259111us
I20260709 13:14:31.629616 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.603531 29653 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:31.571218 29593 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:31.633128 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 291206us
I20260709 13:14:31.637368 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:31.638494 22907 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:31.642371 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 203215us
I20260709 13:14:31.643628 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.643534 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:31.655747 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 295258us
I20260709 13:14:31.657106 22764 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:31.632396 29765 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:31.634394 29754 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:31.662642 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:31.636181 29777 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:31.647766 29686 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:31.674801 29686 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:31.675302 29686 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:31.675225 29777 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.681486 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.684466 23431 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:31.685899 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.648852 29692 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:31.689208 22763 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:31.641688 29644 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:14:31.665786 29653 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:31.698280 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.686338 29768 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.644662 29816 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:14:31.715974 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.659266 29813 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:14:31.718726 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.686659 29754 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.723451 29754 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.724969 22760 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
W20260709 13:14:31.709841 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 278953us
W20260709 13:14:31.717790 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58273.3us
I20260709 13:14:31.708062 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:31.728127 29754 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:31.704311 29653 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.741242 29653 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:31.728190 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 95484us
I20260709 13:14:31.735357 29812 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.735464 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:31.736673 29826 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:31.752074 29818 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:31.753955 29832 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:31.724669 29644 consensus_queue.cc:1243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:14:31.738245 29811 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:14:31.746161 29754 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:31.755906 29754 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:31.756752 29828 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:31.738865 29819 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:31.758505 29827 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:31.758899 29827 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:31.689893 29696 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:14:31.751725 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.763278 22762 raft_consensus.cc:2376] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19. Re-sending same reply.
I20260709 13:14:31.764813 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.768723 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.770361 29593 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:31.771065 29132 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:31.772348 29692 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:31.772922 29692 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:31.773074 29823 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.774133 29732 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:31.773787 29823 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.776322 29823 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:31.774513 29732 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:31.778048 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.778889 29732 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.779417 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.780608 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.689324 29765 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.762706 22841 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:14:31.783581 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.803488 22760 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:31.785653 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.004s (SENT) [suppressed 45 similar messages]
I20260709 13:14:31.729725 29594 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:31.813989 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.815068 22762 raft_consensus.cc:2376] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19. Re-sending same reply.
I20260709 13:14:31.810591 22757 raft_consensus.cc:2376] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15. Re-sending same reply.
W20260709 13:14:31.819842 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 176402us
I20260709 13:14:31.823877 23464 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:14:31.825851 29743 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:31.827682 29673 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:31.821673 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 159139us
I20260709 13:14:31.829700 29732 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:31.832731 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:31.834084 29776 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.008s
I20260709 13:14:31.836509 29823 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:31.837291 29823 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:31.838282 29823 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.841888 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.842760 22760 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:31.843945 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.844871 22760 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:31.845973 22758 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:14:31.846866 29306 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.003s
I20260709 13:14:31.848969 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:31.749828 29719 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:31.855017 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:31.855772 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 129018us
I20260709 13:14:31.856592 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:31.858306 29717 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:31.858718 29717 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:31.859771 29708 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.007s
I20260709 13:14:31.858175 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.860590 22765 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:31.782963 29816 consensus_queue.cc:1243] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:31.865621 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:31.725014 29768 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.840361 29686 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.871148 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.874349 22760 raft_consensus.cc:2376] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23. Re-sending same reply.
I20260709 13:14:31.869002 22912 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:31.872969 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:31.861644 29719 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:31.878386 29719 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.878376 22757 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:31.884274 22757 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:14:31.884713 22757 pending_rounds.cc:85] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 11
I20260709 13:14:31.884981 22757 pending_rounds.cc:107] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:14:31.866998 29751 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:14:31.886098 22765 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:31.759815 29827 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.886485 22765 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:31.886869 29827 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:31.887514 22765 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.889838 22765 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:31.812227 29812 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.896467 29812 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.900424 29711 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.897106 29754 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.906502 29686 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.907662 29671 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:14:31.908130 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92426.7us
I20260709 13:14:31.885249 22757 raft_consensus.cc:2889] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:31.911485 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 59002.6us
W20260709 13:14:31.932845 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 101709us
I20260709 13:14:31.912092 29828 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:31.913033 29704 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:31.914325 29831 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:31.944916 22765 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
W20260709 13:14:31.924666 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59373.5us
I20260709 13:14:31.863771 29819 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:31.946540 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.871958 29813 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:31.893194 29734 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.949148 29813 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:31.951442 29734 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.950891 29813 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.953418 29813 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:31.956490 29734 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:31.946024 22765 pending_rounds.cc:85] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 10
I20260709 13:14:31.724707 29676 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.897059 29812 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:31.959017 22765 pending_rounds.cc:107] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:14:31.959378 22765 raft_consensus.cc:2889] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:31.847368 29306 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.936386 29837 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:31.960978 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.859701 29717 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:31.931277 29594 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:31.940220 29704 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:31.870769 29708 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:31.979482 22911 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:31.970853 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:31.961401 29837 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:31.985622 22759 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:31.968602 29820 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.974139 29686 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:32.007814 29686 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:32.008340 29686 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:31.987403 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:32.011083 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:32.013063 29554 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:32.013489 29554 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:31.974566 29751 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:14:31.990114 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.968215 29653 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:31.993824 29670 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.005s
I20260709 13:14:31.995776 29768 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:31.981845 29812 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:32.020278 29812 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:32.020723 29812 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:32.020773 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:32.022022 29306 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.988355 29837 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:32.032426 29708 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:31.981439 29819 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:31.985689 29795 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 13:14:32.018303 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.019735 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.022658 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.022776 29594 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.023471 29732 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:32.023236 29725 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.021294 29653 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:32.018929 29704 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.028137 29711 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.028220 29839 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:31.989270 29717 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:32.013871 29554 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:32.002100 29820 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.073808 29820 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.015584 29751 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:32.075505 29751 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:32.038250 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.055761 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 110027us
I20260709 13:14:32.001102 29692 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.080092 29692 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:32.077464 29751 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:32.058144 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.084576 22912 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:32.018275 29748 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.598s	user 0.005s	sys 0.008s
W20260709 13:14:32.063314 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 206811us
I20260709 13:14:32.089169 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.057121 29708 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.096473 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:32.061894 29843 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:32.108175 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 95487us
I20260709 13:14:32.100754 29839 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:32.095568 29748 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.118867 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.125837 29846 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:32.108217 29841 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:32.109027 29732 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:14:32.138639 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.103855 29708 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.149220 29846 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:32.101193 29554 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.150990 22836 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:32.150624 29846 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.153139 22761 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:32.121181 29704 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:32.162156 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 65801.4us
I20260709 13:14:32.163018 23432 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:32.165566 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50152.1us
I20260709 13:14:32.171257 29848 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:14:32.173038 29848 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.173591 29848 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.176028 29848 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.140661 29719 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:32.187193 29748 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.188271 29839 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.189386 29708 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.140980 29847 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:32.190507 29765 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.191052 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:32.185217 29831 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:32.192405 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.207212 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:32.209007 29725 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.210793 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:32.211158 29855 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:32.212450 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:32.212800 29855 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:32.213883 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:32.184996 29686 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.215384 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:32.170032 29732 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:32.177273 29848 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:14:32.214483 29696 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:32.203625 29824 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:32.229362 29725 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:32.228785 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:32.204818 29653 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:14:32.169691 29832 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:14:32.187232 29837 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.236104 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.229897 29725 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:32.240355 22765 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:32.122828 29819 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.242846 29725 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:32.244540 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52209.6us
I20260709 13:14:32.245689 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.247277 29696 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:32.261776 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.247486 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.257855 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.264560 22765 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:32.262451 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.316166 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:32.318909 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.320698 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58352.4us
I20260709 13:14:32.236474 29859 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:32.238896 29856 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:32.321285 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.326573 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:32.250222 29848 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:32.329243 22911 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:32.329119 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.315263 29711 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.258684 29554 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.249668 29686 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:32.334605 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:32.324636 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 204862us
I20260709 13:14:32.325735 29847 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:32.342315 22765 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:32.324851 29832 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.344740 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.344726 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.353315 22911 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:32.354732 22764 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:32.345098 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:32.355862 22758 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:32.348212 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.357581 22762 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:32.349293 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.358824 22912 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:32.323228 29754 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.326646 29765 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:32.260129 29774 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:14:32.327982 29717 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:32.365773 22766 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:32.324875 29743 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:32.330191 29853 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:32.342958 29719 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:32.334251 29823 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.334080 29827 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:32.341555 29839 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:32.396082 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.373785 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 126365us
W20260709 13:14:32.400903 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.420688 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.407845 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.419131 22912 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:14:32.362517 29711 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.400024 29743 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.400604 29717 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:32.374711 29832 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.404309 29765 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.446524 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:32.458191 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80541.1us
I20260709 13:14:32.458060 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.420596 29696 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.406178 29719 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:32.432338 29860 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:14:32.470824 22764 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:32.464526 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58455.6us
I20260709 13:14:32.446026 29856 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:32.374430 29732 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.480742 22911 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:32.481092 22908 raft_consensus.cc:2368] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
I20260709 13:14:32.401537 29853 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.462970 29594 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.469627 29848 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:32.477716 29702 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:14:32.485291 29696 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.487522 29717 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.487522 29856 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.489768 29748 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.446641 29839 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.482489 29859 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:32.495653 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.496100 29711 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:32.496481 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.524458 29848 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.500573 22913 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:32.440970 29823 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.500771 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.528340 29853 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.434134 29827 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:32.528728 22906 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:32.529918 29823 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.530999 29859 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.531862 29765 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.533771 29870 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:32.508527 29837 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.512455 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.483795 29708 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.544795 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.510265 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.544020 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.529531 29827 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.545023 22907 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:32.547329 22910 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:32.548688 22912 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:32.555253 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:32.556360 22841 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:14:32.556218 23464 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:32.522516 29832 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.566534 22908 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:14:32.567904 23463 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:32.510231 29725 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.510875 29594 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.567625 22908 pending_rounds.cc:85] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 14
I20260709 13:14:32.576771 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.594851 22910 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:32.577762 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:32.577919 29709 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
W20260709 13:14:32.587843 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.538919 29748 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.594357 22908 pending_rounds.cc:107] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:14:32.577543 29839 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.597589 22908 raft_consensus.cc:2889] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:32.569416 29832 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:32.617437 23459 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:32.599936 29719 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.601003 22834 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:32.601658 29837 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:32.551036 29708 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:32.621924 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44248.3us
I20260709 13:14:32.622956 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.601759 29696 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.604568 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.604948 29676 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:32.606554 29725 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.608287 29594 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:32.608393 29854 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.369s	user 0.004s	sys 0.000s
I20260709 13:14:32.652981 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.568241 29765 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.663098 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.614930 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:32.609391 29748 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.567508 29823 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:32.618803 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.619093 22762 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:32.620285 22907 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:32.663872 22757 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:32.620723 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.626334 22833 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.627668 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:32.645687 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:32.621414 29812 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.598s	user 0.014s	sys 0.000s
I20260709 13:14:32.669071 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:32.662784 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.664041 22832 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:14:32.625480 29837 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.667541 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.698161 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.671043 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.674101 29743 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.674927 22838 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:32.676663 29765 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.678879 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.680476 29843 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:32.681026 29816 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:14:32.684926 29862 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.290s	user 0.000s	sys 0.002s
I20260709 13:14:32.686156 22840 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:14:32.687422 29857 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:32.689086 29708 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.663861 22908 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.693428 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:32.691408 22839 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.704250 22910 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:32.703593 22912 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
W20260709 13:14:32.705744 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.627938 29832 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.720139 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:32.722108 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.663627 29877 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:32.716472 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93630.6us
I20260709 13:14:32.706063 29837 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:32.716048 29776 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:32.743587 29776 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:32.744081 29776 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:32.730598 22833 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.718806 29748 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:32.742133 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:32.723040 29816 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:32.719051 29843 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:32.742811 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2818) took 1255 ms. Trace:
I20260709 13:14:32.754046 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:32.754657 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.756336 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:32.757984 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.760001 22761 rpcz_store.cc:276] 0709 13:14:31.487780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.497096 (+  9316us) service_pool.cc:224] Handling call
0709 13:14:31.497567 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.153552 (+655985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.742540 (+588988us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540024b154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:32.742744 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507279616}
I20260709 13:14:32.757925 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:32.694837 29812 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.776926 29748 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:32.777557 22833 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.725075 29857 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:32.718982 29593 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:14:32.782454 22758 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:32.782657 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:32.783851 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 164966us
I20260709 13:14:32.788755 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.788632 22911 raft_consensus.cc:2376] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20. Re-sending same reply.
I20260709 13:14:32.789660 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:32.792621 22832 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:32.793301 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:32.794862 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52849.5us
I20260709 13:14:32.795656 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.781414 29855 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:32.793442 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:32.797371 22757 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:32.794261 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.799813 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1040) took 1119 ms. Trace:
I20260709 13:14:32.800210 22759 rpcz_store.cc:276] 0709 13:14:31.680579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.683745 (+  3166us) service_pool.cc:224] Handling call
0709 13:14:31.684034 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.988696 (+304662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.799644 (+810948us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b5400312254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:32.799768 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693103872}
I20260709 13:14:32.801786 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.802237 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:32.785642 29795 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.804773 22908 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:32.805320 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.803563 29813 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:32.810145 23462 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:14:32.810563 23462 pending_rounds.cc:85] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 12
I20260709 13:14:32.810789 23462 pending_rounds.cc:107] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:14:32.811026 23462 raft_consensus.cc:2889] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:32.805976 22911 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:32.822003 22833 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:32.828948 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.829833 29748 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.795529 29653 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:32.838148 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:32.864903 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.865723 22833 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.867573 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:32.795347 29831 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:14:32.889182 29593 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.889880 29593 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.892231 29593 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:32.891955 29831 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:32.893388 29831 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:32.871563 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.836071 29855 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:32.791648 29847 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:14:32.903342 22911 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.834347 29843 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:32.870716 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75130.5us
I20260709 13:14:32.906371 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.906823 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.868332 23462 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:32.834084 29877 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:32.856997 29653 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.903478 29813 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:32.916604 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:32.868311 29725 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:32.813239 29795 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.927333 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:32.931495 29886 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:14:32.933043 29855 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.934161 29765 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:32.920990 29653 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:32.935575 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:32.936604 29877 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:32.930080 29725 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:32.940348 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:32.921986 29821 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:32.953016 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:32.954471 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 116442us
I20260709 13:14:32.956504 22836 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:14:32.938476 29887 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:32.856534 29857 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:32.929064 29859 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:32.963495 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.963812 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:32.967278 22911 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:32.949393 29891 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:32.956355 29877 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:32.949100 29676 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:32.995416 29896 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.005026 22911 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:32.995627 29847 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:32.986176 29856 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:33.013988 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:32.985843 29891 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.985966 29892 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:33.017925 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65131.5us
I20260709 13:14:32.997373 29812 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.024456 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.025786 22911 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:32.991463 29831 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 9, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.012367 29857 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:33.042783 29884 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:33.017266 29891 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:33.021957 29717 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.048120 22908 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:14:33.048570 22908 pending_rounds.cc:85] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 9
I20260709 13:14:33.048795 22908 pending_rounds.cc:107] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:14:33.049036 22908 raft_consensus.cc:2889] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.052114 22910 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:33.053304 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:33.057046 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 101609us
W20260709 13:14:33.044668 29863 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.650s	user 0.004s	sys 0.000s
I20260709 13:14:33.060866 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.062666 22909 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:33.062795 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:33.063657 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.064064 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.065131 22911 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:33.009788 29813 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.081347 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.082688 22914 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:14:33.084728 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65713.4us
I20260709 13:14:33.071297 22908 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:14:33.077184 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:33.089685 22761 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:33.091877 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.097319 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.097666 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:33.098083 22915 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:33.099817 29836 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:33.101266 29856 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:33.101745 29856 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:33.102872 29856 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.105063 29856 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.022730 29892 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:33.106721 29897 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:14:33.025070 29896 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.114982 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.115217 22906 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:33.092860 22910 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:33.128463 29676 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:33.066484 29725 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.130771 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.056108 29776 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.056567 29717 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.116703 29894 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:33.169798 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 186915us
I20260709 13:14:33.138342 29870 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:33.178395 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.179201 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.180200 29894 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.181244 29896 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.182626 29894 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:33.183315 22910 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:33.184932 29719 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.185672 29795 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.185474 29719 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.188161 29719 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.192200 29774 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:33.192782 29774 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:33.193802 29774 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.194279 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.195254 29774 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:33.196017 29702 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:33.198015 29702 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:33.198308 29777 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:33.199100 29702 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:33.198964 29777 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.199504 29702 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:33.201994 29754 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:33.202484 29754 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:33.202947 29847 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:33.202030 23430 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.204784 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:33.205770 29732 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:33.206542 29732 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:33.206913 29732 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.200443 29702 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.207372 29732 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:33.207365 29686 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:33.209053 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.210119 29702 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.211338 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:33.223342 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.220563 29686 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:33.229871 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.231057 29686 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.231949 29653 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.233280 22909 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:33.236246 23462 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:14:33.236719 23462 pending_rounds.cc:85] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 14
I20260709 13:14:33.236980 23462 pending_rounds.cc:107] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:14:33.237335 23462 raft_consensus.cc:2889] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.232839 29686 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:33.226126 23457 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:33.242322 29892 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.135324 29855 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:14:33.225392 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.258946 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.260324 23463 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:33.262212 22835 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:33.263832 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:33.265534 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.266726 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.267125 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:33.267781 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.268340 22839 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.244091 29719 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:14:33.172685 29901 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:33.269994 29901 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.272063 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.272887 22907 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:33.277309 22906 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:33.219285 29899 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.282254 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:33.284837 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.286599 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.287348 22835 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:33.288254 29653 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.288267 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:33.298415 23457 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:33.299638 29901 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.307404 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.308570 22838 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
W20260709 13:14:33.307489 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.313714 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.324229 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.325162 22835 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:33.328399 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:33.334345 22839 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:33.335819 22915 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:33.336212 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.336721 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:33.338783 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:33.339749 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.341365 22907 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:14:33.321842 29837 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.002s
I20260709 13:14:33.338954 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:33.343896 23464 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:33.344060 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.345002 22832 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:33.345824 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:33.301681 29899 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.350620 22915 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:33.352573 22915 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:33.358108 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.372254 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.375289 22910 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:33.376437 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.377351 22910 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:33.378429 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.364004 29748 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:33.347596 29837 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.367353 29765 raft_consensus.cc:2764] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:14:33.387111 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.399058 22910 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:33.381456 29777 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:33.403539 22841 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23.
I20260709 13:14:33.398644 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:33.404328 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.409016 22914 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: [22.9-22.9]
I20260709 13:14:33.414366 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:33.381973 29910 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:14:33.419595 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 76644.1us
I20260709 13:14:33.426890 22833 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:33.430541 22838 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:33.420676 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.425555 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.431974 22910 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:33.421123 29765 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:33.419951 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.432536 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.423672 29816 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:33.433815 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.434435 22879 connection.cc:256] RPC call timeout handler was delayed by -2.3077345119997972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 13:14:33.434616 29816 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.435990 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.422784 29777 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:33.437731 29751 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.438246 29751 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.440464 29751 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:33.441730 22879 connection.cc:256] RPC call timeout handler was delayed by -1.7110345119997419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 13:14:33.443557 22879 connection.cc:256] RPC call timeout handler was delayed by -1.0195881539998481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2663s for a response
I20260709 13:14:33.446975 29899 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.450877 22839 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:33.452008 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.454125 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.455883 22911 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:33.456063 22832 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:33.457260 22911 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:33.459250 22910 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:33.458026 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:33.419850 29874 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:33.466475 22835 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:33.464033 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:33.465247 22908 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:33.452024 29832 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.469610 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:33.467752 29859 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:33.472654 29859 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.473155 29859 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.490477 29837 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:33.425809 29856 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:14:33.493736 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.495929 29832 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:33.496371 29696 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:33.496371 29832 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.497193 29832 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:33.497946 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 52606.5us
I20260709 13:14:33.502241 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.503324 29901 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:33.503851 29901 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:33.509073 29843 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:33.504783 29901 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.514241 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.468492 29874 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.515904 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:33.524037 22758 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:33.524722 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:33.526188 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:33.526949 22758 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.525832 22763 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:33.532089 22762 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:33.493170 29857 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:33.534078 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:33.534313 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:33.534617 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:33.535116 22762 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:33.535194 22763 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:33.530712 29870 consensus_queue.cc:1059] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:33.466214 29676 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:14:33.496778 29696 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:33.534861 22760 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:33.538842 22838 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:33.539551 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.536648 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:33.540124 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.547526 22759 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:33.547713 22761 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:33.547528 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:33.549376 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.552585 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:33.553189 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:33.553613 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.554064 22765 raft_consensus.cc:2410] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:14:33.527683 29874 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:33.522704 29702 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:33.555883 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 151688us
I20260709 13:14:33.559000 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.550872 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.559281 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.560526 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.560945 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.561365 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:33.563264 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.563978 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.565819 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.568310 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.571982 22761 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:33.573261 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:33.578018 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62210.9us
I20260709 13:14:33.517797 29732 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.552649 29816 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:33.557457 29765 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:33.562994 29839 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:33.604638 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72284.2us
I20260709 13:14:33.541312 29751 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:14:33.578389 29892 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:33.602205 22758 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
W20260709 13:14:33.611685 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 64231.6us
I20260709 13:14:33.568549 29916 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:33.603550 29671 raft_consensus.cc:2764] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:14:33.562634 29831 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:33.567485 29686 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:33.546304 29777 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:14:33.571566 29653 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:33.570008 29748 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:33.589448 29676 consensus_queue.cc:1243] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:33.610756 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:33.604700 29857 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:33.631337 22913 raft_consensus.cc:606] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:33.631723 22913 raft_consensus.cc:614] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:33.574790 29593 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:14:33.583455 29896 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:33.633534 29896 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.633931 29896 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:33.634361 29593 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:33.635159 29593 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:33.584120 29845 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:33.615875 23462 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:33.566671 29899 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:14:33.594781 29813 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.571492 29910 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:14:33.651193 29910 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:33.651612 29910 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:33.652601 29910 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:33.645172 29907 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.319s	user 0.002s	sys 0.000s
W20260709 13:14:33.625674 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64410.2us
I20260709 13:14:33.611552 29892 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.615592 29919 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:33.673732 29919 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:33.674172 29919 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.638492 29857 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:33.678323 29857 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.679754 29857 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.647763 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2224) took 1075 ms. Trace:
I20260709 13:14:33.649762 29870 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:33.592088 29765 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:33.627218 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68295.4us
I20260709 13:14:33.657191 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:33.699257 23463 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:33.620158 29831 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.616001 29916 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.659555 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.620409 29823 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:33.629693 29653 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:33.640328 29891 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.006s
I20260709 13:14:33.708572 22913 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:33.708738 29891 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.709159 29812 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:33.709610 29812 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:33.710444 29812 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.664820 29924 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.711712 29812 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:33.711545 29924 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.713624 29848 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:33.660723 29855 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:14:33.670817 29732 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.671938 29702 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:14:33.672609 29676 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:33.692538 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:33.676911 29832 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:33.702432 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.706892 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 101369us
I20260709 13:14:33.707386 29916 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.707839 29823 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.708199 29653 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:33.708227 22914 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:14:33.711006 29891 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:33.721135 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61688.3us
I20260709 13:14:33.682974 29892 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:33.684092 22837 rpcz_store.cc:276] 0709 13:14:32.572372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.572503 (+   131us) service_pool.cc:224] Handling call
0709 13:14:32.573641 (+  1138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.780873 (+207232us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:32.799804 (+ 18931us) log.cc:844] Serialized 14 byte log entry
0709 13:14:32.984510 (+184706us) log.cc:844] Serialized 14 byte log entry
0709 13:14:32.984716 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:32.986800 (+  2084us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:32.986857 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.388632 (+401775us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b180049a500. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1
0709 13:14:33.501930 (+113298us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b180049a500. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:33.505258 (+  3328us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b180049a500. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:33.647487 (+142229us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54002b3254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:33.647544 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.647695 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7160,"prepare.queue_time_us":11964,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":97,"raft.queue_time_us":23487,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":207,"spinlock_wait_cycles":443219968,"thread_start_us":137219,"threads_started":1,"wal-append.queue_time_us":241120}
I20260709 13:14:33.764403 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.729383 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:33.773150 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43864.2us
I20260709 13:14:33.713771 22911 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:33.735859 22909 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:33.673247 29922 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:14:33.704916 29765 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:33.715200 29831 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:33.713755 29717 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.745710 29846 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:33.792634 29846 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:33.793628 29846 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.746371 29823 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:33.658366 29813 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.748492 29702 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:33.748592 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:33.764207 29910 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:33.752038 29859 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.753448 29751 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.754853 29748 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:14:33.777921 23463 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:33.693552 29837 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:14:33.780876 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:33.732839 29711 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.784861 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:33.790359 29593 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:33.782362 29925 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:33.791797 29717 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:33.819402 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.664700 29923 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:14:33.750588 29732 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:33.799255 22840 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:33.801064 22833 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:33.729092 29777 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:14:33.772193 29924 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.707865 29918 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:14:33.757277 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:33.816697 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:33.817070 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.800729 29846 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.817421 22907 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:33.779811 29795 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:14:33.782013 29916 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:33.833035 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.809968 29929 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:33.844308 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63386.7us
I20260709 13:14:33.847512 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2269) took 1111 ms. Trace:
I20260709 13:14:33.846048 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.818544 29593 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.830817 29837 consensus_queue.cc:1243] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:33.862906 22730 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23.
I20260709 13:14:33.876483 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.818701 29933 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:33.812148 29751 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.829872 29711 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.854254 29918 consensus_queue.cc:1243] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 23.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:14:33.836836 29676 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:14:33.854905 29925 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:33.869722 22912 rpcz_store.cc:276] 0709 13:14:32.735997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.736412 (+   415us) service_pool.cc:224] Handling call
0709 13:14:32.736924 (+   512us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:32.780136 (+ 43212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:32.780171 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:32.794164 (+ 13993us) log.cc:844] Serialized 52 byte log entry
0709 13:14:32.924459 (+130295us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:33.677676 (+753217us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b1800452980. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a54e93c7 00007f68a1afb9a7 00007f68a1b0b0bf 00007f68a1b0b038
0709 13:14:33.812552 (+134876us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.835889 (+ 23337us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.836432 (+   543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.836623 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:33.847136 (+ 10513us) raft_consensus.cc:1685] finished
0709 13:14:33.847186 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.847443 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":166052,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":4147,"raft.queue_time_us":15393,"raft.run_cpu_time_us":455,"raft.run_wall_time_us":16581,"spinlock_wait_cycles":2433792,"thread_start_us":176543,"threads_started":2,"wal-append.queue_time_us":136332}
I20260709 13:14:33.881878 22732 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.875473 29593 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.900518 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:33.899544 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:33.867671 29926 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:33.901837 29816 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:33.904178 29711 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.905450 29813 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:33.886823 29734 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:33.917328 29916 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.880510 29748 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:33.936393 29748 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:33.937417 29748 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.934243 29734 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:33.934563 29942 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.934227 29832 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.928851 29732 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:33.860890 29929 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.945345 22910 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:33.945344 22913 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: [22.9-22.9]
W20260709 13:14:33.948864 29935 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:14:33.948418 29874 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.949569 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:33.950356 29765 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.887635 29837 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:14:33.918834 29776 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:33.941884 29891 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:33.930068 29939 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:33.946375 29925 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:33.969867 29856 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:14:33.977689 29859 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.977612 29653 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.974537 29939 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:33.979241 29939 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:33.979568 29754 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:33.980122 29653 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:33.980691 29754 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.982062 29765 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:33.983359 29891 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:33.974788 29925 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.984109 29925 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:33.984468 29891 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:33.986871 29751 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:33.987375 29934 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:33.978197 29856 consensus_queue.cc:1243] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:33.926913 29919 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:34.003684 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 121882us
I20260709 13:14:34.005113 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.013859 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.014711 22910 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:34.021338 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.973407 29918 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:14:34.018720 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:34.034977 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 157393us
I20260709 13:14:33.963343 29942 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.016928 29765 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:34.039036 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.045526 23463 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:34.041260 29918 consensus_queue.cc:1243] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 23.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:14:34.045451 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.039223 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.047574 22834 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:34.050848 29765 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:34.052145 23461 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:34.052798 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:34.054508 23459 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:34.058674 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.010476 29896 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.062666 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.068217 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.069212 22835 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:34.069692 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51074us
I20260709 13:14:34.070745 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:34.084443 23460 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:33.999682 29933 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:34.091325 22908 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:34.094743 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.104389 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.105370 22832 raft_consensus.cc:2368] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
I20260709 13:14:34.067819 29732 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:34.109727 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.077890 29922 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:34.067157 29725 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:14:34.105949 29949 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:34.111481 29946 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:14:34.111913 29926 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:34.118914 29795 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:14:34.140403 22763 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:34.139959 29933 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.140729 29795 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:34.141162 29795 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:34.141436 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.141782 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.142436 22760 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:34.122735 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.151787 22762 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:34.152089 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.119552 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:34.138375 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.143576 22760 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: [22.9-22.9]
W20260709 13:14:34.205483 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.146447 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.202987 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.135962 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:34.136304 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.146090 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.123495 22764 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:14:34.211069 22764 pending_rounds.cc:85] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 8
I20260709 13:14:34.211536 22764 pending_rounds.cc:107] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 13.9
W20260709 13:14:34.208464 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.211866 22764 raft_consensus.cc:2889] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.211634 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.125872 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.146399 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.208808 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.123035 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:34.221035 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 109619us
I20260709 13:14:34.207850 22758 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:14:34.205787 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55591.2us
I20260709 13:14:34.126443 22765 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:34.147883 22910 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:34.212656 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.207567 23464 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:34.228192 22832 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:34.228858 22759 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:34.165849 23457 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:34.122083 29776 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:34.222015 22841 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:34.230212 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.240155 22760 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:34.122248 29922 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:34.130270 29874 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:34.112035 29732 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:34.123380 29916 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:34.243063 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.247593 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.244771 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.139792 29926 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.248632 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.243345 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:34.249348 22841 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:34.248018 22838 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:34.109148 29824 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.120s
I20260709 13:14:34.128391 29942 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:34.150977 29812 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.252184 22766 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:34.139578 29765 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:34.238889 29948 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:14:34.246352 29922 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:34.258132 29812 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.231967 29919 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.241520 29894 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:34.264794 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.134984 29929 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.229122 29899 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:34.275619 29899 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.275368 29696 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.276916 29696 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:34.210592 29831 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.251608 29711 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:34.253628 29926 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:34.253679 29732 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:34.262249 29795 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.284220 29795 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:34.263599 29944 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:34.288115 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2243) took 1094 ms. Trace:
I20260709 13:14:34.245517 29776 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.288583 22840 rpcz_store.cc:276] 0709 13:14:33.193775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.195352 (+  1577us) service_pool.cc:224] Handling call
0709 13:14:33.195691 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:33.835261 (+639570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.287790 (+452529us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54001bbf54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:34.288041 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387061504}
I20260709 13:14:34.252133 29916 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.244140 29751 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:34.119994 29950 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 13:14:34.138535 29593 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.308220 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.309468 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.309719 29593 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.267653 22760 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:34.310882 22832 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:34.311209 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.311793 22760 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:34.256104 22879 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.314339 29894 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:34.314453 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 58653.9us
I20260709 13:14:34.315075 29894 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:34.316236 29894 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.271509 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.274281 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.319525 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.320304 22766 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:34.289921 22836 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:34.290576 22839 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:14:34.277036 29823 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.323457 29823 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.324862 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.325690 22766 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:34.326419 22840 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:34.134096 29939 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.327847 22762 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:34.329026 29939 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.329813 22834 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:34.297426 29776 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:34.311216 22832 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:34.335505 22832 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.336901 22832 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:34.307495 29916 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:34.307319 29957 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.353972 29957 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:34.355271 29892 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.003s
I20260709 13:14:34.355752 29892 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.359071 29831 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:34.339283 29812 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:34.341234 29843 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:34.342032 29923 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:34.367062 29923 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.133994 29949 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.371960 22763 pending_rounds.cc:85] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 7
I20260709 13:14:34.372272 22763 pending_rounds.cc:107] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:14:34.372510 22763 raft_consensus.cc:2889] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.378613 29899 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:34.379684 29732 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:34.384951 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.386077 29901 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:34.386615 29901 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:34.387755 29901 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.389005 29923 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.389424 29732 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:34.390095 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72624.3us
I20260709 13:14:34.390938 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.391785 29950 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:34.392053 22759 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:14:34.392334 29950 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.393010 29776 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.396328 29892 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:34.398753 29939 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:34.399379 29939 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.400766 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:34.401315 29919 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.401998 29754 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.402465 29919 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.402969 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.403679 23465 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:34.405306 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:34.406325 29946 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:34.406893 29946 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:34.402535 29754 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.407897 29946 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.377068 29925 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.414938 29877 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:34.415386 29877 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:34.395637 22759 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:14:34.418155 29941 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.480s	user 0.002s	sys 0.000s
I20260709 13:14:34.420977 29919 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:34.423020 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.425400 22766 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:34.426254 22760 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:34.429665 23465 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:34.433966 29901 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:34.436059 29947 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.504s	user 0.001s	sys 0.000s
W20260709 13:14:34.439118 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48311.8us
I20260709 13:14:34.440197 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.443100 29812 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:34.443619 29899 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:14:34.444800 29812 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:34.445277 29812 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:34.460913 29892 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:34.496249 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:34.466454 29926 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.499524 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.408578 29922 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.502722 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:34.511459 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.512470 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.518165 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.518924 22908 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:34.519622 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:34.520321 22832 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:14:34.520681 22832 pending_rounds.cc:85] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 10
I20260709 13:14:34.520924 22832 pending_rounds.cc:107] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:14:34.521054 22760 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:34.498974 29892 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:34.521198 22832 raft_consensus.cc:2889] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.523286 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.524662 23461 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:34.525401 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.525666 22914 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:34.514231 29845 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:34.503448 29751 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:34.528097 22910 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:34.518710 29891 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:34.529036 29891 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.529291 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.530478 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.531278 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.534474 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.535300 22913 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:34.535851 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.536921 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.494122 29812 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:34.533056 29734 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 22.9, Last appended by leader: 9, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.527851 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.538954 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.537453 22758 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:34.544219 22766 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 23 index: 9. (index mismatch)
I20260709 13:14:34.552613 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.537925 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.554443 22914 raft_consensus.cc:2376] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16. Re-sending same reply.
I20260709 13:14:34.555860 22758 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:34.557165 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.557068 29923 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:34.560087 29957 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.560664 29812 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:34.554538 29732 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:34.529443 29891 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:34.562070 29734 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:34.527786 29845 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:34.563575 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.564394 22762 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:34.563120 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.565138 29949 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.566056 22758 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 13:14:34.569119 23460 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1250) took 1776 ms (client timeout 1776 ms). Trace:
W20260709 13:14:34.569585 23460 rpcz_store.cc:269] 0709 13:14:32.792753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.792869 (+   116us) service_pool.cc:224] Handling call
0709 13:14:34.568948 (+1776079us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400735fd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:34.569050 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2372584320}
W20260709 13:14:34.534528 29952 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 13:14:34.572207 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.573246 22762 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:34.575217 22762 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:34.583156 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.585041 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.586062 22915 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:34.585007 22762 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:34.596796 22760 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:34.527683 29776 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.603044 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.604084 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:34.606491 23431 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.604708 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:14:34.589985 29836 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:14:34.609741 22762 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:14:34.555382 29892 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:34.611234 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.612046 22906 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:34.519857 29916 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.610286 29923 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:34.623955 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.624680 22841 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:34.628342 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:34.632416 22839 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:34.634682 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.635372 22912 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:34.636314 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:34.637373 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.637749 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.638461 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.639742 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:34.638485 22834 raft_consensus.cc:2393] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:34.642478 29754 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:34.644542 29925 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:34.646193 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.646106 22759 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:34.657029 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2685) took 1410 ms. Trace:
I20260709 13:14:34.657675 22836 rpcz_store.cc:276] 0709 13:14:33.246709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.246849 (+   140us) service_pool.cc:224] Handling call
0709 13:14:34.289710 (+1042861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.321703 (+ 31993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.656761 (+335058us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001bbf50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:34.656947 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353688448}
I20260709 13:14:34.619052 29924 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:14:34.665864 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.670873 23464 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:34.679024 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.679842 23460 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:34.684397 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:34.687662 23464 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:34.694307 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:34.695958 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.697139 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.698101 22836 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:34.701932 22841 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:34.702838 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:34.714141 29964 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:14:34.722913 22834 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:14:34.723505 22834 pending_rounds.cc:85] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 10
I20260709 13:14:34.723786 22834 pending_rounds.cc:107] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:14:34.724078 22834 raft_consensus.cc:2889] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.726732 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.735937 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:34.736656 23457 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:34.686208 29754 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:14:34.743945 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:34.709594 29933 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:34.766817 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.731293 29925 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:34.725283 29926 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:34.827155 23460 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:14:34.828810 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 82393.2us
I20260709 13:14:34.720973 29748 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:14:34.750461 29776 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:34.747067 29946 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:34.840302 29944 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:34.702064 29892 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:34.841030 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 541714us
I20260709 13:14:34.829191 29922 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:34.848451 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.849700 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:34.852154 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.847028 29776 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:34.869213 22836 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:14:34.856714 29963 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:14:34.847486 29916 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:34.854729 29939 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:34.917800 29973 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:14:34.945523 22835 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 9. Preceding OpId from leader: term: 23 index: 9. (term mismatch)
I20260709 13:14:34.960942 22835 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 8
I20260709 13:14:34.961241 22835 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 22.9
I20260709 13:14:34.961488 22835 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.859381 29776 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.958313 22757 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:14:34.916615 29944 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:34.925101 23457 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:34.969203 22915 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
W20260709 13:14:34.976660 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.980070 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:34.974459 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 682200us
W20260709 13:14:34.988772 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 106879us
I20260709 13:14:34.988978 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:34.989969 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:34.997215 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:34.990257 29901 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.000s
I20260709 13:14:35.005856 22730 leader_election.cc:400] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
I20260709 13:14:35.014217 22730 leader_election.cc:403] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:14:35.015530 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:35.022099 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:35.010574 29922 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.031028 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:35.033025 29896 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.033862 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.034781 29774 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:35.035225 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.035470 29774 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:35.021796 29845 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.037904 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:35.038635 22833 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:35.039523 29874 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.040073 29874 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.040613 29845 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:35.043841 29918 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:35.044580 29918 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:35.045239 29960 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.396s	user 0.000s	sys 0.003s
I20260709 13:14:35.045109 29918 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.010660 29945 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:14:35.047890 29891 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.061793 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:35.063042 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:35.030069 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:35.050180 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:35.050731 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:35.065837 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:35.050854 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:35.021802 29754 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:35.053434 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:35.057516 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:35.068403 22836 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:35.002998 29824 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:14:35.019343 29916 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:35.029826 29732 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:35.070587 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.081144 22907 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:35.083261 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.096791 22906 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:14:35.084125 22834 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:35.056027 29922 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.061021 29774 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:14:35.075285 29939 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:35.060770 29856 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:14:35.064689 29910 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:14:35.116093 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.117116 22836 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:35.071669 29717 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 13:14:35.120200 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104777us
I20260709 13:14:35.121177 29874 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:35.098348 29933 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:14:35.105072 29916 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:35.121795 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.132390 29926 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:35.132814 29926 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.133136 29926 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:35.123430 29922 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.126520 29916 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:35.124480 29918 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:35.220899 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 207973us
I20260709 13:14:35.208405 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:35.112522 29942 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:35.210220 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.212810 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:35.214000 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.214605 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:35.218652 22758 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:35.126140 29933 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:35.223127 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 861) took 1076 ms. Trace:
I20260709 13:14:35.237592 29939 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:35.243697 29916 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.244289 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:35.244833 29704 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:35.245268 29704 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:35.246363 29704 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.248006 29704 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:35.223407 29812 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:35.250052 29812 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.237576 29933 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:35.252506 29933 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.097801 22839 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:35.260596 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:35.238214 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.257084 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:35.254119 29933 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:35.253149 29812 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:35.274986 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64112.4us
W20260709 13:14:35.226804 29977 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.126s	user 0.002s	sys 0.003s
I20260709 13:14:35.277536 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:35.278146 22914 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:35.278980 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:35.275826 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:35.283895 22908 raft_consensus.cc:1217] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Deduplicated request from leader. Original: 17.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 13:14:35.287194 22908 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:35.291739 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:35.292757 22761 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:35.241878 23466 rpcz_store.cc:276] 0709 13:14:34.146746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.146992 (+   246us) service_pool.cc:224] Handling call
0709 13:14:34.209417 (+ 62425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:34.213268 (+  3851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:34.213317 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:34.236322 (+ 23005us) log.cc:844] Serialized 26 byte log entry
0709 13:14:34.495291 (+258969us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:34.499273 (+  3982us) log.cc:844] Serialized 14 byte log entry
0709 13:14:34.500617 (+  1344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.500711 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:35.044568 (+543857us) spinlock_profiling.cc:243] Waited 911 us on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:35.137580 (+ 93012us) raft_consensus.cc:1685] finished
0709 13:14:35.222629 (+ 85049us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.222785 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4431,"prepare.queue_time_us":1501,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":5411,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":13418,"spinlock_wait_cycles":2094336,"thread_start_us":244353,"threads_started":1,"wal-append.queue_time_us":349244}
I20260709 13:14:35.270076 29891 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.272490 29983 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.271579 29939 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:35.274431 29901 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:35.278308 29653 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:35.304504 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.286945 29717 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:35.313364 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.275606 29943 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:14:35.311350 29983 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.285826 29874 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:14:35.311650 29653 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:35.313596 29942 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:35.340859 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:35.341933 22910 raft_consensus.cc:606] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.327270 29918 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:35.323752 29774 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:35.342721 22910 raft_consensus.cc:614] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:35.333137 29874 consensus_queue.cc:1243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 21.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:14:35.350700 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76676.9us
I20260709 13:14:35.374264 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.319885 29891 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:14:35.314195 29901 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:35.377331 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60246.7us
W20260709 13:14:35.378000 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62235.8us
I20260709 13:14:35.336645 29653 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:35.339699 29892 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.347229 29732 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:35.389658 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:35.325335 29717 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.375278 29845 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:35.340464 29734 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:14:35.378757 29918 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:35.410068 29774 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:35.411626 29774 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.384351 29812 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:35.393886 29732 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.348419 29942 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:35.421180 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.427307 23466 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:14:35.392918 29939 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:35.401067 29989 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.402042 29988 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:35.418644 29734 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:35.426434 29732 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:35.397567 29892 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:35.435281 29981 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
W20260709 13:14:35.432507 29978 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:14:35.438494 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59364.3us
I20260709 13:14:35.403051 29845 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.486896 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.466121 29918 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.474539 29901 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.475278 29926 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:35.476862 29986 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:14:35.436543 29980 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.217s	user 0.000s	sys 0.002s
I20260709 13:14:35.477547 29942 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:35.478101 29939 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.480422 29892 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.439481 29734 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:35.481334 29983 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:35.440883 29891 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:14:35.483136 29985 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:14:35.484057 29696 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.489676 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.434569 29988 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:35.430778 29812 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:35.490392 29976 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:35.434659 29989 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.525895 29754 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18.
W20260709 13:14:35.549907 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 60375.3us
I20260709 13:14:35.553067 29950 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:35.543722 29988 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:35.555704 29856 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:35.556241 29856 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:35.553375 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.557318 29856 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.563844 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3392) took 1293 ms. Trace:
W20260709 13:14:35.565070 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78246.4us
I20260709 13:14:35.564940 22759 rpcz_store.cc:276] 0709 13:14:34.270509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.418112 (+147603us) service_pool.cc:224] Handling call
0709 13:14:34.418541 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.650494 (+231953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.563429 (+912935us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54001f95d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:35.563748 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493255936}
I20260709 13:14:35.566735 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:35.571789 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:35.547499 29901 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:35.544291 29892 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:14:35.562527 29996 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:35.576352 29845 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.550133 29696 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.614261 29812 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:35.616132 29988 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:35.588044 29856 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:35.564517 30000 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
W20260709 13:14:35.635353 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68738.8us
W20260709 13:14:35.628808 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57138.4us
W20260709 13:14:35.630402 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 314132us
W20260709 13:14:35.632081 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 195483us
I20260709 13:14:35.604943 30004 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:35.641439 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:35.623248 29717 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:35.623212 29696 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:35.643926 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:35.625361 29950 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:35.639747 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.622452 29926 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:35.647070 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:35.617439 29975 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.778s	user 0.004s	sys 0.000s
W20260709 13:14:35.727780 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.645766 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:35.809713 22763 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:35.789453 23466 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:35.582626 29734 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.633651 29845 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:35.654035 29996 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:35.794543 22759 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
W20260709 13:14:35.851651 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 213327us
W20260709 13:14:35.797431 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 157778us
I20260709 13:14:35.873415 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.787961 29922 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.757339 29846 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.789137 29999 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.788450 29812 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:35.633325 29653 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:35.788166 29994 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:35.810561 29913 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:35.855949 29704 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.007s
I20260709 13:14:35.871243 29996 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.866703 30006 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:35.936522 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 63186.1us
I20260709 13:14:35.813148 29950 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.015878 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:35.940332 29734 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:35.934497 29891 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:35.937575 29696 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.882987 29901 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:36.033207 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:36.040452 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1043) took 1462 ms. Trace:
I20260709 13:14:36.028571 29892 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.015268 29996 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:36.053292 22911 rpcz_store.cc:276] 0709 13:14:34.578466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.578613 (+   147us) service_pool.cc:224] Handling call
0709 13:14:34.579042 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:34.605681 (+ 26639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:34.605723 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:34.611498 (+  5775us) log.cc:844] Serialized 26 byte log entry
0709 13:14:34.749729 (+138231us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:34.749808 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.749916 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:35.616688 (+866772us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:36.035907 (+419219us) raft_consensus.cc:1685] finished
0709 13:14:36.035942 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.040370 (+  4428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8517,"raft.queue_time_us":19180,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":10298,"spinlock_wait_cycles":22038272,"thread_start_us":103891,"threads_started":1,"wal-append.queue_time_us":214272}
I20260709 13:14:35.933255 29776 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.891527 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:36.032407 29812 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:35.933490 29732 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:36.063798 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 172395us
I20260709 13:14:35.889994 29846 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.069741 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.071241 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.075410 22764 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:36.075717 22762 raft_consensus.cc:1217] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 17.9->[19.10-22.12]   Dedup: 19.10->[20.11-22.12]
W20260709 13:14:36.080107 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 433299us
I20260709 13:14:36.046103 29891 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:36.082535 22762 pending_rounds.cc:85] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 10
I20260709 13:14:36.085291 22762 pending_rounds.cc:107] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:14:36.085705 22762 raft_consensus.cc:2889] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:35.888680 29922 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.081952 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:36.004232 29877 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.031086 30006 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.100983 23466 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:35.877533 29845 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.064343 29874 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:36.095412 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:36.124639 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:36.133199 23466 raft_consensus.cc:2376] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16. Re-sending same reply.
I20260709 13:14:36.126240 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:35.947769 29999 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.014984 29994 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.078797 30005 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.055262 29696 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.090077 29824 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:36.084170 29704 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:36.142170 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2294) took 1628 ms. Trace:
I20260709 13:14:36.066483 29776 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.093171 30014 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:36.066689 29913 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:14:36.096438 29896 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:36.089701 30009 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:36.061446 29892 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.067512 29989 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:36.106914 30006 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.148945 29988 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.148415 22913 rpcz_store.cc:276] 0709 13:14:34.513675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.535804 (+ 22129us) service_pool.cc:224] Handling call
0709 13:14:34.536103 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:34.568832 (+ 32729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:34.568867 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:34.569349 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:14:34.599726 (+ 30377us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:34.619618 (+ 19892us) log.cc:844] Serialized 14 byte log entry
0709 13:14:35.138218 (+518600us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b180035bea0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:14:35.209986 (+ 71768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.210070 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:35.943976 (+733906us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b180035bea0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:36.141919 (+197943us) raft_consensus.cc:1685] finished
0709 13:14:36.141948 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.142093 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50820,"prepare.queue_time_us":19647,"prepare.run_cpu_time_us":707,"prepare.run_wall_time_us":4917,"raft.queue_time_us":624900,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":3993,"spinlock_wait_cycles":17559552,"thread_start_us":29795,"threads_started":1,"wal-append.queue_time_us":49143}
W20260709 13:14:36.155987 22880 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.022s
I20260709 13:14:36.089635 29942 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 13:14:36.185993 22880 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.006s	sys 0.022s
W20260709 13:14:36.186771 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 550102us
I20260709 13:14:36.156961 29774 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.158303 30009 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.152426 29704 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.275859 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.282557 22913 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:36.273880 30015 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.278899 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.283459 30015 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.284093 22911 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:36.285692 30013 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:36.286324 30013 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.274068 30009 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:36.293681 30019 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.294420 30019 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.145702 30005 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.162817 30014 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 13:14:36.188517 29913 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 13:14:36.308319 29653 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:36.233435 29988 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
W20260709 13:14:36.339630 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 244347us
I20260709 13:14:36.159351 29901 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.155555 29891 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.236824 29774 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:36.336792 30010 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:14:36.160446 29812 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.326149 29824 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:36.344040 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:36.328253 29942 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:36.331826 29924 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:36.358520 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 302356us
I20260709 13:14:36.340682 30021 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.344925 29950 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.377465 29950 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.378257 29950 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.380218 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:36.380900 22835 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:36.347734 30020 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:36.383630 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:36.364076 29846 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.404824 29846 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.392410 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:36.364527 30022 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.394887 29892 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.392467 29999 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.364616 29824 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.402002 29922 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.403107 29997 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:36.403594 29774 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.368672 29942 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:36.441360 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:36.441737 29942 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.393244 30023 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.443137 29942 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:36.443174 30023 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.445946 30023 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:36.408792 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 172548us
W20260709 13:14:36.418330 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 330722us
I20260709 13:14:36.420220 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:36.464324 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:36.389209 30006 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:14:36.414319 29845 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.427211 29824 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:36.470881 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50820.3us
I20260709 13:14:36.374656 29924 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.471944 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.423454 30030 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:36.468189 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.407172 29846 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.477384 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.478636 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:36.464411 29988 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.385414 30021 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.440466 29997 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:36.497774 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:36.474712 30014 consensus_queue.cc:1243] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:36.487263 30030 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.406239 30020 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.531629 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.416042 30022 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.475947 29896 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:36.537439 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:36.537717 30027 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.542061 30040 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:14:36.490449 29916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.108s	user 0.011s	sys 0.005s
W20260709 13:14:36.574674 29916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.012s	sys 0.005s
I20260709 13:14:36.576067 29916 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:36.471468 29874 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:36.578150 29874 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.496901 29812 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:36.581945 29812 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:36.537168 30026 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.586089 30026 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.490280 29942 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.589653 29942 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.507505 29845 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:36.570747 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.594926 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.595798 22759 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:36.596947 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.500411 29892 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:14:36.599257 29845 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.583321 29812 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.490602 30023 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:14:36.500830 29988 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:36.612818 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.615070 22759 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:14:36.553799 30030 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:36.624264 22764 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:36.626797 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:36.628494 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.632880 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:36.619652 30037 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.620998 29774 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:36.652750 23464 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:36.653723 29922 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:36.655519 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.704471 23462 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:36.706923 30044 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:36.707669 30044 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.653355 30015 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.607616 30027 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.677407 30020 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.711216 30044 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.711509 30027 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.711944 29845 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.705617 29774 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.719394 30009 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.720453 29774 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.721835 29944 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:36.722602 29944 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.723932 30046 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.724604 30046 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.728615 30046 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:36.729346 30038 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:36.730013 30046 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.730646 30046 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.733078 30046 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:36.736637 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 199297us
I20260709 13:14:36.754794 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.714452 29922 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.776238 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.777297 23465 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:36.764894 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:36.783919 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.675941 22759 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:36.678400 30021 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.679199 29997 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:36.680950 30039 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:14:36.682348 30034 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:14:36.684487 29942 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.686040 29870 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.686841 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.689770 30022 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:36.690510 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.691519 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.773712 30023 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.692368 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.689785 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.693238 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.796950 29901 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.699637 22912 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:36.793298 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:36.851646 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 58440us
I20260709 13:14:36.795706 29846 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:36.761430 29856 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.125s	user 0.011s	sys 0.000s
I20260709 13:14:36.800580 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.813076 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.815577 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:36.856959 30042 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.818497 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.875171 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:36.831995 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.876308 22913 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:14:36.692440 23466 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:36.832602 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.829936 22911 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:36.735373 29774 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:14:36.796885 30044 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:36.838167 22914 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:36.890040 30044 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:36.890671 30044 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:36.696933 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.891997 22915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:36.892619 22915 raft_consensus.cc:606] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:36.659159 30037 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.839464 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.896090 22907 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:36.862470 30054 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:36.897595 30054 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.898130 30054 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:36.898504 22907 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:36.890838 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:36.900241 22914 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:36.860925 29812 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:14:36.810288 29896 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.860666 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:36.902280 22910 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:36.902230 29896 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.881947 30042 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.816143 22759 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:36.906384 29983 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.906949 29983 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:36.821557 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.907979 30042 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:36.908270 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.884491 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.834106 30014 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:36.914736 22758 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:36.886824 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:36.889410 30049 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:14:36.914255 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.918378 22910 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:36.866544 29997 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:36.920681 23466 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:36.857199 29944 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.924683 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:36.927790 23459 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:36.869091 29922 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:36.905248 29896 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:36.933413 23431 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:36.934325 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.935173 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:36.948823 22759 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
W20260709 13:14:36.939590 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 234230us
W20260709 13:14:36.941653 22730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 2.021s (SENT) [suppressed 9 similar messages]
I20260709 13:14:36.917806 29846 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.855346 30023 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.953280 23458 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:36.839555 29870 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:36.932432 29996 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.852s	user 0.014s	sys 0.001s
I20260709 13:14:36.918040 29856 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:36.956869 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:36.865584 29901 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:36.918190 29774 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:36.957943 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.967097 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:36.967674 22910 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:36.924131 30009 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:36.959779 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.931573 29988 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:36.986253 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.987716 22910 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:36.993633 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.994704 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.994997 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.995903 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.996049 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.996651 22908 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:36.997752 29845 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:36.998389 29845 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:36.998986 29845 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.001888 22907 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:37.004066 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:36.958679 29944 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:37.018270 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.019098 23458 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:36.996605 22910 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
W20260709 13:14:37.035933 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60068.4us
W20260709 13:14:37.037758 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:37.039376 29734 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.993s	user 0.009s	sys 0.009s
I20260709 13:14:36.925557 30014 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:36.992440 29983 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.958983 29846 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.061599 29994 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:36.982918 29774 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.027324 30011 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.068743 30054 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.078567 30011 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.079782 29774 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.080255 30054 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.081758 22914 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
W20260709 13:14:37.084160 30057 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:14:37.086135 29924 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.086963 30074 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:37.087359 29776 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.047978 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:36.990396 30035 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:37.003470 30032 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.050851 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46306.3us
I20260709 13:14:37.058122 30029 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:14:37.068811 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:37.006368 29856 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.059844 30026 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:37.096432 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48550.5us
I20260709 13:14:37.061966 29846 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.106621 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:36.971220 29996 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.084920 30030 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.131131 23463 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21.
I20260709 13:14:37.068429 30017 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:37.140084 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:37.140744 30017 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:37.141613 30040 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:37.062485 30066 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.120175 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 72794.3us
W20260709 13:14:37.126694 30059 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:37.114182 30035 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.087738 30074 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:37.138657 29846 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:37.068665 30045 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:37.036535 29944 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.150944 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:37.151456 30066 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.112248 29831 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:37.156636 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.158742 23458 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:37.148658 29988 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:14:37.164688 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:37.165982 22913 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:37.166767 22807 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:37.166914 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.167511 23461 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:37.148360 30027 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:37.167634 23459 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:37.168027 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:37.171842 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:37.173112 23458 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:14:37.176982 22836 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:37.180048 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.162240 29831 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.180912 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:37.183240 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:37.167052 30074 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.191941 22908 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:14:37.202111 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:37.202713 22909 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:37.203094 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.116204 30032 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.206486 22835 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.187831 22836 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:37.184187 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.141922 30017 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.188359 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.205871 22837 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:37.205700 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.209741 30078 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:37.210057 30030 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.222031 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:37.183959 29831 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:37.220890 22838 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:37.170933 30027 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:37.168632 30026 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:14:37.172864 30040 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:37.201800 29939 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:37.210628 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.168217 29846 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:37.223286 22841 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:37.223788 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.228978 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.219928 30078 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.243335 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3405) took 1168 ms. Trace:
I20260709 13:14:37.213402 30076 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.181185 30045 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:14:37.228632 30054 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.222734 22909 raft_consensus.cc:2376] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21. Re-sending same reply.
I20260709 13:14:37.253446 22836 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:37.220911 22764 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:37.253894 22840 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:37.268818 29988 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:37.307497 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.269666 30069 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:37.271086 30076 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.285336 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:37.311090 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:37.249233 30040 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:37.292672 29824 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.800s	user 0.009s	sys 0.006s
I20260709 13:14:37.272864 29845 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.294276 29754 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.326745 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.296895 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:37.297734 29901 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.204958 30070 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:14:37.257203 22762 rpcz_store.cc:276] 0709 13:14:36.074839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.075125 (+   286us) service_pool.cc:224] Handling call
0709 13:14:36.075503 (+   378us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:36.239627 (+164124us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:36.475486 (+235859us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.544516 (+ 69030us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:36.545691 (+  1175us) log.cc:844] Serialized 52 byte log entry
0709 13:14:36.545936 (+   245us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:36.546164 (+   228us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.612853 (+ 66689us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.615244 (+  2391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.615452 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:37.243029 (+627577us) raft_consensus.cc:1685] finished
0709 13:14:37.243054 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.243246 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11551,"prepare.queue_time_us":167523,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1792,"raft.queue_time_us":343549,"raft.run_cpu_time_us":741,"raft.run_wall_time_us":18404,"spinlock_wait_cycles":9583488,"thread_start_us":499214,"threads_started":4,"wal-append.queue_time_us":280577}
I20260709 13:14:37.327529 22907 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:37.299245 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.299325 29944 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.296947 30066 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.302644 29856 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.287750 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 1008) took 1188 ms. Trace:
I20260709 13:14:37.259564 30078 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:37.335108 23460 rpcz_store.cc:276] 0709 13:14:36.099065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.099227 (+   162us) service_pool.cc:224] Handling call
0709 13:14:36.099504 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.326127 (+226623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.326165 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:36.326225 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:36.326260 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.287434 (+961174us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54002c8b54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:37.287500 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.287675 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520626944}
I20260709 13:14:37.244462 29870 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.253906 29939 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.337255 29939 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:37.317127 30030 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:37.251475 30020 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.211879 30074 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.371618 30020 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.304414 29776 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:37.380193 29776 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:37.316974 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.385886 22840 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:37.385593 29776 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.386732 30064 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:37.311909 30069 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.294863 30025 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:37.387588 30069 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:37.387789 30025 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:37.329142 30079 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.331328 30076 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.321884 30027 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.322359 30040 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.314464 30054 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.328419 22837 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:37.403421 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.326787 23458 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 26.
I20260709 13:14:37.317399 29824 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.387743 30042 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.389889 29892 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.407094 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:37.345578 29754 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.346303 29704 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.347640 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.351464 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.355677 29923 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:37.414350 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:37.415946 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.423115 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.360214 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:37.363128 29913 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:37.367524 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.368597 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3647) took 2109 ms. Trace:
I20260709 13:14:37.369002 29998 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.369969 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:37.373979 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.315383 22835 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:37.387018 29776 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:37.387068 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.314703 29988 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.390841 23464 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:37.325582 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.440546 22915 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:14:37.441466 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.442214 22915 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:37.443040 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.444093 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.445061 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.319948 29845 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:37.372864 29987 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 2.007s	user 0.011s	sys 0.009s
W20260709 13:14:37.400839 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.323850 30044 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.423380 29892 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.404448 22909 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:37.407812 23463 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:37.342340 30032 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.389457 29924 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.480448 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.432688 30002 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.434154 22840 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:37.435030 22911 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:37.436218 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.489435 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:37.399556 30079 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.493741 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.425177 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.449065 22915 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:37.434135 22906 rpcz_store.cc:276] 0709 13:14:35.259457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.260652 (+  1195us) service_pool.cc:224] Handling call
0709 13:14:35.271794 (+ 11142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.402275 (+130481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.402344 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.402394 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:35.402439 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.368268 (+1965829us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540010e5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:37.368334 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.368497 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299505408}
W20260709 13:14:37.452378 29910 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.104s	user 0.005s	sys 0.021s
I20260709 13:14:37.476016 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.388759 30025 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.468520 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:37.477376 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.478521 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.502341 22759 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:37.502722 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.498883 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.504228 22906 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:37.505154 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:37.506604 22763 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:37.418951 30042 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.424678 30076 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:37.424388 29996 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.478727 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.438261 29913 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.520927 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.508227 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.521651 22764 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:37.521915 22762 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:37.508872 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.433470 22912 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:37.513898 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.514650 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.525094 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.515390 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.480422 29924 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.527086 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.423794 29923 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.518468 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:37.506191 29910 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.161s	user 0.005s	sys 0.021s
I20260709 13:14:37.525915 22912 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
W20260709 13:14:37.527873 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.527884 22915 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:37.529631 22759 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:37.550376 22912 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:14:37.493152 29892 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.532908 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:37.534191 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:37.552012 22836 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:37.485601 29856 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:14:37.540460 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.540736 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:37.553174 22835 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:37.540601 30026 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:37.541471 29924 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.543226 29913 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.508355 30002 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.559898 22911 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:37.548173 22907 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:14:37.554733 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:37.531767 30076 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:37.522751 30025 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:37.552111 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:37.572840 30085 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:37.576457 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:37.531677 29846 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.577911 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:37.574424 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:37.589963 30035 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.590255 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.594933 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.557039 29997 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:37.606053 22841 raft_consensus.cc:2376] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16. Re-sending same reply.
I20260709 13:14:37.557592 29996 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:37.546977 29918 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.447062 29998 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.579006 30020 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:37.583449 30086 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.130s	user 0.000s	sys 0.005s
I20260709 13:14:37.602646 30022 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:37.602105 30068 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:37.605402 29913 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:37.602847 29892 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:37.606835 29943 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.635967 29943 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.643015 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:37.652750 29653 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:37.654398 30076 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.656358 30037 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.659732 29998 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.660735 29754 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.662030 30002 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.663194 30079 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.664175 29988 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.665023 30042 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.618860 30021 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.666258 30044 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.670099 29824 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.673815 29996 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:37.674440 29996 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.674976 29732 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.665992 30021 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.603389 29846 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.620994 29918 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.653268 29653 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:37.624662 30022 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.629583 30068 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:37.629427 29913 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:37.643610 29997 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:37.643806 29892 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.703054 29831 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.716662 29831 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.719763 29950 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.720844 22906 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 26: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:37.721915 29997 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:37.722831 30019 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.723009 29997 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.724529 29923 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.730362 29696 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.680441 30003 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.742758 30074 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.744099 29653 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.716683 29877 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.748811 29996 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.752054 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.706631 30022 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.753544 29892 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:37.765295 30011 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:37.770356 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.708693 30068 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:37.767755 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.695446 30006 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:37.682878 30093 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:37.819890 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:37.792007 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.752768 30069 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.797845 23460 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:14:37.826833 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.828600 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.799530 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:37.808987 23464 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:37.625701 30078 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.812554 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:37.784565 30096 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:14:37.814985 23463 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:37.816167 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:37.816817 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:37.817700 30074 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:37.788717 30003 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:37.896523 30078 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.941912 29943 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.898329 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.867100 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 1009) took 1767 ms. Trace:
I20260709 13:14:37.946323 22841 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:37.874346 22832 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:37.946417 23457 rpcz_store.cc:276] 0709 13:14:36.099895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.100041 (+   146us) service_pool.cc:224] Handling call
0709 13:14:36.135474 (+ 35433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.669749 (+1534275us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b18003a9c60. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:14:37.669910 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.669941 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:37.670504 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:14:37.809857 (+139353us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:37.809903 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.809971 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:37.828035 (+ 18064us) raft_consensus.cc:1685] finished
0709 13:14:37.828058 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.866978 (+ 38920us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6533,"raft.queue_time_us":174104,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":23870,"spinlock_wait_cycles":2933504,"thread_start_us":315789,"threads_started":2,"wal-append.queue_time_us":145268}
I20260709 13:14:37.949981 23459 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 26: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:37.951625 22879 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21.
I20260709 13:14:37.952102 22879 leader_election.cc:403] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 13:14:37.920485 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:37.953974 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 485571us
I20260709 13:14:37.829968 23466 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:37.906297 23464 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:14:37.876832 22758 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:37.836848 23460 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 23: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:37.870239 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57835.4us
I20260709 13:14:37.807739 29837 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.818748 29653 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:37.959950 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 143249us
I20260709 13:14:37.814007 29992 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:37.966958 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:37.974926 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:37.978641 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:37.978838 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:37.975296 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:37.981680 30011 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:37.982380 30011 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:37.982797 30011 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:37.986402 23457 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:37.986583 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:37.935979 30013 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.989681 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:37.993881 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:37.869666 29696 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:37.983965 30000 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:37.973311 30027 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.976365 30042 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:37.964004 30004 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:37.972625 29933 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:37.979794 29939 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.642s	user 0.014s	sys 0.000s
I20260709 13:14:37.997013 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:38.012271 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:37.988200 29925 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:37.991355 30078 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:37.992149 30052 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:38.048588 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.027451 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:38.000619 30000 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.997534 29846 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:37.974323 29837 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.061115 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:37.965443 29943 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:14:37.998229 30053 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:37.984171 30009 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:14:38.057410 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:38.002367 30093 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:38.065011 30095 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:38.057294 30000 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:38.067262 29837 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:37.832041 30006 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.072118 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:38.139462 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67479.5us
I20260709 13:14:38.140403 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:38.026360 30018 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:37.985984 29992 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:37.814682 29913 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.055193 29776 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:38.055457 30052 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:38.163793 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.073405 30078 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.169667 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:38.073967 30053 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:38.030804 29925 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:38.155907 29748 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.052s	sys 0.090s
I20260709 13:14:38.137634 30074 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:38.084000 30021 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:38.204377 22840 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 13:14:38.204805 22840 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:38.197870 30003 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.201941 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.152230 29717 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:38.203011 29925 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:38.157025 29950 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:38.215652 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:38.248937 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:38.250608 22763 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.250579 22840 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.218165 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.252094 23459 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:38.173254 29913 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:38.062422 30004 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.175894 30078 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.203611 30074 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:38.310046 30074 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:38.212384 30040 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.002s
I20260709 13:14:38.313946 30040 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.314208 30068 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.189776 30053 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.190044 30079 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:38.175148 30093 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:38.030332 29996 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.194s
I20260709 13:14:38.214886 29717 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.171247 30052 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:38.166699 30018 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.340425 30018 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:38.219158 30069 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.250201 29950 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:38.275544 30097 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
W20260709 13:14:38.276484 30099 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:14:38.279179 30004 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.264577 30013 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.289070 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:38.226960 30085 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.295351 30006 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.227943 30020 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.306111 30054 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:38.308282 30033 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.754s	user 0.009s	sys 0.000s
I20260709 13:14:38.034271 29939 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.315681 22839 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:38.317152 22840 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:38.368454 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2726) took 1009 ms. Trace:
I20260709 13:14:38.368938 22840 rpcz_store.cc:276] 0709 13:14:37.359386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.215368 (+855982us) service_pool.cc:224] Handling call
0709 13:14:38.368344 (+152976us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.320726 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.322840 30040 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.331408 29717 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.331799 29996 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:38.334393 30093 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.336082 29992 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.291599 29892 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.422827 29892 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:38.329941 30025 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.736s	user 0.008s	sys 0.005s
I20260709 13:14:38.327538 30079 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.348263 29950 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.424794 30079 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:38.377683 22763 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
W20260709 13:14:38.378983 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 267215us
I20260709 13:14:38.385257 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.426352 22840 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:38.363034 30069 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.387835 30068 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:38.415386 29877 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.000s
I20260709 13:14:38.425462 22838 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:38.433646 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.435258 22840 raft_consensus.cc:2410] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 13:14:38.438107 22766 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:38.442037 22833 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:14:38.441475 30022 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.004s
I20260709 13:14:38.381717 29996 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:38.450354 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.367666 30011 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.430603 29696 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.453089 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:38.458724 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.365947 30085 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.459735 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.459676 22840 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:38.365702 30020 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.463989 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.465682 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.467325 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.468127 22840 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:38.448541 30060 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.470599 30060 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.471347 30011 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.452215 30022 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.474270 30022 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:38.475713 30000 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.451236 30040 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.477931 30040 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.480466 30040 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.482795 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3403) took 2410 ms. Trace:
I20260709 13:14:38.483318 22760 rpcz_store.cc:276] 0709 13:14:36.072753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.072900 (+   147us) service_pool.cc:224] Handling call
0709 13:14:36.073297 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:36.073613 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.073652 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:36.085603 (+ 11951us) log.cc:844] Serialized 26 byte log entry
0709 13:14:36.352548 (+266945us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:36.352590 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.352672 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:38.307476 (+1954804us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:38.482340 (+174864us) raft_consensus.cc:1685] finished
0709 13:14:38.482383 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.482710 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5325184,"thread_start_us":201707,"threads_started":1,"wal-append.queue_time_us":328315}
I20260709 13:14:38.477999 29696 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.491573 29696 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:38.469280 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.494241 22840 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:38.463047 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:38.373116 30013 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.514380 29933 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:38.516045 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:38.516870 22840 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.447305 30069 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.528411 22835 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:38.535403 22840 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:38.507622 29877 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:38.540134 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.542824 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:38.533324 30042 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.001s
I20260709 13:14:38.541638 29877 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.546849 22840 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:38.548466 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2729) took 1188 ms. Trace:
I20260709 13:14:38.548033 29877 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.550148 30042 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21.
I20260709 13:14:38.552459 22760 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:38.537482 29939 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.526156 30078 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.567106 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.568781 22840 rpcz_store.cc:276] 0709 13:14:37.360121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.542521 (+1182400us) service_pool.cc:224] Handling call
0709 13:14:38.548387 (+  5866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.573127 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:38.576678 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.021s	sys 0.054s
W20260709 13:14:38.577174 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.021s	sys 0.054s
I20260709 13:14:38.576925 22840 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
W20260709 13:14:38.577884 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.00348e+06us
I20260709 13:14:38.577953 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:38.578464 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:38.579900 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 305608us
I20260709 13:14:38.577963 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2735) took 1198 ms. Trace:
I20260709 13:14:38.583981 22840 rpcz_store.cc:276] 0709 13:14:37.379235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.572796 (+1193561us) service_pool.cc:224] Handling call
0709 13:14:38.577878 (+  5082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.585201 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.587500 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.588807 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.591068 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:38.537787 29996 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.560259 30000 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.606195 29696 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:38.601274 30069 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.611963 22763 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:38.632622 29696 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:38.595607 30052 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.634408 29996 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.593290 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:38.634701 29696 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.635478 30006 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:38.635991 30052 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.618830 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:38.636718 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43503.5us
I20260709 13:14:38.590519 30042 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.607136 30085 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.637432 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:38.638216 30085 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:38.638373 30082 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:38.637730 30042 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.641433 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:38.643213 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:38.645015 30040 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:38.607522 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2744) took 1204 ms. Trace:
I20260709 13:14:38.646108 30040 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:38.645895 22840 rpcz_store.cc:276] 0709 13:14:37.402895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.584866 (+1181971us) service_pool.cc:224] Handling call
0709 13:14:38.607450 (+ 22584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.646919 30040 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.647477 30040 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.647632 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:38.648365 22840 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:38.651146 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:38.593763 30068 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:38.595320 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.591684 30011 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.638761 30085 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:38.598429 30053 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:38.655021 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.659770 29846 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:38.671677 29776 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.002s
W20260709 13:14:38.684894 22791 heartbeater.cc:646] Failed to heartbeat to 127.21.33.62:44143 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.33.62:44143 timed out after 15.000s (SENT)
I20260709 13:14:38.700481 29717 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.712875 22912 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:38.714980 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.715617 22912 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:38.650380 29933 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 13:14:38.717494 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.718266 22912 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:38.699167 30020 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:38.699928 22766 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3408) took 1512 ms (client timeout 1960 ms). Trace:
I20260709 13:14:38.701838 30078 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.724021 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.724745 22759 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:38.701661 30011 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.705386 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:38.727885 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.656533 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:38.720845 22766 rpcz_store.cc:269] 0709 13:14:37.186995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.187133 (+   138us) service_pool.cc:224] Handling call
0709 13:14:38.699696 (+1512563us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002db4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:38.699841 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2728880384}
W20260709 13:14:38.723500 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.726861 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.730922 22914 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:38.732218 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75764us
I20260709 13:14:38.732551 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.733136 22907 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:38.733695 22840 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:38.734215 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.732254 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.735463 22908 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.735229 30085 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.737818 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.738468 22909 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:38.739173 29925 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.740484 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.741290 22766 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:38.734326 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.741448 29925 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.743360 30022 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:38.741791 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.744287 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3411) took 1270 ms. Trace:
I20260709 13:14:38.742810 22911 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:38.745328 22760 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:38.742187 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:38.744760 22765 rpcz_store.cc:276] 0709 13:14:37.474225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.474375 (+   150us) service_pool.cc:224] Handling call
0709 13:14:37.493709 (+ 19334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.499609 (+  5900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.499663 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:37.531410 (+ 31747us) log.cc:844] Serialized 26 byte log entry
0709 13:14:37.912404 (+380994us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:37.912466 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.912566 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:38.743160 (+830594us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b240012a328. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea42fb0 00007f68a16b9488 00007f68a16b9be8 00007f68a1691398 00007f68a168c76a 00007f68a168bbb6
0709 13:14:38.743908 (+   748us) raft_consensus.cc:1685] finished
0709 13:14:38.743937 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.744199 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5391,"raft.queue_time_us":46177,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":20313472,"thread_start_us":366168,"threads_started":1,"wal-append.queue_time_us":414363}
I20260709 13:14:38.749297 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:38.719421 29776 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:38.751567 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46279.5us
I20260709 13:14:38.752740 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:38.751287 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.754410 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1255) took 1013 ms. Trace:
I20260709 13:14:38.723773 30013 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.756348 22762 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:38.757454 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.758564 22759 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:38.759449 30078 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:38.760084 30078 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.760689 30078 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:38.765790 29925 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.766431 29925 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.766848 29717 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.767525 30053 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.769091 29925 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.774938 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.775796 22766 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:38.754974 22840 rpcz_store.cc:276] 0709 13:14:37.740583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.750933 (+1010350us) service_pool.cc:224] Handling call
0709 13:14:38.754325 (+  3392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.776733 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:38.780782 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 202892us
I20260709 13:14:38.773524 30014 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.803728 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:38.804149 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.808182 22914 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:38.806121 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.808542 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.809382 22911 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:38.809628 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.814982 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.816078 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.817001 22840 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:38.817682 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2741) took 1433 ms. Trace:
I20260709 13:14:38.818131 22840 rpcz_store.cc:276] 0709 13:14:37.384417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.808878 (+1424461us) service_pool.cc:224] Handling call
0709 13:14:38.817600 (+  8722us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.819641 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.820820 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2283) took 1049 ms. Trace:
I20260709 13:14:38.821833 22907 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:38.824519 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.825384 22759 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:38.828749 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:38.821224 22840 rpcz_store.cc:276] 0709 13:14:37.771722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.819310 (+1047588us) service_pool.cc:224] Handling call
0709 13:14:38.820753 (+  1443us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.829916 23460 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:38.831957 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.832839 22765 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:38.833297 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.782491 30013 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.808648 30020 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:38.835642 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:38.841635 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.848942 22840 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.854331 30030 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:38.848129 22766 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:38.837167 22907 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:38.847010 30020 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.861490 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:38.875034 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 177442us
I20260709 13:14:38.876750 22908 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:38.887068 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:38.889693 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.890852 22766 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:38.893457 22840 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:38.898093 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.846087 29950 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.895488 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.884788 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:38.861425 30020 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:38.902083 22840 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:38.855420 29925 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:38.905251 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1261) took 1158 ms. Trace:
I20260709 13:14:38.905689 22840 rpcz_store.cc:276] 0709 13:14:37.746798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.897836 (+1151038us) service_pool.cc:224] Handling call
0709 13:14:38.905138 (+  7302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.903515 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:38.928828 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 117105us
I20260709 13:14:38.929339 22766 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:38.880107 30093 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.908123 29950 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.909013 22764 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:38.910298 30011 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.912132 29918 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.843129 30013 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.899758 30079 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.905754 30040 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.909405 30078 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.923995 29891 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:38.866336 30042 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.926420 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:38.932227 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.951992 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.937646 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.936900 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:38.940011 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.941444 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.954903 23457 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:38.927137 30046 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:38.942818 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.957242 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.957930 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.958627 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:38.953974 23461 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:38.959532 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.961082 30046 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:38.961695 30046 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:38.962092 29696 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.962831 29696 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:38.929649 29925 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:38.966073 30112 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:14:38.967005 22914 raft_consensus.cc:2368] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 25. Current term is 26.
I20260709 13:14:38.967722 29925 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.968976 30078 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.939220 30116 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:14:38.971632 22911 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:38.973636 30040 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.968031 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.975069 30079 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:38.975118 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:38.976487 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.943419 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:38.979095 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1262) took 1232 ms. Trace:
I20260709 13:14:38.979498 22840 rpcz_store.cc:276] 0709 13:14:37.747093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.936317 (+1189224us) service_pool.cc:224] Handling call
0709 13:14:38.979028 (+ 42711us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:38.937191 30093 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.981469 30026 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:38.982362 30026 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:38.982839 30093 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:38.981686 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:38.981701 30045 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:14:38.985026 30023 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:38.930644 29717 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:14:38.982754 30026 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:38.985358 30074 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:38.986395 30074 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.986624 29717 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:38.986840 30074 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.987077 29717 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:38.987607 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3416) took 1508 ms (client timeout 2008 ms). Trace:
W20260709 13:14:38.988341 22758 rpcz_store.cc:269] 0709 13:14:37.479897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.480149 (+   252us) service_pool.cc:224] Handling call
0709 13:14:38.987329 (+1507180us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400393354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:38.987534 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900367104}
W20260709 13:14:38.990248 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2259) took 3940 ms (client timeout 2021 ms). Trace:
I20260709 13:14:38.991751 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.946285 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:38.924535 29877 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:38.956036 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:38.985526 30045 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:38.946158 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:38.942561 30067 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:38.996977 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:38.997941 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.014012 22914 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:39.016819 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.013082 23460 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
W20260709 13:14:38.943835 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57634.5us
I20260709 13:14:39.007637 23466 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:38.991154 22833 rpcz_store.cc:269] 0709 13:14:35.049787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.049905 (+   118us) service_pool.cc:224] Handling call
0709 13:14:38.989990 (+3940085us) spinlock_profiling.cc:243] Waited 3.38 s on lock 0x7b540025ec54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:38.990150 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7763233152}
I20260709 13:14:39.002170 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.020213 22833 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:39.020093 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:38.988637 29717 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.021698 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:39.021895 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.023283 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.024845 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.015172 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:39.026608 23457 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.028354 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2727) took 1668 ms. Trace:
I20260709 13:14:39.028980 22833 rpcz_store.cc:276] 0709 13:14:37.359622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.019597 (+1659975us) service_pool.cc:224] Handling call
0709 13:14:39.019976 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.028003 (+  8027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.028275 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.033841 22840 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:39.037597 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.038180 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.040542 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.042845 22833 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:39.039268 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:38.996856 29696 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.045683 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.036449 30052 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:39.051321 22758 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:39.052584 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.007306 30045 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:39.055567 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.016292 30023 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.056434 22914 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:39.053802 22758 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:39.039178 29939 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:39.066354 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.037344 30078 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:39.040910 30040 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:39.047168 30051 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.291s	user 0.004s	sys 0.011s
I20260709 13:14:39.079511 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.084270 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.092489 22914 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:39.094316 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.087774 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.087915 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.039472 30000 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:39.055097 30042 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:39.097837 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.119611 22907 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:39.123952 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:39.099537 22914 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.100404 29846 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:39.089696 29992 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:14:39.139428 22835 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2733) took 1748 ms (client timeout 2131 ms). Trace:
I20260709 13:14:39.106936 30045 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.047952 30082 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:39.044289 30044 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:39.109912 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.087021 22880 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.112928 23457 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:14:39.149847 22912 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:39.135708 23466 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:39.136013 22915 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:39.108160 30020 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.106045 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.137276 29846 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:39.140276 22835 rpcz_store.cc:269] 0709 13:14:37.378727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.016509 (+637782us) service_pool.cc:224] Handling call
0709 13:14:39.125413 (+1108904us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54000c62d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:39.127104 (+  1691us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995033344}
I20260709 13:14:39.137768 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.164610 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.144049 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.086752 30040 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.089174 29918 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:39.095599 29950 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.144169 30014 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:14:39.095623 30005 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.039641 30067 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:39.162060 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75122.4us
W20260709 13:14:39.167014 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.119277 30042 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:39.084725 30023 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.176594 30022 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:39.175944 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.174518 30040 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.175758 29989 raft_consensus.cc:2764] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
W20260709 13:14:39.192016 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 146769us
I20260709 13:14:39.192360 22758 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:39.043454 30046 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.170683 29846 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:39.170938 30018 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:39.163430 30045 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.203959 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.189311 29877 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:39.222662 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.154934 30082 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.139308 30000 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.205297 30022 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:39.165728 30044 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.201659 30042 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:39.192524 29950 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.226069 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.210585 30014 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:39.242735 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.246272 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:39.228641 30013 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.230121 29653 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:39.230440 29992 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:39.208751 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 120907us
I20260709 13:14:39.239099 30022 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.233088 30004 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:14:39.247810 30014 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:39.260335 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 169087us
I20260709 13:14:39.265704 23432 leader_election.cc:400] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 25. Current term is 26.
I20260709 13:14:39.262826 29933 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:39.252743 29653 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.247284 30052 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:39.251281 30078 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:39.272289 30126 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:14:39.256670 30134 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:39.275645 30000 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.276602 30060 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.280130 29988 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.267838 30014 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.261345 30013 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:39.270961 23432 leader_election.cc:403] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:39.280942 29653 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.272814 30046 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.284504 29870 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.303830 29950 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.307468 30023 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:39.309791 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2280) took 1961 ms (client timeout 2295 ms). Trace:
I20260709 13:14:39.331329 30013 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.311095 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2266) took 2138 ms. Trace:
I20260709 13:14:39.338037 30013 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.338310 22839 rpcz_store.cc:276] 0709 13:14:37.173074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.174060 (+   986us) service_pool.cc:224] Handling call
0709 13:14:37.174351 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:38.368080 (+1193729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.310776 (+942696us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001bbf54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:39.311002 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2624224512}
I20260709 13:14:39.339929 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:39.337647 22832 rpcz_store.cc:269] 0709 13:14:37.348492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.351161 (+  2669us) service_pool.cc:224] Handling call
0709 13:14:39.309532 (+1958371us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540027f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:39.309707 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058208128}
I20260709 13:14:39.341917 30078 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.342828 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.313920 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1274) took 2158 ms. Trace:
I20260709 13:14:39.345739 23462 rpcz_store.cc:276] 0709 13:14:37.155112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.155532 (+   420us) service_pool.cc:224] Handling call
0709 13:14:37.642349 (+486817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:37.642591 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.642619 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:37.642665 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:37.642697 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.642766 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.313614 (+1670848us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400740c50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:39.313834 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118728448}
W20260709 13:14:39.314227 30128 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:39.331106 29988 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:39.348122 30041 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.629s	user 0.006s	sys 0.001s
I20260709 13:14:39.350234 30044 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.350800 29988 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:39.351120 30067 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:39.351807 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3400) took 3280 ms. Trace:
I20260709 13:14:39.351962 22906 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:39.352326 22757 rpcz_store.cc:276] 0709 13:14:36.071510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.071709 (+   199us) service_pool.cc:224] Handling call
0709 13:14:36.095328 (+ 23619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:36.307104 (+211776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.307164 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:36.316192 (+  9028us) log.cc:844] Serialized 26 byte log entry
0709 13:14:36.427733 (+111541us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:36.478368 (+ 50635us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.491233 (+ 12865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.491325 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:37.718666 (+1227341us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b18003b1880. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:39.347488 (+1628822us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:39.351524 (+  4036us) raft_consensus.cc:1685] finished
0709 13:14:39.351554 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.351732 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18129,"prepare.queue_time_us":50302,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":7317,"raft.queue_time_us":297974,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":10225,"spinlock_wait_cycles":189528192,"thread_start_us":282868,"threads_started":2,"wal-append.queue_time_us":194696}
I20260709 13:14:39.356693 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.357537 22757 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:39.358646 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.359789 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.325899 30134 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:39.360636 30074 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.361537 30134 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:39.327648 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.280050 30026 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.286166 29933 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:39.331763 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 66264.3us
I20260709 13:14:39.260078 29992 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.304932 30052 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.377060 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:14:39.332163 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.332846 30142 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 25. Current term is 26.
I20260709 13:14:39.336464 29870 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.312945 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.386229 22760 raft_consensus.cc:2410] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:14:39.386533 30074 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.315615 30082 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.355199 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.387565 22766 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:39.388463 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.370225 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.389488 22915 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.370138 22906 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:39.387149 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.392580 22760 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:39.393427 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.394115 22760 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:39.395326 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:39.395793 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.345196 22832 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:39.372179 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.396633 22914 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:39.397840 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.371137 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:39.328889 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 59284.2us
W20260709 13:14:39.330462 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 106141us
I20260709 13:14:39.399509 22839 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:39.372442 22908 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:39.383927 30123 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:39.386550 29992 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.387113 29870 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.402575 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:39.382372 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.405525 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.410113 23466 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:39.410430 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.411679 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.412382 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.413450 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.413581 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.413431 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:39.414708 22839 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:39.415952 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.417737 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.418375 23466 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:39.415707 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:39.419317 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.420128 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.422034 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.423609 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.423731 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.424738 22914 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:39.425704 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.426992 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.427831 22832 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:39.428965 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.424618 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.430009 22908 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:39.432617 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.433624 22839 raft_consensus.cc:2376] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20. Re-sending same reply.
I20260709 13:14:39.433668 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.437393 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.439857 22759 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:14:39.440922 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.441658 22832 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:39.447840 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.454494 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.455438 23466 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:39.425485 29812 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:39.458314 29812 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:39.459326 29812 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.407152 30123 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.462292 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.466734 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.467690 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.437736 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.424751 29988 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:39.473997 22760 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:39.423193 30067 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:39.479118 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.438825 30023 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:39.483187 22760 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:14:39.479712 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.438931 29696 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:39.492267 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.487885 22832 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:39.492997 22839 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:39.494135 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.494433 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:39.494853 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.494431 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.498819 22760 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:14:39.499384 22760 pending_rounds.cc:85] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 12
I20260709 13:14:39.499680 22760 pending_rounds.cc:107] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:14:39.499986 22760 raft_consensus.cc:2889] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.491966 29696 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.493100 29989 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:39.461809 30123 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.503093 22879 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.495693 22909 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:39.501932 29696 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.504266 22910 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:14:39.505096 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.509886 22832 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:39.505928 22839 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:39.514734 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.515827 22832 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.516188 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.516806 22839 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:39.480824 29988 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:39.489905 30023 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.481904 30067 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.502017 29989 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.519315 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.493249 29926 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.524745 22832 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:39.505420 30148 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:39.545877 30148 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:39.546409 30148 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:39.512605 30018 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:39.550103 29926 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.523007 29988 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:39.527204 30023 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.562397 30040 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.543429 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.571642 22839 raft_consensus.cc:2410] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:39.527906 30067 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.531993 29891 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:39.571743 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:39.592929 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.601203 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.602234 22832 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:39.603009 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.604144 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.607095 23459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 1018) took 1016 ms. Trace:
I20260709 13:14:39.607510 23459 rpcz_store.cc:276] 0709 13:14:38.590212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.590355 (+   143us) service_pool.cc:224] Handling call
0709 13:14:38.594305 (+  3950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.594551 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:14:38.828581 (+234030us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.606575 (+777994us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b1800201dc0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287
0709 13:14:39.606732 (+   157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.606786 (+    54us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:39.606839 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.606915 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.607032 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76,"prepare.queue_time_us":344674,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":66038,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":28403,"spinlock_wait_cycles":9018240,"thread_start_us":338062,"threads_started":2,"wal-append.queue_time_us":11494}
I20260709 13:14:39.505679 29933 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.618485 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2737) took 2238 ms. Trace:
I20260709 13:14:39.618881 22835 rpcz_store.cc:276] 0709 13:14:37.379941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.174847 (+1794906us) service_pool.cc:224] Handling call
0709 13:14:39.175292 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:39.251892 (+ 76600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.251928 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:39.252549 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 13:14:39.425185 (+172636us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:39.457133 (+ 31948us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.457371 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.457451 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.618264 (+160813us) raft_consensus.cc:1685] finished
0709 13:14:39.618290 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.618426 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32308,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":96677,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":42134,"thread_start_us":205408,"threads_started":2,"wal-append.queue_time_us":259005}
I20260709 13:14:39.617260 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.532557 29653 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.543233 29989 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.610936 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.626673 22839 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:39.621531 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.628770 22832 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:39.629853 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:39.630395 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 127390us
I20260709 13:14:39.634677 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.635478 22835 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:39.601593 29891 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:39.616273 30026 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.659685 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2886) took 1071 ms. Trace:
I20260709 13:14:39.639935 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.660215 22762 rpcz_store.cc:276] 0709 13:14:38.588320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.588578 (+   258us) service_pool.cc:224] Handling call
0709 13:14:39.659463 (+1070885us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002224d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:39.659608 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383835008}
I20260709 13:14:39.662307 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.663447 22762 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.657881 22915 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.631083 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.660077 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.666550 22834 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
W20260709 13:14:39.667157 22834 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1252) took 1927 ms (client timeout 1691 ms). Trace:
W20260709 13:14:39.667623 22834 rpcz_store.cc:269] 0709 13:14:37.739568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.834897 (+ 95329us) service_pool.cc:224] Handling call
0709 13:14:39.659897 (+1825000us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6ab70a0 00007f68a6ab6984 00007f68a00316d4 00007f68a0031662
0709 13:14:39.667073 (+  7176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68686080}
I20260709 13:14:39.669157 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.663041 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.681618 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.686760 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.687150 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.688012 22832 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:39.686683 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.658258 30040 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:39.688558 30085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.010s	sys 0.012s
I20260709 13:14:39.632628 30013 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:39.702709 22837 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2734) took 2323 ms (client timeout 1779 ms). Trace:
W20260709 13:14:39.712064 22837 rpcz_store.cc:269] 0709 13:14:37.378991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.402295 (+ 23304us) service_pool.cc:224] Handling call
0709 13:14:39.702436 (+2300141us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b540028a0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:39.702629 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321486592}
I20260709 13:14:39.691193 29812 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:39.693631 30006 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:39.714579 22832 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 13:14:39.706674 30085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.011s	sys 0.012s
I20260709 13:14:39.718087 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2740) took 2334 ms. Trace:
I20260709 13:14:39.718513 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.718567 22832 rpcz_store.cc:276] 0709 13:14:37.384067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.707369 (+2323302us) service_pool.cc:224] Handling call
0709 13:14:39.707853 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:39.717680 (+  9827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.718003 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.719511 22835 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:39.660463 30014 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.739923 30013 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.716996 30006 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.741564 30006 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.742057 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2879) took 1156 ms. Trace:
I20260709 13:14:39.742460 22763 rpcz_store.cc:276] 0709 13:14:38.586088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.586251 (+   163us) service_pool.cc:224] Handling call
0709 13:14:38.586601 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:38.634250 (+ 47649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.741756 (+1107506us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b54000816d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:39.741992 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57485696}
I20260709 13:14:39.743924 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:39.745887 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.746666 22837 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.747476 29653 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.748190 29653 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:39.749621 29926 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.747473 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.752229 22837 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:39.753166 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.754319 29996 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:39.754753 29996 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:39.720860 30156 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:39.758936 22837 raft_consensus.cc:1217] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Deduplicated request from leader. Original: 12.10->[13.11-15.12]   Dedup: 13.11->[15.12-15.12]
I20260709 13:14:39.759917 22837 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:14:39.760038 22907 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.760758 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2746) took 2323 ms. Trace:
I20260709 13:14:39.761269 22839 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:39.761611 22837 rpcz_store.cc:276] 0709 13:14:37.437288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.755605 (+2318317us) service_pool.cc:224] Handling call
0709 13:14:39.758647 (+  3042us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:39.760392 (+  1745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.760674 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.723588 22757 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:14:39.764915 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.766036 23462 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:39.765239 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.779798 23459 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:39.803613 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:39.804790 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.806825 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.812803 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.828296 22837 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:14:39.831377 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 144715us
I20260709 13:14:39.808202 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.834946 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2750) took 2306 ms. Trace:
I20260709 13:14:39.835395 22837 rpcz_store.cc:276] 0709 13:14:37.528475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.762611 (+2234136us) service_pool.cc:224] Handling call
0709 13:14:39.806965 (+ 44354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.832838 (+ 25873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.834883 (+  2045us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.836010 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:39.838052 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:39.838698 23459 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:39.757186 29891 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.817147 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:39.838536 22906 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:39.843703 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:39.843905 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:39.840554 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.840525 22839 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:39.845026 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2748) took 2340 ms. Trace:
I20260709 13:14:39.845439 22839 rpcz_store.cc:276] 0709 13:14:37.504943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.762333 (+2257390us) service_pool.cc:224] Handling call
0709 13:14:39.765614 (+  3281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.844748 (+ 79134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.844967 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.846455 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.846719 22762 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:39.848661 22762 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:39.849395 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.852320 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:39.847010 22839 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:39.854923 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.832041 30153 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:14:39.801888 30156 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:39.861970 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:39.860723 23462 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 23. Current term is 24. Ops: [23.12-23.12]
W20260709 13:14:39.839803 30022 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.575s	user 0.015s	sys 0.003s
I20260709 13:14:39.873991 30022 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.875514 30018 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:39.876283 30022 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:39.867549 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.864009 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:39.879690 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:39.856865 30042 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.002s
I20260709 13:14:39.863811 30028 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:39.870427 30014 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:39.882431 30042 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:39.872040 29846 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.004s
I20260709 13:14:39.866480 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1257) took 2121 ms. Trace:
I20260709 13:14:39.882906 29988 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.828868 29996 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.883441 29846 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:39.884006 29996 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:39.884781 30042 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.885092 29988 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:39.831676 29950 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:39.885705 22913 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.886135 29950 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:39.828569 29926 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:14:39.831792 30074 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:39.887336 30074 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:39.887943 29846 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:39.886595 22834 raft_consensus.cc:2393] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 25: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:39.881368 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.859768 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:39.886528 29950 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:39.853116 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:39.883541 22835 rpcz_store.cc:276] 0709 13:14:37.745405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.840403 (+2094998us) service_pool.cc:224] Handling call
0709 13:14:39.865940 (+ 25537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.866148 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.866175 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.866234 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:39.866268 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.866324 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.866420 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.894774 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.898924 22906 raft_consensus.cc:2393] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:39.903048 22839 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:39.905965 22760 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:39.906776 23463 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:14:39.903185 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:39.894488 22807 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.910324 22837 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:14:39.910380 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50700.9us
I20260709 13:14:39.848270 30052 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:39.861066 30044 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:39.913923 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2289) took 2140 ms. Trace:
I20260709 13:14:39.921311 22837 rpcz_store.cc:276] 0709 13:14:37.773696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.909657 (+2135961us) service_pool.cc:224] Handling call
0709 13:14:39.909967 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.910803 (+   836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.913863 (+  3060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.923208 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1265) took 1964 ms. Trace:
I20260709 13:14:39.923592 22837 rpcz_store.cc:276] 0709 13:14:37.958782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.922299 (+1963517us) service_pool.cc:224] Handling call
0709 13:14:39.922648 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.922848 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.922873 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.922917 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:39.922967 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.923022 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.923147 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.919710 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:39.925105 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:39.925698 22835 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:39.925985 22762 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:39.927261 23462 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:39.927364 22762 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:14:39.922672 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:39.927534 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:39.913468 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1260) took 2167 ms. Trace:
I20260709 13:14:39.933605 22765 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:39.934370 22762 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:39.933887 22839 rpcz_store.cc:276] 0709 13:14:37.746467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.905344 (+2158877us) service_pool.cc:224] Handling call
0709 13:14:39.905653 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.911659 (+  6006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.911698 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.913210 (+  1512us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:39.913252 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.913316 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.913413 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:39.913163 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.897653 29846 raft_consensus.cc:2764] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:14:39.956543 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:39.983896 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:39.975720 29812 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:39.928342 30074 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:40.090291 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 894196us
I20260709 13:14:40.008056 22834 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:40.002257 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2738) took 2618 ms. Trace:
I20260709 13:14:40.096457 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:39.954984 29823 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:39.992548 23462 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:40.022637 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.010661 22837 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:39.929217 29870 raft_consensus.cc:2764] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:14:40.103648 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.056833 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:40.104939 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 82399.6us
W20260709 13:14:40.081372 30023 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.005s
I20260709 13:14:39.932667 30044 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:40.107546 29989 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.107407 30164 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:40.108767 30164 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:40.109205 30164 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.110795 22909 raft_consensus.cc:2368] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:40.108208 29846 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:40.112646 29989 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:40.055985 29942 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:14:40.114408 29846 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:40.114810 29846 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:40.115676 29846 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.116763 29846 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:40.050153 22912 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:40.119236 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2292) took 1911 ms. Trace:
I20260709 13:14:40.119634 22834 rpcz_store.cc:276] 0709 13:14:38.208266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.007527 (+1799261us) service_pool.cc:224] Handling call
0709 13:14:40.007845 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.118934 (+111089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.119171 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:40.097388 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2907) took 1022 ms. Trace:
W20260709 13:14:40.007277 30160 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:14:40.055087 30148 consensus_meta.cc:287] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:14:39.921030 29926 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:40.129835 22835 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:40.140897 30045 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:40.009526 30156 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.139251 29989 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:40.150151 30123 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.631s	user 0.007s	sys 0.006s
I20260709 13:14:40.150692 30123 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.124197 30044 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:40.154664 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 232074us
I20260709 13:14:40.156396 22807 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.156625 23459 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
W20260709 13:14:40.131075 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 219379us
I20260709 13:14:40.132622 23463 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:40.158982 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:40.133309 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.096801 22836 rpcz_store.cc:276] 0709 13:14:37.383538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.250547 (+1867009us) service_pool.cc:224] Handling call
0709 13:14:39.250931 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:39.251310 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.251338 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:39.251822 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:14:39.374723 (+122901us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b18004a6e00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:14:39.502563 (+127840us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:39.502607 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.502672 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.990747 (+488075us) raft_consensus.cc:1685] finished
0709 13:14:39.990785 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.002164 (+ 11379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":373,"raft.queue_time_us":19039,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":8394,"spinlock_wait_cycles":11642240,"thread_start_us":127638,"threads_started":1,"wal-append.queue_time_us":336402}
I20260709 13:14:40.009651 30040 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:40.136864 22765 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:14:40.176708 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:40.177764 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.179946 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.187522 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:40.160962 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:40.162135 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2294) took 1953 ms. Trace:
I20260709 13:14:40.108201 30023 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.194984 30045 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:40.195570 22908 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:40.196933 30023 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:40.173733 30074 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:40.174463 29812 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.198078 30122 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:40.198865 30122 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.197739 30074 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.202083 30074 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:40.103627 29823 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:40.196964 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:40.206117 29823 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:40.206524 30045 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:40.194613 22836 rpcz_store.cc:276] 0709 13:14:38.208719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.160710 (+1951991us) service_pool.cc:224] Handling call
0709 13:14:40.161210 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.161473 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:40.161536 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.161606 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:40.161658 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.161740 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.162044 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:40.210952 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:40.211366 22836 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:40.211897 30045 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:40.212488 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:40.217877 30045 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:40.218374 30045 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:40.219362 30122 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.232520 29823 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.235766 22836 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:14:40.345213 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.243563 30045 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.010303 29696 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:40.248112 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60705.8us
I20260709 13:14:40.361187 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.362933 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.184691 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:40.176162 30040 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.180910 29989 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:40.181654 29926 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:40.187562 29891 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:39.990578 29950 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.154220 29942 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.128s
I20260709 13:14:40.138986 30148 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.253038 29996 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:40.287253 30162 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:40.176157 30164 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.285537 30123 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:40.282579 30015 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:40.282850 30163 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:40.188638 30006 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:40.335991 30067 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.735s	user 0.010s	sys 0.003s
W20260709 13:14:40.339239 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 365877us
W20260709 13:14:40.358532 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 234660us
I20260709 13:14:40.331046 29846 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:40.245633 30074 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.375041 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:40.449996 22764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 1098) took 2171 ms (client timeout 2162 ms). Trace:
I20260709 13:14:40.358626 30023 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:40.337438 29892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.003s	sys 0.032s
I20260709 13:14:40.422771 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2731) took 3062 ms. Trace:
I20260709 13:14:40.384922 29988 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:40.456875 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:40.467753 22764 rpcz_store.cc:269] 0709 13:14:38.248900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.249022 (+   122us) service_pool.cc:224] Handling call
0709 13:14:40.420220 (+2171198us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b5400368b54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.420455 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268649856}
I20260709 13:14:40.421950 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2724) took 3063 ms. Trace:
I20260709 13:14:40.474851 22833 rpcz_store.cc:276] 0709 13:14:37.378127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.030392 (+1652265us) service_pool.cc:224] Handling call
0709 13:14:39.030832 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.046781 (+ 15949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.422466 (+1375685us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b54000c6cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:40.422688 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26828160}
I20260709 13:14:40.477546 22840 rpcz_store.cc:276] 0709 13:14:37.358833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.980488 (+1621655us) service_pool.cc:224] Handling call
0709 13:14:39.033217 (+ 52729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.040889 (+  7672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.421669 (+1380780us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b540025ec50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.421864 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119045504}
I20260709 13:14:40.420912 22761 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:40.479226 22840 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:40.418264 29950 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:40.480885 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 106083us
I20260709 13:14:40.371536 30040 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.450858 30163 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:40.454416 29891 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:40.483881 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 271451us
I20260709 13:14:40.384658 29653 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:40.455128 29989 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:40.486940 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:40.476038 29996 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:40.474673 29892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.003s	sys 0.032s
I20260709 13:14:40.482656 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.477265 30023 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.467410 30067 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.471376 30162 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.489878 30163 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:40.530740 29823 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:40.494362 29926 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.534677 30164 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.535297 29942 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:14:40.540000 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2881) took 1953 ms. Trace:
I20260709 13:14:40.540441 22761 rpcz_store.cc:276] 0709 13:14:38.586743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.588950 (+  2207us) service_pool.cc:224] Handling call
0709 13:14:40.420620 (+1831670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.482424 (+ 61804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.539754 (+ 57330us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b5400368b50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.539926 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4211806080}
I20260709 13:14:40.541123 29696 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.535475 29653 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:40.542459 29653 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:40.542870 29653 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:40.543102 30074 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:40.543860 29653 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.545161 29653 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.491250 29891 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.547435 29891 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.492033 30169 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 13:14:40.548870 22757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2902) took 1713 ms (client timeout 2244 ms). Trace:
W20260709 13:14:40.549440 22757 rpcz_store.cc:269] 0709 13:14:38.834979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.359548 (+524569us) service_pool.cc:224] Handling call
0709 13:14:40.548634 (+1189086us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400310e54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.548794 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67160448}
I20260709 13:14:40.497857 29846 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:40.554273 29988 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:14:40.559291 30067 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.559831 30044 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.517324 29996 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:40.493345 29989 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.573226 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:40.573596 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 388970us
I20260709 13:14:40.574926 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.570314 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3441) took 1303 ms. Trace:
I20260709 13:14:40.579805 22766 rpcz_store.cc:276] 0709 13:14:39.267306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.904283 (+636977us) service_pool.cc:224] Handling call
0709 13:14:39.904465 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.904602 (+   137us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:40.082282 (+177680us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.569771 (+487489us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.569854 (+    83us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:40.569952 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.570027 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.570208 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":97557,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":77027,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"thread_start_us":639116,"threads_started":3}
I20260709 13:14:40.592193 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.569977 30175 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:40.554937 29846 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.597108 30175 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:40.601157 22879 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.018s	sys 0.045s
W20260709 13:14:40.610247 22879 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.018s	sys 0.045s
W20260709 13:14:40.610774 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 773904us
I20260709 13:14:40.600678 29999 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:40.561435 29891 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:14:40.609115 29846 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.607664 30067 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.579907 30170 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:14:40.608932 30045 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:40.629374 29989 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.608939 30179 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:40.630370 30044 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:40.601223 29988 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:14:40.620609 30162 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.634148 30162 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:40.634521 30162 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:40.635556 30162 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.637341 30162 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:40.637647 29992 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:40.638195 29992 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:40.639256 29992 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.627399 30175 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.607471 29996 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 23.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:14:40.651376 30148 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:40.666412 30167 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.344s	user 0.001s	sys 0.001s
W20260709 13:14:40.699564 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.701349 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.703905 22762 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:40.706179 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 221058us
I20260709 13:14:40.711828 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2293) took 2503 ms. Trace:
W20260709 13:14:40.706457 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 249142us
I20260709 13:14:40.714413 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:40.719337 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 127130us
W20260709 13:14:40.604674 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.724865 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.712410 22834 rpcz_store.cc:276] 0709 13:14:38.208505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.120617 (+1912112us) service_pool.cc:224] Handling call
0709 13:14:40.121088 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:40.121487 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:40.133068 (+ 11581us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:40.139194 (+  6126us) log.cc:844] Serialized 26 byte log entry
0709 13:14:40.710757 (+571563us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:40.710901 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.711098 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:40.711210 (+   112us) raft_consensus.cc:1685] finished
0709 13:14:40.711226 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.711516 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16898,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":18666,"thread_start_us":423432,"threads_started":1,"wal-append.queue_time_us":410896}
I20260709 13:14:40.726493 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:40.720701 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:40.738025 29874 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:40.741485 29874 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:40.747984 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2755) took 2748 ms. Trace:
I20260709 13:14:40.717330 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:40.736056 29823 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:40.750519 29823 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:40.751976 29823 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:40.752605 29823 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:40.759305 29999 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.760867 29999 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:40.762607 22807 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.765617 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3688) took 1391 ms. Trace:
I20260709 13:14:40.766821 29823 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.766438 22908 rpcz_store.cc:276] 0709 13:14:39.374218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.431873 (+ 57655us) service_pool.cc:224] Handling call
0709 13:14:39.432114 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.195982 (+763868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.764589 (+568607us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b540010dbd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:40.765275 (+   686us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1755364736}
W20260709 13:14:40.773139 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 56141.8us
I20260709 13:14:40.767793 29999 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:40.773397 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3427) took 1998 ms. Trace:
I20260709 13:14:40.774854 22760 rpcz_store.cc:276] 0709 13:14:38.774707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.511922 (+737215us) service_pool.cc:224] Handling call
0709 13:14:39.517569 (+  5647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.909196 (+391627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.772575 (+863379us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400368654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:40.773070 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892764032}
W20260709 13:14:40.750516 22838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2736) took 3370 ms (client timeout 1942 ms). Trace:
I20260709 13:14:40.748672 22837 rpcz_store.cc:276] 0709 13:14:37.999167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.924525 (+1925358us) service_pool.cc:224] Handling call
0709 13:14:40.010440 (+ 85915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.103997 (+ 93557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.747390 (+643393us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54001bba50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.747731 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196699008}
W20260709 13:14:40.776940 22838 rpcz_store.cc:269] 0709 13:14:37.379654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.417216 (+ 37562us) service_pool.cc:224] Handling call
0709 13:14:40.749959 (+3332743us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.750218 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2133994752}
I20260709 13:14:40.781764 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:40.783298 22908 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:40.786409 22910 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:40.786511 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:40.788693 22758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 2906) took 1722 ms (client timeout 1831 ms). Trace:
I20260709 13:14:40.814481 22914 raft_consensus.cc:1240] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:40.816501 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:40.817823 22838 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.788793 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:40.807500 22758 rpcz_store.cc:269] 0709 13:14:39.066279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.110428 (+ 44149us) service_pool.cc:224] Handling call
0709 13:14:40.788239 (+1677811us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5400222ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.788427 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61004032}
I20260709 13:14:40.814813 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.835599 29874 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:14:40.855252 30166 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.524s	user 0.005s	sys 0.006s
I20260709 13:14:40.857897 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3682) took 1486 ms. Trace:
I20260709 13:14:40.858399 30169 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:40.859903 30183 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:40.860735 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2291) took 2652 ms. Trace:
I20260709 13:14:40.862462 30023 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:40.854272 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:40.866377 22914 raft_consensus.cc:2376] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21. Re-sending same reply.
I20260709 13:14:40.121970 22763 rpcz_store.cc:276] 0709 13:14:39.074867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.762671 (+687804us) service_pool.cc:224] Handling call
0709 13:14:39.806181 (+ 43510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.806473 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:39.856059 (+ 49586us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.096776 (+240717us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.096986 (+   210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.097080 (+    94us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:40.097122 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.097191 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.097319 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23333,"prepare.queue_time_us":54305,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3514,"raft.queue_time_us":5312,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":72009728,"thread_start_us":239930,"threads_started":1}
I20260709 13:14:40.873687 30054 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:40.874847 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:40.877619 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.875974 29992 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:40.877964 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.879293 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:40.880789 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3694) took 1459 ms. Trace:
I20260709 13:14:40.881351 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:40.882819 22839 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.649488 30015 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:40.885337 30015 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.886034 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:40.888159 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:40.889520 22758 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:40.858489 22912 rpcz_store.cc:276] 0709 13:14:39.371413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.371547 (+   134us) service_pool.cc:224] Handling call
0709 13:14:40.010844 (+639297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.118702 (+107858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.857247 (+738545us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54002111d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.857619 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469612800}
I20260709 13:14:40.890923 29989 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:40.891526 29989 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:40.892577 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3684) took 1519 ms. Trace:
I20260709 13:14:40.893146 29989 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 23.9, Last appended by leader: 9, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.895499 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:40.895979 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:40.896657 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:40.881289 22915 rpcz_store.cc:276] 0709 13:14:39.417956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.424568 (+  6612us) service_pool.cc:224] Handling call
0709 13:14:40.877105 (+1452537us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540020b2d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.877412 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435331840}
W20260709 13:14:40.858603 30193 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:14:40.914430 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:40.915059 23462 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.915653 22910 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:40.893077 22907 rpcz_store.cc:276] 0709 13:14:39.372836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.372957 (+   121us) service_pool.cc:224] Handling call
0709 13:14:40.892088 (+1519131us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54003c96d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:40.892367 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805092608}
I20260709 13:14:40.922112 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:40.886054 30186 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:14:40.927577 30197 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:40.928282 30186 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:40.928804 30186 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:40.851498 29874 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:40.926119 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:40.861244 22835 rpcz_store.cc:276] 0709 13:14:38.208046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.982433 (+1774387us) service_pool.cc:224] Handling call
0709 13:14:40.006308 (+ 23875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.140571 (+134263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.860062 (+719491us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54001bd354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:40.860474 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283462528}
W20260709 13:14:40.931914 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.931690 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:40.932569 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:40.931553 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:40.944293 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:40.945459 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:40.930660 29837 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:40.946404 22761 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:40.947050 29837 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.948168 23459 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:40.948499 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.950229 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:40.951072 22764 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.934235 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:40.954149 22766 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.954866 22765 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 13:14:40.931170 30068 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:40.958458 30068 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.970116 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:40.970504 29997 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:40.971489 22907 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:14:40.967036 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:40.975477 22656 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 11 to 13, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 13 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:14:40.967023 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 152427us
I20260709 13:14:40.978312 30006 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:40.929479 23432 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.655s (SENT)
I20260709 13:14:40.981489 29846 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:40.983543 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 776baf137f8745dd866379ca15ab8cee from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:40.985598 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.986254 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:40.987219 22907 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:14:40.987344 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58139.9us
I20260709 13:14:40.988826 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:40.989950 29696 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:40.991763 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:40.991991 30163 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:40.995066 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:40.995994 22763 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:40.966602 22839 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:40.999323 22758 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 13:14:41.000146 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.933632 29874 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.011560 29874 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.982967 30020 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:41.016280 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.016965 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:41.017122 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.018080 22835 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 17: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:41.018333 22837 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:41.019927 22915 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:41.021353 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:41.023536 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.024868 22837 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:14:40.982062 29837 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.026652 22758 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:41.026949 30186 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.027817 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:41.028317 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.029001 30186 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:41.029322 22758 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:41.029474 23430 connection.cc:256] RPC call timeout handler was delayed by -1.1624086839999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
I20260709 13:14:41.030433 30006 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:41.030957 30006 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:41.031399 30006 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:41.031828 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:41.032873 23430 connection.cc:256] RPC call timeout handler was delayed by -1.0238086839999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 13:14:41.018352 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 531409us
I20260709 13:14:41.033871 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.034180 30068 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:41.035102 30068 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:41.037298 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:41.037704 30179 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:41.038272 22837 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:41.037923 29653 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:41.039103 29653 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:41.039608 29846 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.008253 30067 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:41.046629 22760 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:41.048434 22760 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:41.050403 30163 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:41.050834 30158 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.313s	user 0.005s	sys 0.003s
W20260709 13:14:41.051628 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 265386us
I20260709 13:14:41.052553 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.053488 29846 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:41.054005 29846 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:41.054445 29846 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:41.059204 22907 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:41.067093 29717 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:41.059760 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1114) took 1642 ms. Trace:
I20260709 13:14:41.062124 23462 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:41.040529 29653 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.072188 23459 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:41.077931 22837 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:41.043349 22764 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:40.907161 29989 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:41.086699 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:41.088900 22838 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:41.067883 22762 rpcz_store.cc:276] 0709 13:14:39.417336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.935091 (+517755us) service_pool.cc:224] Handling call
0709 13:14:39.935357 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.705207 (+769850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.059432 (+354225us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54000028d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:41.059688 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765881728}
I20260709 13:14:41.094743 30005 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:41.079442 29696 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.095252 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:41.095582 30005 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.095748 29696 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:41.125634 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.125190 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:41.127367 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43998.2us
I20260709 13:14:41.136312 22913 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:41.128027 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.152380 22763 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:41.155544 22766 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:41.157588 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:41.158103 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.151023 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:41.148380 29653 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:41.158350 22757 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 14.
I20260709 13:14:41.165910 22759 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:41.168646 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2887) took 2580 ms. Trace:
W20260709 13:14:41.169803 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 117385us
I20260709 13:14:41.169986 22913 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:41.171613 30170 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:14:41.171805 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.169018 22759 rpcz_store.cc:276] 0709 13:14:38.588546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.454099 (+865553us) service_pool.cc:224] Handling call
0709 13:14:39.454448 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:41.166470 (+1712022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.168291 (+  1821us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400311354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:41.168573 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3926427904}
W20260709 13:14:41.178148 30040 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.007s
I20260709 13:14:41.178665 30040 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.183205 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.184597 22913 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:41.185545 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:41.186594 22912 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:41.186257 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.192111 22915 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:14:41.184756 22907 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:41.193990 30005 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.187131 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.200563 22910 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:41.186165 29696 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:41.221166 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:41.222560 22757 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:41.228928 22765 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:41.238358 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.244014 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.233243 30040 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:41.251423 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.290119 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 162168us
I20260709 13:14:41.290424 23459 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:41.295179 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.301002 30204 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:41.301581 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 313004us
I20260709 13:14:41.301894 29846 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.302999 29696 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:14:41.305619 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.361325 29696 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:41.306155 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 181108us
I20260709 13:14:41.361819 29696 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:41.362525 23430 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.307371 22910 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:41.366454 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.367424 22910 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:41.309650 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:41.310871 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.312762 30005 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.314018 22765 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:41.369776 22910 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:41.318401 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:41.321317 22761 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:41.322197 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:41.321960 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.323834 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.322211 22914 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:41.325336 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 153723us
I20260709 13:14:41.169806 22656 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:41.289796 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:41.299907 30006 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.376322 22763 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 24: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:41.378046 30148 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.379034 30205 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:41.378815 30148 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.380185 30005 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.381453 30013 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:41.382547 30013 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.382956 22910 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:14:41.383632 22765 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:14:41.383987 29892 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:14:41.385001 30156 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.385236 29892 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:41.385761 29892 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:41.386072 30156 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.386897 22758 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:14:41.387027 29892 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.388468 29892 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:41.388967 30006 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:41.389479 30029 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.390111 22914 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:41.390443 22762 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:41.390082 30029 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.331226 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:41.331326 29943 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.358132 30204 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:41.394158 30204 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:41.393931 29943 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.394709 30029 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:41.334466 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:41.335964 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:41.336730 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:41.327031 30015 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:41.238510 30068 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:14:41.347256 22915 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:41.233582 29992 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:41.356978 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111900us
W20260709 13:14:41.358351 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.375222 23458 raft_consensus.cc:2368] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:41.393112 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:41.399448 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.404335 22912 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:41.415995 22841 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:41.416311 22908 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:41.417851 22760 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:41.416455 29823 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:14:41.403713 29943 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:41.424796 22911 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:41.424173 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.425758 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.431849 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:41.442965 22841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2754) took 3648 ms (client timeout 1728 ms). Trace:
I20260709 13:14:41.434216 29653 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:14:41.444268 30013 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:41.446714 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:41.443616 22841 rpcz_store.cc:269] 0709 13:14:37.794723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.083229 (+288506us) service_pool.cc:224] Handling call
0709 13:14:41.442659 (+3359430us) spinlock_profiling.cc:243] Waited 3.23 s on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:41.442853 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7432462592}
I20260709 13:14:41.449239 22911 raft_consensus.cc:2368] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:41.451098 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.416106 30015 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.457377 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:41.464732 22762 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:41.453231 30175 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:41.480882 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.481602 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.482429 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.484647 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92270.3us
I20260709 13:14:41.486474 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:41.487411 23463 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:41.478490 29653 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:14:41.510046 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81328.9us
I20260709 13:14:41.514140 22880 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:41.515288 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.479022 29823 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:14:41.522356 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 102467us
W20260709 13:14:41.526938 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.511152 30013 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:41.525206 22911 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:41.537416 22906 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:41.520519 29653 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:41.556964 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.558676 22906 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:41.560202 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.578015 30205 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:41.580077 30205 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:41.576226 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:41.549083 30013 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.586683 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.588141 30191 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.581615 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:41.591096 29717 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:41.596691 30011 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:41.596899 29717 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.597481 30011 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:41.598425 22915 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:41.598779 30011 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.599472 30005 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:41.599838 29717 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:41.600102 30068 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:41.600685 30068 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:41.601048 30068 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:41.604076 30213 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:41.604492 29943 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:41.605126 29943 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:41.605615 29943 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:41.607563 30013 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:41.588572 30015 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.573738 30206 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:41.628669 22664 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 246) took 93209 ms (client timeout 15000 ms). Trace:
W20260709 13:14:41.629246 22664 rpcz_store.cc:269] 0709 13:13:08.419389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.419667 (+   278us) service_pool.cc:224] Handling call
0709 13:13:08.440495 (+ 20828us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b5400310010. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cd0b2 00007f68a7946a89 00007f68a78deda6 00007f68a78b0088 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc
0709 13:14:41.628373 (+93187878us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:30.826448 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:30.827733 (+  1285us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:30.827838 (+   105us) write_op.cc:435] Acquired schema lock
0709 13:14:30.827867 (+    29us) tablet.cc:662] Decoding operations
0709 13:14:30.829030 (+  1163us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:30.829182 (+   152us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:30.829235 (+    53us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:14:30.937839 (+108604us) tablet.cc:701] Row locks acquired
0709 13:14:30.967391 (+ 29552us) write_op.cc:260] PREPARE: finished
0709 13:14:30.972269 (+  4878us) write_op.cc:270] Start()
0709 13:14:30.990425 (+ 18156us) write_op.cc:276] Timestamp: P: 1783602870972178 usec, L: 0
0709 13:14:30.997116 (+  6691us) op_driver.cc:348] REPLICATION: starting
0709 13:14:31.066096 (+ 68980us) log.cc:844] Serialized 2064 byte log entry
0709 13:14:31.749047 (+682951us) op_driver.cc:464] REPLICATION: finished
0709 13:14:31.988881 (+239834us) write_op.cc:301] APPLY: starting
0709 13:14:31.990102 (+  1221us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:31.994364 (+  4262us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:31.994526 (+   162us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:31.997830 (+  3304us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:31.998091 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:14:31.998166 (+    75us) write_op.cc:312] APPLY: finished
0709 13:14:31.998530 (+   364us) log.cc:844] Serialized 57 byte log entry
0709 13:14:31.999496 (+   966us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:31.999613 (+   117us) write_op.cc:454] Released schema lock
0709 13:14:31.999680 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:32.010212 (+ 10532us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:32.010260 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7066514,"reported_tablets":51,"spinlock_wait_cycles":21947264,"child_traces":[["op",{"apply.queue_time_us":192077,"apply.run_cpu_time_us":9564,"apply.run_wall_time_us":28731,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":233383,"prepare.run_cpu_time_us":71403,"prepare.run_wall_time_us":406338,"raft.queue_time_us":177516,"raft.run_cpu_time_us":11700,"raft.run_wall_time_us":242327,"replication_time_us":716954,"thread_start_us":525499,"threads_started":4,"wal-append.queue_time_us":137785,"wal-append.run_cpu_time_us":16110,"wal-append.run_wall_time_us":1799774}]]}
I20260709 13:14:41.634733 29846 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.582412 30004 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:41.594290 30191 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:41.655341 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104509us
I20260709 13:14:41.658672 30191 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.604527 30213 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:41.670917 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 120005us
I20260709 13:14:41.592471 29696 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:41.674705 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 281702us
I20260709 13:14:41.651587 30004 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:41.693871 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.696102 23430 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:41.668794 30156 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:41.701182 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.700448 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:41.687857 29989 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:14:41.689792 30163 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:14:41.715448 29874 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.716236 29943 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.719092 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:41.719333 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.720381 22763 raft_consensus.cc:2368] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:41.721181 30206 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:41.722184 22760 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:41.723625 30206 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:41.724659 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.704959 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.726796 22841 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:41.721403 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:41.716169 29874 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.724596 30206 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:41.738204 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.739272 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.697115 30162 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.594609 22656 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.700642 30148 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:41.759994 22664 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 246) took 15513 ms (client timeout 15000 ms). Trace:
I20260709 13:14:41.759778 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:41.761018 22664 rpcz_store.cc:269] 0709 13:14:26.233598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.742093 (+15508495us) service_pool.cc:211] Skipping call since client already timed out
0709 13:14:41.747093 (+  5000us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:14:41.675051 29653 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.764230 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:41.770485 30212 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:41.771503 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101924us
W20260709 13:14:41.769289 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.772809 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.777431 22908 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:41.745566 30206 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:41.781229 22806 leader_election.cc:400] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:41.782038 22806 leader_election.cc:403] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:14:41.773798 30212 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.786504 22760 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:41.785305 30204 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.839457 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.840296 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:41.845947 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 80046.1us
W20260709 13:14:41.789358 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.783128 30162 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.783319 30004 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.852218 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.853647 22834 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:41.854396 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.855103 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:41.855300 29943 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:41.889345 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49006.1us
I20260709 13:14:41.857286 30013 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.784915 30212 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:41.865872 22807 leader_election.cc:400] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
I20260709 13:14:41.892881 22807 leader_election.cc:403] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:14:41.874435 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.884002 22838 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
W20260709 13:14:41.884668 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.856062 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:41.861259 30175 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:41.902529 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:41.906764 22837 raft_consensus.cc:2393] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:41.909703 23458 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:41.860610 29823 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:14:41.920977 22841 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:41.864948 30163 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.909837 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:41.930953 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.931712 22837 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
W20260709 13:14:41.911675 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 215623us
W20260709 13:14:41.939105 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.782557 30213 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:41.944396 22839 raft_consensus.cc:2368] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
W20260709 13:14:41.946847 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 378136us
W20260709 13:14:41.948418 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 440873us
I20260709 13:14:41.929224 29943 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17.
W20260709 13:14:41.956728 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.933439 29997 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.965210 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.948235 30156 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:41.945374 30040 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.943557 29696 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:41.911608 30013 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:41.969619 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.967013 22834 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:41.998016 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:41.998808 22838 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:41.999951 22836 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:14:42.002537 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 143325us
I20260709 13:14:42.002717 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.003603 22837 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:42.003981 22807 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:41.949071 30163 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.010645 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:42.015865 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 296889us
I20260709 13:14:42.018368 29992 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:14:42.020165 29992 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:42.021086 29992 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:41.947324 30206 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:42.039021 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.988489 30040 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.048823 29696 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:42.006139 29717 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:14:42.002537 29933 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 13:14:42.021492 29992 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:42.070300 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.079679 29933 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:42.080266 30005 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:42.080775 30005 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:42.081866 30005 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.082471 30162 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:42.083565 30162 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:42.083721 30005 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:42.081626 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 74356.2us
I20260709 13:14:42.007974 29997 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:42.085317 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57300us
I20260709 13:14:42.047209 30213 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:42.080322 29933 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:42.097196 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 104535us
I20260709 13:14:42.084264 30162 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.111234 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:42.112588 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 143124us
I20260709 13:14:42.087550 30204 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:42.052294 30156 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:42.122295 29992 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.128118 22656 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:42.115355 29823 consensus_queue.cc:1243] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:42.173604 29933 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.175341 29933 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:42.120942 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.177601 23460 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:42.178052 22833 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:42.179343 30212 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.176288 23466 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:14:42.139446 22730 leader_election.cc:400] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:42.181020 22730 leader_election.cc:403] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:14:42.182111 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.043938 30191 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:14:42.143784 29997 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.165946 22761 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:42.178821 22656 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:42.133452 30068 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.002s
I20260709 13:14:42.187525 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.202419 22834 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:42.206032 22841 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:42.138150 30162 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:42.210491 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 226335us
I20260709 13:14:42.133694 30227 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.208403 29943 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.135689 30013 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.187197 29823 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.186714 30005 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:42.195555 30234 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:42.210427 29992 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:42.237978 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 148268us
W20260709 13:14:42.242466 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 238621us
I20260709 13:14:42.248275 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:42.261365 30221 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:42.231766 29997 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.266834 29997 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.269184 29997 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.210187 30068 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.237197 30212 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.234164 30234 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:42.175375 30156 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:42.295691 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 96056.4us
W20260709 13:14:42.280802 29892 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.892s	user 0.007s	sys 0.013s
I20260709 13:14:42.288925 22833 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:42.227681 29943 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.227893 30148 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:14:42.271266 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.234547 30162 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:14:42.233201 29823 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:42.329531 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.320681 30206 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.331512 30206 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.332370 22656 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 7 to 10, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 10 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:42.334054 30206 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.334878 30068 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:42.335156 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.335434 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87328.4us
I20260709 13:14:42.335584 30068 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.336167 30068 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.335836 22656 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:42.340610 30068 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:42.341683 22807 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:42.344339 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.345355 22833 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:42.233659 30227 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:42.347977 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104807us
I20260709 13:14:42.273923 30236 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:14:42.316138 30234 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:42.352345 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:42.353346 22839 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:42.319121 30163 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:42.357546 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 173893us
I20260709 13:14:42.357220 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.359690 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.362870 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:42.362927 22836 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:42.365818 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.368892 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:42.370422 30224 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:42.372138 30224 consensus_queue.cc:1059] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:42.373661 30210 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:42.373777 22837 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:42.374286 30210 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.374471 23466 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:42.374761 30210 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:42.373786 23431 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.162s (SENT)
I20260709 13:14:42.378158 23431 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:42.379841 30191 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:42.380123 22836 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:42.380401 30191 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:42.338827 30243 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.389848 30227 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:42.389780 30243 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.392342 30243 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.354224 30230 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:42.394223 30040 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.394510 30212 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:42.398367 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.338497 30162 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:42.404271 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:42.404531 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:42.404879 30162 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.407598 30162 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:42.387794 29823 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.388271 30241 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:14:42.408432 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:42.418305 30004 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:42.419597 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.420351 22832 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
W20260709 13:14:42.424633 30231 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.315s	user 0.004s	sys 0.006s
I20260709 13:14:42.386106 22834 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:42.381217 30191 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:42.432528 29943 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.435439 30233 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:42.436031 30233 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:42.440146 23430 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:42.441625 30205 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:42.442407 30205 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:42.442898 22832 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:14:42.443213 29874 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.443753 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.444314 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.444406 22838 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:42.445868 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:42.446230 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.446776 30204 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:42.447224 30204 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:42.447602 30204 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:42.448213 22837 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:42.448371 30219 raft_consensus.cc:3055] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:42.448752 30219 raft_consensus.cc:740] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:42.450189 22837 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:42.450443 29846 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:42.449721 30219 consensus_queue.cc:260] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.450909 29846 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:42.451779 30219 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:42.451972 29846 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.453392 29846 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:42.453914 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.443198 30205 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.454542 30205 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:42.455322 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.455987 22837 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
W20260709 13:14:42.457008 30228 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
I20260709 13:14:42.447642 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:42.459137 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.459992 22839 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:42.462375 30224 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:42.462993 30224 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:42.463950 30224 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.486341 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:42.487233 22761 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
W20260709 13:14:42.488444 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.486835 22841 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:42.400879 30175 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:42.494468 22832 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:42.493949 22838 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:42.496748 22765 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:42.496963 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:42.497241 22758 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:14:42.497789 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.498817 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:42.501612 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:42.502158 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 150102us
I20260709 13:14:42.502229 30011 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:42.495296 22732 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:42.501649 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47674us
I20260709 13:14:42.501766 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:42.508469 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:42.511443 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.511958 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.512358 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.512382 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:42.508118 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.513212 22761 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 15.
I20260709 13:14:42.522800 30234 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:42.525969 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60071.4us
I20260709 13:14:42.437117 30233 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.526839 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:42.527258 30233 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:42.531339 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 145984us
I20260709 13:14:42.539625 22765 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:42.544071 22835 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:42.553007 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:42.558403 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 189591us
I20260709 13:14:42.559834 30234 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:42.565874 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:42.566772 22763 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:42.567972 30204 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.572110 29933 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:42.572198 30234 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:42.573725 30246 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:42.575469 30247 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:42.576761 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 71753us
I20260709 13:14:42.577908 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:42.579403 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71390.6us
I20260709 13:14:42.580477 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.581614 29892 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:42.581974 30227 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:42.582597 30227 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:42.583292 30235 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:42.583853 29892 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.596853 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.540132 30241 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.540346 30236 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.542233 30206 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:14:42.616430 22805 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.4:37663 timed out after 1.636s (SENT) [suppressed 37 similar messages]
I20260709 13:14:42.609791 29846 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:42.622722 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:42.626986 22835 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:42.630555 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:42.626361 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.638967 30204 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:42.611079 30247 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:14:42.640285 30247 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:42.640712 30247 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:42.603749 30235 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:42.641844 30204 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:42.642227 30235 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:42.642510 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:42.643420 30204 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:14:42.650312 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76901us
I20260709 13:14:42.652261 22732 leader_election.cc:400] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
W20260709 13:14:42.626271 30231 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:14:42.659230 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:42.656111 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:42.661190 23458 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:42.663831 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45641.9us
I20260709 13:14:42.673576 29989 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:42.674280 29989 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:42.674679 29989 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:42.617117 30011 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:42.623405 30221 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:42.683421 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.626170 30068 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:42.690603 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:42.617306 29653 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:14:42.619056 30243 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:14:42.618556 29892 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.679244 30013 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:42.622951 30241 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.739778 22835 raft_consensus.cc:2468] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:42.728286 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.728668 22758 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:42.742767 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:42.744099 22915 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:42.750216 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.751269 22909 raft_consensus.cc:2410] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:14:42.751842 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.752609 22906 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:42.655104 29846 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:42.760741 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79140.1us
I20260709 13:14:42.627322 30236 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.762290 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:42.762530 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:42.763392 22911 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 26.
I20260709 13:14:42.713063 29992 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:42.767410 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 86143.6us
I20260709 13:14:42.768283 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:42.739683 30013 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:42.686060 30212 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.068s
W20260709 13:14:42.774735 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.774602 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:42.737963 30068 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:42.791703 22909 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:14:42.774456 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119380us
I20260709 13:14:42.780323 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.778148 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.758585 30243 consensus_queue.cc:1243] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:42.711809 30247 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.778863 30244 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:42.802455 22915 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 24. Ops: []
I20260709 13:14:42.815006 23458 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:42.777802 30005 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:42.772230 30204 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:42.839579 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:42.822971 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:42.823345 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.841630 22915 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 24: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:42.824620 22906 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:42.826442 30156 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:42.830740 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:42.845705 22760 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:42.836459 29653 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:42.839509 30005 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:42.841522 22911 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:42.785885 30013 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:42.772341 29992 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.850239 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:42.851920 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89731.5us
I20260709 13:14:42.885751 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:42.792465 30246 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:42.852293 22913 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:42.853065 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:42.855584 30068 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.859093 29846 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:42.930585 22914 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:42.771888 30227 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.788151 29933 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:14:42.860037 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:42.861909 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.862509 23457 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: [13.11-16.13]
I20260709 13:14:42.862131 30191 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.863171 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:42.801452 30240 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:42.796628 30236 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:42.852367 30239 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.236s
I20260709 13:14:42.868382 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:42.869589 30224 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:42.869230 30204 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.871224 30234 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:42.814026 30247 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:42.872515 30241 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:43.004721 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 144743us
W20260709 13:14:42.880708 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 264328us
I20260709 13:14:42.809715 30244 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:42.887796 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.916863 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.885053 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:42.928431 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:42.929073 22910 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:42.938680 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
W20260709 13:14:42.849241 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.056960 23430 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:42.951334 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:42.881098 30206 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:42.983645 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.028263 30244 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:42.991312 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:42.956442 30210 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
W20260709 13:14:43.017092 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 353602us
I20260709 13:14:43.021811 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:42.865393 30235 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:42.948203 30156 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:43.083786 30156 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.086308 30156 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:43.053642 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.049302 22906 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:43.087102 22910 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:43.088392 22906 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:43.091817 22906 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:14:42.992725 29846 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.093035 29846 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:42.982263 30224 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:42.996502 30234 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:43.068213 30148 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:42.927843 29653 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.098080 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:43.101723 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:43.063983 30011 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.053750 22909 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:43.022769 30236 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:43.062860 22913 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
W20260709 13:14:43.063494 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.064685 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.111457 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.066069 30226 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:14:43.061849 30219 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.008s
I20260709 13:14:43.070483 22914 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:43.073647 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.122273 22912 raft_consensus.cc:2410] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 13:14:43.077477 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.080646 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 194894us
I20260709 13:14:43.123338 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:43.081391 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74749.4us
I20260709 13:14:43.050354 22915 raft_consensus.cc:2368] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:42.978585 29943 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.052822 30257 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:43.101063 22837 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:43.113816 22834 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:14:43.120213 22908 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:43.077700 30212 raft_consensus.cc:2804] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:43.169696 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:43.171257 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:43.172174 23457 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:43.146914 29933 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:43.150827 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:43.099359 30004 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:43.177625 30004 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.180536 30004 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:43.101176 30213 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:43.101666 29846 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.158133 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.186847 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.110601 29717 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.097929 30148 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:43.112046 30243 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.198525 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:43.184201 30205 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.000s
I20260709 13:14:43.113008 30040 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:43.193339 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70133.5us
I20260709 13:14:43.191083 22908 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:43.212522 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:43.151449 30236 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:43.198508 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.190375 22909 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:43.203186 30246 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:43.204663 22762 raft_consensus.cc:1240] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:43.072113 30005 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.125974 30011 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.207986 30224 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.208642 30235 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:43.081815 30210 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.161266 29892 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:43.195683 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.190346 30213 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:43.191058 29943 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.355083 29653 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:43.322672 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.357366 22760 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:43.355875 30254 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:14:43.358948 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.359733 30254 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:43.360146 30254 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:43.360280 30267 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 13:14:43.360548 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:43.363262 30246 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.225392 30206 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.365847 30148 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.366920 30148 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:43.323506 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.369442 22759 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:43.369834 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.370081 30236 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.370680 23460 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:43.323185 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:43.221273 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 123151us
I20260709 13:14:43.247087 22834 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:43.376245 22766 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:14:43.376416 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 162539us
I20260709 13:14:43.376559 22730 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:43.337628 22914 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:14:43.282502 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:43.383347 22911 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.004s
I20260709 13:14:43.383898 22911 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:43.351632 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:43.235460 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.195420 30212 raft_consensus.cc:697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:43.352828 30260 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:14:43.401947 22757 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
W20260709 13:14:43.323236 30271 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:14:43.211014 29717 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:43.330927 30255 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:14:43.429700 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.412740 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.403259 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:43.412580 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:43.323511 30268 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:43.374284 29846 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.277470 30175 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.225415 29992 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.387907 30213 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:43.382603 30240 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:43.456897 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:43.436167 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:43.463027 22757 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:43.233242 30272 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.215147 30205 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.470300 30276 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:14:43.470813 30264 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:14:43.470948 29717 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.386770 30221 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
W20260709 13:14:43.471395 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79975.3us
I20260709 13:14:43.471446 30261 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.122s
I20260709 13:14:43.472023 30245 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:43.455785 30235 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:43.218297 30040 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.216598 30243 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.471203 30273 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.455408 30204 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.488874 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.489715 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:43.474357 30156 raft_consensus.cc:2764] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:14:43.442274 29653 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.212787 30234 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:43.522356 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 145923us
I20260709 13:14:43.495427 22914 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:43.524214 22914 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:43.465802 29846 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.497853 30212 consensus_queue.cc:237] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.500409 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:43.501420 22911 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:43.503829 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:43.509083 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79364.5us
I20260709 13:14:43.490037 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:43.514550 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.486657 30276 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:43.493706 29992 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:43.528204 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:43.503433 30235 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.501257 30245 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.530440 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:43.531191 23463 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:43.474725 30175 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.539095 22807 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:43.526990 30206 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.528484 30239 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:43.616184 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.533387 30004 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:43.638504 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 110388us
I20260709 13:14:43.530283 30276 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:43.537081 30235 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:43.532972 30191 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:43.640401 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104635us
I20260709 13:14:43.613047 22914 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.684572 22914 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:43.680212 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.538450 30245 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:43.527985 29653 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.541184 30156 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:43.536458 29823 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:14:43.579613 29874 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:43.522220 30204 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.714839 22879 leader_election.cc:400] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:14:43.719545 22879 leader_election.cc:403] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:43.721652 30275 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:43.722565 30278 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:14:43.727437 29989 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.052s	user 0.014s	sys 0.000s
I20260709 13:14:43.728005 29989 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.729262 22906 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:43.705376 30239 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.637871 30234 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:43.506413 30272 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.746673 30272 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.748920 30175 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.749748 30175 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:43.770190 30210 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:43.771106 30247 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.767s	user 0.007s	sys 0.005s
W20260709 13:14:43.721845 30274 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:14:43.737713 22914 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:14:43.739635 30265 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 13:14:43.772258 22760 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:43.773332 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:43.774039 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 234996us
I20260709 13:14:43.523643 30273 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.710781 30227 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.784268 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:43.582489 30244 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.714143 30156 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:43.760077 29943 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.719004 29874 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:43.788715 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:43.795854 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.798110 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 83433.8us
I20260709 13:14:43.799165 30240 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:14:43.801710 30289 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:14:43.788921 30247 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:14:43.803000 30205 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:43.791538 30227 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.803735 30011 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:43.805290 30224 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:43.805464 30273 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:43.806109 30275 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:43.807178 29653 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:43.816139 22840 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:43.707955 30206 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.821484 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:43.819591 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:43.815963 29874 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:43.837659 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.826782 30243 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:43.827690 30013 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.951s	user 0.011s	sys 0.005s
I20260709 13:14:43.791306 29933 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:14:43.796561 30156 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:43.828382 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55140.1us
I20260709 13:14:43.823211 29943 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:43.842767 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 302419us
I20260709 13:14:43.844764 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:43.873759 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 166636us
W20260709 13:14:43.880000 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65504.2us
I20260709 13:14:43.880918 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:43.881796 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.885757 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.886801 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:43.833170 29989 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:43.890905 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.890795 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.893059 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.893777 22908 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:43.894698 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.898536 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:43.899250 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.899555 22915 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19. Re-sending same reply.
I20260709 13:14:43.845638 30243 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:43.865770 30275 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:43.879259 30240 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:43.904421 22910 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:43.906464 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.911780 22912 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:14:43.913028 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:43.906551 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.902313 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:43.909327 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:43.910698 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:43.841058 30011 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:14:43.913682 22909 raft_consensus.cc:2376] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23. Re-sending same reply.
I20260709 13:14:43.914422 22913 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:43.845124 30013 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:43.901114 30148 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:43.930589 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.907507 30275 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.949671 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.904645 30243 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:43.928916 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:43.986377 22912 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:14:43.927443 30240 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:43.913626 30293 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:43.913298 29943 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:14:43.934592 30148 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:43.935978 30266 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.876s	user 0.001s	sys 0.003s
W20260709 13:14:43.936379 30282 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.204s	user 0.000s	sys 0.002s
W20260709 13:14:43.944024 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.927749 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 94424.1us
I20260709 13:14:43.923918 22762 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:43.886946 30213 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:43.922981 30040 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:43.925853 30191 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:43.912365 30204 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:43.996219 22764 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:44.001529 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.010603 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1090) took 1129 ms. Trace:
I20260709 13:14:44.015961 22906 rpcz_store.cc:276] 0709 13:14:42.880176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.105410 (+225234us) service_pool.cc:224] Handling call
0709 13:14:43.152201 (+ 46791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:43.729754 (+577553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.009081 (+279327us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54001e3cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:44.009379 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326634752}
I20260709 13:14:43.984606 30275 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:43.991988 30243 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:43.947911 30246 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:44.035239 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 190635us
W20260709 13:14:44.027303 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 81178.6us
I20260709 13:14:43.992887 30293 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.048355 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
W20260709 13:14:44.049331 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48071.6us
I20260709 13:14:44.029183 30191 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:44.055289 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.047415 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.054247 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.060563 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.076965 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:44.093995 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:44.098923 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.103480 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.104586 22840 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:14:44.072444 30292 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:14:44.119612 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.120476 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.120759 22837 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:44.124325 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.090025 30241 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:44.135352 30241 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.135823 30241 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:44.135993 23460 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:44.137368 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.139572 22838 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 22. Current term is 23. Ops: [20.16-22.17]
I20260709 13:14:44.140612 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.142613 30148 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:44.143576 30287 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.391s	user 0.000s	sys 0.001s
I20260709 13:14:44.093462 30240 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.150640 30212 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.151336 30212 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.141993 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.152386 22835 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:44.153725 30205 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:44.154284 30206 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.155293 30040 consensus_queue.cc:1059] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:44.090258 30278 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:44.101066 29823 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.050187 29933 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:14:44.163861 22805 leader_election.cc:400] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:44.164350 22805 leader_election.cc:403] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:14:44.134668 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.156085 30256 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:44.125492 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.044585 30293 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:44.156582 30278 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:44.165486 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.165985 22915 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:14:44.168685 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 194646us
W20260709 13:14:44.186607 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 109777us
I20260709 13:14:44.126924 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:44.184998 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.189332 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.156898 29823 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.190126 30277 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:14:44.190549 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.190937 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.106933 30295 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.196121 22840 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:14:44.156338 30040 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.199391 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.188753 23457 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:44.191506 23460 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:44.197073 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 142511us
I20260709 13:14:44.193161 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:44.201573 30206 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.154313 30213 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.205693 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.205827 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.206338 22762 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:44.203536 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:44.194934 22834 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:44.207490 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:44.207949 22915 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:44.057403 30233 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.137s
I20260709 13:14:44.205293 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.205029 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.174885 29933 consensus_queue.cc:1243] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:44.157768 30227 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:44.090831 30276 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.123447 30191 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.197345 29943 consensus_queue.cc:1243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:14:44.217366 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67950.8us
I20260709 13:14:44.217803 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:44.211584 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 164239us
I20260709 13:14:44.192319 30278 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.212611 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.212762 23466 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:14:44.212309 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8815691629999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:14:44.211146 22841 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:44.222146 23463 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:44.224294 22834 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:44.227049 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.223089 30277 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:44.228029 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.209978 30206 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.228153 30277 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.230593 30227 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.230681 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:44.230842 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:44.231365 30013 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.231572 29823 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:44.208794 30295 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.233418 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.234423 23457 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 26. Current term is 27. Ops: []
I20260709 13:14:44.236140 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.236508 30275 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.237320 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.225823 30234 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:44.237880 29943 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:44.238271 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.271612 22841 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:44.238382 30302 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:44.243589 22834 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:44.215463 30245 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.215873 30212 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.254537 30163 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.255900 22807 leader_election.cc:400] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:44.259611 22915 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:44.260267 22833 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:44.225982 30297 raft_consensus.cc:3055] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:44.258740 23459 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:44.259891 22836 raft_consensus.cc:2410] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:44.226424 30213 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.201977 30040 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.272202 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.256296 30277 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.283416 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.274974 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:44.274752 22834 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:44.291374 22834 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:44.291471 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.300222 30204 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:44.300796 30204 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:44.296033 22915 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:44.271867 30309 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:44.274881 30302 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:44.276525 30245 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:44.261574 30227 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.317991 23432 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:44.285859 30297 raft_consensus.cc:740] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:44.319901 22910 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:44.293334 30234 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:44.321053 30148 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:44.322276 22833 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:14:44.341028 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 66151.6us
I20260709 13:14:44.285351 30163 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.305864 30213 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.280097 30310 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:14:44.371220 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 100650us
W20260709 13:14:44.348618 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 117866us
I20260709 13:14:44.302135 29823 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:14:44.374333 29823 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:44.377432 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:44.372238 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:44.381712 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86298.9us
I20260709 13:14:44.335096 30291 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:44.355033 30311 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 13:14:44.355556 30243 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.342370 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:44.342392 30212 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:44.397192 30212 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.397758 30212 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:44.398108 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.399027 22915 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:44.400200 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.277760 30308 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.136s
I20260709 13:14:44.401851 30308 consensus_queue.cc:1243] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:44.403529 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:44.380822 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:44.352017 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 161523us
I20260709 13:14:44.405992 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:44.393520 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.395675 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:44.408557 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.410003 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:44.318881 30309 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.355599 30313 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:44.419130 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:44.427109 22758 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:44.420042 22766 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:44.382550 30291 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.395005 30302 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.422672 22760 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:14:44.432919 22760 pending_rounds.cc:85] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 12
I20260709 13:14:44.423372 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.340413 30204 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.389393 30148 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:44.405324 30206 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.426230 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:44.427146 22757 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:44.414069 30256 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:44.433239 22760 pending_rounds.cc:107] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:14:44.410562 29933 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.378183 29874 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:44.450171 30312 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:14:44.451169 30315 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:14:44.435392 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.451495 30243 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:44.445375 29933 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.454613 29933 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:44.455014 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.433346 30310 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:14:44.439987 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:44.440384 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 157077us
I20260709 13:14:44.433094 30191 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:44.464382 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.441869 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:44.465600 22762 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:44.465998 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59874.3us
I20260709 13:14:44.467021 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:44.467465 30309 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.468137 30309 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:44.463040 30310 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:44.443401 22760 raft_consensus.cc:2889] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.470105 30310 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.471563 30310 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:44.471753 30254 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.429677 30313 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.441051 30240 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:44.473783 30313 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.474161 30240 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.474639 30240 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:44.475781 30322 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:14:44.439654 30302 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.478113 22836 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:44.473239 30254 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:44.480055 30213 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.482225 30148 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.445948 30256 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.486042 30295 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.427553 30276 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:44.486730 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.448182 30204 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:44.448817 30327 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:44.487605 30327 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.488019 30327 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:44.491405 30321 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.492100 30321 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.494406 30239 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:44.495813 30294 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:44.496243 30244 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:44.496469 30294 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.496949 30294 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:44.497740 22762 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:44.500351 30278 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.464898 30040 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:44.502611 23458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1294) took 1644 ms (client timeout 1636 ms). Trace:
I20260709 13:14:44.503410 30278 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.448685 30304 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:14:44.420612 30245 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:44.446583 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.486178 30256 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.495911 30321 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:44.431193 30291 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:44.521106 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.497834 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:44.498940 29992 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.959s	user 0.014s	sys 0.001s
I20260709 13:14:44.499462 30191 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:44.448071 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:44.437940 30263 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.020s	sys 0.017s
I20260709 13:14:44.528389 29992 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:44.538167 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:44.538785 30227 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:44.530012 30321 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:44.511104 30239 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:44.539804 30321 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:44.425479 30297 consensus_queue.cc:260] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:44.506896 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67028.5us
I20260709 13:14:44.508942 22762 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:14:44.543694 22762 pending_rounds.cc:85] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 11
I20260709 13:14:44.511045 22915 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:14:44.543954 22762 pending_rounds.cc:107] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:14:44.544229 22762 raft_consensus.cc:2889] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.514050 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.505755 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:44.503269 23458 rpcz_store.cc:269] 0709 13:14:42.857777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.868083 (+ 10306us) service_pool.cc:224] Handling call
0709 13:14:44.502363 (+1634280us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54005136d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:44.502497 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245089664}
W20260709 13:14:44.514845 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.521369 30244 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:44.522692 22908 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:44.547210 30304 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:44.547740 30304 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.548302 30304 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:44.525840 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 56795.2us
I20260709 13:14:44.541002 30321 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.547109 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:44.480091 30302 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:44.518649 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 72146.8us
I20260709 13:14:44.518268 30245 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.539059 22761 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.520316 22758 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
W20260709 13:14:44.545243 30263 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.020s	sys 0.018s
I20260709 13:14:44.552389 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:44.429368 30234 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.527243 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:44.564654 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.565390 23458 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:44.501313 30241 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.554181 22807 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.556061 30219 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.562799 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.570814 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:44.570815 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:44.565907 22730 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.570s (SENT)
I20260709 13:14:44.576459 22807 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.584728 30325 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:44.585877 30163 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:44.586472 30295 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:44.586903 30295 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.587248 30295 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.566756 30244 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:44.591198 30244 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.528380 30191 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:44.594280 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45063.4us
I20260709 13:14:44.595484 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:44.596563 30013 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:44.599464 30224 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.770s	user 0.006s	sys 0.006s
W20260709 13:14:44.599948 30281 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.993s	user 0.004s	sys 0.000s
I20260709 13:14:44.600551 30224 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.601254 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1087) took 1846 ms. Trace:
I20260709 13:14:44.603255 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.606395 30311 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:44.571900 30239 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:44.607241 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2954) took 1369 ms. Trace:
I20260709 13:14:44.624167 22763 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:44.606415 22910 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:44.608518 30275 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.610692 30305 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:44.560200 30293 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.611984 30276 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:44.552763 30206 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.614007 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.615069 29653 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.642105 29653 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.616425 22835 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:44.616031 22907 rpcz_store.cc:276] 0709 13:14:42.754918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.755118 (+   200us) service_pool.cc:224] Handling call
0709 13:14:42.767346 (+ 12228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:42.786259 (+ 18913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:42.786309 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:42.795336 (+  9027us) log.cc:844] Serialized 26 byte log entry
0709 13:14:43.055244 (+259908us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b18003f6580. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:14:43.435100 (+379856us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:43.435161 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.435287 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:44.599232 (+1163945us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:44.600983 (+  1751us) raft_consensus.cc:1685] finished
0709 13:14:44.601013 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:44.601185 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9762,"raft.queue_time_us":11431,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7877,"spinlock_wait_cycles":168308864,"thread_start_us":379565,"threads_started":1,"wal-append.queue_time_us":716947}
I20260709 13:14:44.565465 30245 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.643296 22837 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:44.648204 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.648802 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.618899 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:44.619247 22761 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:44.616367 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:44.621440 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:44.651891 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:44.652547 22839 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.634570 23466 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:14:44.618445 30235 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.904s	user 0.014s	sys 0.000s
I20260709 13:14:44.639185 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.632251 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.640166 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.621274 30239 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:44.618247 29992 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:14:44.542374 30040 consensus_queue.cc:1175] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6545453e74344bb2957dd3729c5b3870" responder_term: 14 status { last_received { term: 14 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 14 index: 9 } }
I20260709 13:14:44.577891 30219 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.646688 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.650964 23460 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:44.655316 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.618813 30205 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:44.629853 30156 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.002s
W20260709 13:14:44.657648 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:44.672724 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.661691 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:44.673619 22835 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:44.664198 30326 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:44.622112 22765 rpcz_store.cc:276] 0709 13:14:43.237984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.238428 (+   444us) service_pool.cc:224] Handling call
0709 13:14:43.238854 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:43.533327 (+294473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:43.533363 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:43.579480 (+ 46117us) log.cc:844] Serialized 26 byte log entry
0709 13:14:43.824072 (+244592us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:43.824121 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.606744 (+782623us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540035f054. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:44.606798 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:44.606848 (+    50us) raft_consensus.cc:1685] finished
0709 13:14:44.606862 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:44.607177 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":999,"raft.queue_time_us":596,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4872,"spinlock_wait_cycles":676302592,"thread_start_us":117641,"threads_started":1,"wal-append.queue_time_us":292762}
I20260709 13:14:44.608528 30277 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.735286 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:44.709115 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.746233 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.684482 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.748258 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:44.749033 22758 raft_consensus.cc:2368] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25.
I20260709 13:14:44.656479 29943 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:44.743873 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.747005 22763 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:44.756953 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.759749 22839 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:44.761399 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.635604 30224 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.666265 30239 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:44.771522 22839 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:44.775362 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:44.660925 30013 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:44.781633 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:44.683202 30326 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.750196 30205 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:44.785669 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 136950us
I20260709 13:14:44.786635 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:44.787245 22835 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:44.829874 23431 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.650930 30206 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.835147 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:44.829874 22760 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:44.831528 22840 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:44.747152 30163 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.747005 29992 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:44.747305 30275 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:44.859735 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:44.666455 30235 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.683014 30204 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.147s
I20260709 13:14:44.860733 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:44.864092 23431 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:44.786216 30311 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.827926 30326 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.743204 30148 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:14:44.735967 30156 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.786024 30323 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:44.877718 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.883807 22907 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:44.834255 29943 raft_consensus.cc:2764] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:14:44.891212 22909 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:44.891695 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:44.892385 22910 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.902508 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:44.902984 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.904939 22911 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:44.869479 30337 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:44.910313 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.936270 22907 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:44.938910 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:44.867571 30275 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.863649 30163 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:44.911397 30332 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.390s	user 0.002s	sys 0.000s
I20260709 13:14:44.901649 30191 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:44.907374 29943 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.907814 30272 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.970703 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.986662 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:44.967338 30241 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.989734 30345 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:44.990729 30345 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:44.990887 30333 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:14:44.991286 30345 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:44.968571 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104569us
I20260709 13:14:44.891835 30311 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:44.955410 30293 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.958676 30334 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:45.028405 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89543.2us
W20260709 13:14:45.036046 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 419763us
I20260709 13:14:44.895460 30323 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:45.038781 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 295001us
I20260709 13:14:45.048246 30163 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.049628 30323 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:44.993129 30345 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:45.051030 30341 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:14:45.051368 22833 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 13:14:45.052836 30339 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:14:44.982677 30204 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:45.054309 22835 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:44.994908 30004 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.059832 22806 leader_election.cc:400] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25.
I20260709 13:14:45.060335 22806 leader_election.cc:403] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:14:45.040212 22841 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:45.040755 22907 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:14:44.938012 30240 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.079550 22841 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:45.084157 22833 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:14:45.089444 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:45.104817 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.103916 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.037835 30342 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:45.107698 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2414) took 1057 ms. Trace:
I20260709 13:14:44.985064 30191 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.108147 22908 rpcz_store.cc:276] 0709 13:14:44.049843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.546317 (+496474us) service_pool.cc:224] Handling call
0709 13:14:44.546739 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:44.548576 (+  1837us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:44.548758 (+   182us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:44.555066 (+  6308us) log.cc:844] Serialized 26 byte log entry
0709 13:14:44.684604 (+129538us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:44.684902 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.684985 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.107441 (+422456us) raft_consensus.cc:1685] finished
0709 13:14:45.107462 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.107620 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38299,"raft.queue_time_us":2840,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6354,"thread_start_us":78091,"threads_started":1,"wal-append.queue_time_us":259476}
I20260709 13:14:45.105949 30323 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:44.966276 29992 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:44.988085 29943 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:45.088074 30331 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:14:44.997414 30272 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.118224 22915 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:44.984114 30291 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.110111 30293 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:45.126480 30175 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:45.126713 30191 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.087594 30148 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.131346 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.102365 30277 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.086882 30219 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.112696 30342 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.123538 22908 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 22. Current term is 23. Ops: []
W20260709 13:14:45.139894 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 83618.3us
I20260709 13:14:45.043202 30334 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.069612 30204 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.119268 30323 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.077005 30004 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.171263 30357 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.171833 30357 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.172644 30004 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.167404 30254 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:14:45.173861 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.174089 30357 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:45.174558 22833 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:45.174787 30323 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:45.175691 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.176014 29943 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.177059 30219 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:45.178263 30204 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.120280 30224 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.141136 30175 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.148118 30356 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:45.149717 30354 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.197993 30354 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.150370 30358 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:45.151939 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2411) took 1152 ms. Trace:
I20260709 13:14:45.191805 30219 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:14:45.201910 22913 rpcz_store.cc:276] 0709 13:14:43.999563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.452907 (+453344us) service_pool.cc:224] Handling call
0709 13:14:44.453283 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:44.453710 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:44.453736 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:44.454251 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:14:44.514269 (+ 60018us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:44.514315 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.514391 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.151699 (+637308us) raft_consensus.cc:1685] finished
0709 13:14:45.151726 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.151872 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1175,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":59622,"threads_started":1,"wal-append.queue_time_us":184957}
I20260709 13:14:45.203997 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.204674 22913 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:45.152933 30349 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:14:45.157150 22807 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.167799 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1094) took 1018 ms. Trace:
I20260709 13:14:45.207037 22908 rpcz_store.cc:276] 0709 13:14:44.149021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.122796 (+973775us) service_pool.cc:224] Handling call
0709 13:14:45.123264 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.167503 (+ 44239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.167738 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:45.207754 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 50716.9us
I20260709 13:14:45.136508 30205 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.166082 30342 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:45.174646 30357 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:45.209553 30358 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:45.209466 30357 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:45.190775 30275 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.158773 30235 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:45.163445 30212 consensus_meta.cc:287] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.765s	user 0.015s	sys 0.000s
I20260709 13:14:45.210985 30357 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.211908 30212 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.212648 30357 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:45.192627 30337 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:45.217123 22910 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:45.194178 30240 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.206655 30175 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.208911 22807 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.208912 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:45.146976 30206 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.218302 22913 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:45.168354 30156 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.223317 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.191730 30356 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.226377 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.227674 22807 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:45.228001 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.162760 30148 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:45.224748 30294 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.001s
I20260709 13:14:45.231489 22910 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:45.238641 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1095) took 1064 ms. Trace:
I20260709 13:14:45.228694 30356 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:45.235822 22908 raft_consensus.cc:3055] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:45.240410 22908 raft_consensus.cc:740] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:45.239114 22910 rpcz_store.cc:276] 0709 13:14:44.174465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.221421 (+1046956us) service_pool.cc:224] Handling call
0709 13:14:45.231232 (+  9811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.238237 (+  7005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.238558 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:45.224460 30275 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:45.226346 30358 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:45.226584 30212 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.261381 22910 pending_rounds.cc:85] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 12
I20260709 13:14:45.261770 22910 pending_rounds.cc:107] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 13:14:45.262024 22910 raft_consensus.cc:2889] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:45.265283 30346 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:14:45.265869 22908 consensus_queue.cc:260] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.266914 22908 raft_consensus.cc:3060] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.271396 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:45.240401 30175 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:45.274904 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.275451 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.240692 30354 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.275820 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.230501 30359 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:45.277390 22762 raft_consensus.cc:2410] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:14:45.281424 22908 raft_consensus.cc:1275] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:14:45.260071 30313 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.786s	user 0.005s	sys 0.009s
I20260709 13:14:45.282249 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.283210 22908 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:45.256810 29874 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:45.285010 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57388.8us
W20260709 13:14:45.276473 30327 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.788s	user 0.015s	sys 0.000s
I20260709 13:14:45.289610 22807 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:45.247710 30294 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.276609 30246 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:45.282827 30275 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:45.283003 30358 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:45.290683 30337 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:45.296128 30295 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.707s	user 0.002s	sys 0.012s
W20260709 13:14:45.299194 30256 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.002s
I20260709 13:14:45.329813 30256 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.330511 30148 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.332350 30256 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.319875 30295 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.333973 29874 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:45.334923 29874 consensus_queue.cc:1243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:45.335448 30295 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.320827 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:45.322520 30272 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.337139 22908 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:45.300025 30246 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.339989 30246 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.340183 22908 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:45.314386 30275 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.327741 30342 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.334049 22761 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:45.342059 30310 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.870s	user 0.015s	sys 0.001s
I20260709 13:14:45.341656 30275 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.345855 22834 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.347141 23460 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:45.347144 30342 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.348757 30308 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:45.349241 30308 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:45.349601 30353 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:14:45.350173 30308 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.352324 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:45.355258 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.296443 30359 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:45.293061 30311 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.367226 22837 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 27.
I20260709 13:14:45.359349 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:45.314913 30304 consensus_meta.cc:287] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.001s
I20260709 13:14:45.371446 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.299904 30358 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:45.288066 30313 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:45.375558 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104252us
W20260709 13:14:45.377776 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.379724 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.294533 30327 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.374176 29823 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:45.405112 30334 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.407922 30313 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.409570 30294 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.374969 30358 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:45.374063 30356 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.380106 30359 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.412325 29823 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.423964 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.435734 22761 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:45.415503 30239 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.001s
I20260709 13:14:45.444834 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.445785 30356 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.448170 30371 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:45.450271 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.451200 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.452378 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.453177 22760 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:45.375375 30304 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.454705 30246 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:45.457886 30246 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.438959 29823 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:45.464066 30245 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.791s	user 0.011s	sys 0.004s
I20260709 13:14:45.443660 30239 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.474327 22761 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:45.477366 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:45.478638 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:45.479347 22914 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:45.480439 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.481725 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.483868 29653 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:45.497789 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.486171 30371 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:45.501469 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.502987 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.503403 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2403) took 1608 ms. Trace:
I20260709 13:14:45.503821 22909 rpcz_store.cc:276] 0709 13:14:43.894795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.924903 (+ 30108us) service_pool.cc:224] Handling call
0709 13:14:45.488044 (+1563141us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b5400041354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:45.503318 (+ 15274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155136768}
I20260709 13:14:45.474759 30245 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.507107 22912 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:45.507386 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3473) took 2357 ms. Trace:
I20260709 13:14:45.507818 22764 rpcz_store.cc:276] 0709 13:14:43.156222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.190113 (+ 33891us) service_pool.cc:224] Handling call
0709 13:14:43.213306 (+ 23193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:44.015536 (+802230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.507068 (+1491532us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54002210d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:45.507295 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800073600}
I20260709 13:14:45.508895 30275 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:45.509378 30275 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.509810 30275 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:45.510100 30246 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.514505 30293 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25.
I20260709 13:14:45.516829 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2409) took 1590 ms. Trace:
I20260709 13:14:45.508944 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2412) took 1484 ms. Trace:
I20260709 13:14:45.518113 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.518812 22832 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:45.519942 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.521153 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.523062 30245 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.523418 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:45.524307 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:45.524447 30356 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:45.542311 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.542827 23459 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:14:45.525658 22833 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:14:45.525734 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:45.527084 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:45.528592 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2415) took 1469 ms. Trace:
I20260709 13:14:45.545722 23462 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:45.528719 30327 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.530318 30323 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.532275 30239 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:45.559149 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.479139 30359 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:45.536695 30348 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.487s	user 0.000s	sys 0.001s
I20260709 13:14:45.517808 22912 rpcz_store.cc:276] 0709 13:14:44.024584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.455974 (+431390us) service_pool.cc:224] Handling call
0709 13:14:44.489967 (+ 33993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.507553 (+1017586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.508463 (+   910us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002116d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:45.508838 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338733312}
I20260709 13:14:45.517252 22911 rpcz_store.cc:276] 0709 13:14:43.926310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.017041 (+ 90731us) service_pool.cc:224] Handling call
0709 13:14:44.018767 (+  1726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:44.905351 (+886584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.516509 (+611158us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54000fb254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:45.516761 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037310336}
I20260709 13:14:45.544732 23466 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
W20260709 13:14:45.540119 30338 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.498s	user 0.005s	sys 0.000s
I20260709 13:14:45.540887 30304 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:45.543540 30371 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.543977 30334 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:45.560487 30313 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.566541 22838 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:45.568053 30311 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:14:45.572356 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.572845 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.545627 22915 rpcz_store.cc:276] 0709 13:14:44.058853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.567109 (+508256us) service_pool.cc:224] Handling call
0709 13:14:44.604718 (+ 37609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.136853 (+532135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.528286 (+391433us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54000fad54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:45.528523 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180402560}
I20260709 13:14:45.573419 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.575636 30371 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.569322 30313 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.591919 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.593025 22764 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:14:45.593266 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:45.591560 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:45.595630 30368 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.079s	user 0.002s	sys 0.000s
I20260709 13:14:45.596221 30334 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:45.598212 30323 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.598901 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.599745 22838 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:45.600652 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.601220 22838 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:45.603075 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2817) took 1058 ms. Trace:
W20260709 13:14:45.610663 30326 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.708s	user 0.014s	sys 0.000s
I20260709 13:14:45.610924 22835 rpcz_store.cc:276] 0709 13:14:44.544385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.830484 (+286099us) service_pool.cc:224] Handling call
0709 13:14:44.830851 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.054816 (+223965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.602811 (+547995us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001bbf54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:45.603006 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513138048}
I20260709 13:14:45.611148 30326 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.612783 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.614048 22835 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:14:45.614614 30291 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.574826 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:45.618021 23466 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:45.617615 30326 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.622339 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.606608 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1293) took 1097 ms. Trace:
I20260709 13:14:45.609191 23458 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:45.622709 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.625281 22764 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:45.624469 22836 rpcz_store.cc:276] 0709 13:14:44.509158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.509336 (+   178us) service_pool.cc:224] Handling call
0709 13:14:44.525869 (+ 16533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:44.552931 (+ 27062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:44.552966 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:44.572327 (+ 19361us) log.cc:844] Serialized 26 byte log entry
0709 13:14:44.732977 (+160650us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:44.733023 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.733108 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.327683 (+594575us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b4800130158. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:14:45.532026 (+204343us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:45.606354 (+ 74328us) raft_consensus.cc:1685] finished
0709 13:14:45.606379 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.606512 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24781,"raft.queue_time_us":17046,"raft.run_cpu_time_us":3448,"raft.run_wall_time_us":778488,"spinlock_wait_cycles":12534400,"thread_start_us":143015,"threads_started":1,"wal-append.queue_time_us":377592}
I20260709 13:14:45.605973 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:45.627672 22805 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.629154 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1100) took 1150 ms. Trace:
I20260709 13:14:45.629577 22913 rpcz_store.cc:276] 0709 13:14:44.478605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.226725 (+748120us) service_pool.cc:224] Handling call
0709 13:14:45.235635 (+  8910us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:45.236041 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.236073 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:45.236637 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:14:45.622365 (+385728us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b18004bece0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1
0709 13:14:45.622613 (+   248us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:45.622667 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.622782 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.622851 (+    69us) raft_consensus.cc:1685] finished
0709 13:14:45.622868 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.629084 (+  6216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9203,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":1952,"spinlock_wait_cycles":9612544,"thread_start_us":21091,"threads_started":1,"wal-append.queue_time_us":84942}
I20260709 13:14:45.627930 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:45.635201 23458 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:45.636842 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.637744 22766 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:45.581319 30313 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.621755 30275 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.646670 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.650473 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.647904 22807 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.651397 22836 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:14:45.652571 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:45.632684 30233 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:45.655117 30233 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:45.653873 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:45.655604 30233 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.656317 30356 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:45.656797 30356 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.657213 30356 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:45.658440 30275 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.626045 30357 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.659466 30357 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.632849 30323 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:45.662575 30340 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.663s	user 0.002s	sys 0.000s
I20260709 13:14:45.662763 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.663473 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.650223 30334 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:45.664286 30334 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:45.664728 30334 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.645781 30326 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:45.652807 30358 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.664397 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.664736 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.649168 22764 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:45.677127 22835 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:45.679298 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:45.666653 30326 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.684850 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.686190 23430 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:45.685071 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.688696 22835 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:45.690025 30236 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.690809 30236 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.692078 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.692919 22835 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:45.696099 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.698705 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.694105 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:45.703440 22911 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:45.693427 30236 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.718047 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.727418 30316 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:45.743688 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57593.4us
I20260709 13:14:45.744716 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:45.747402 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.701723 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.749186 22764 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:45.749899 22835 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:45.752257 23466 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:45.753074 22911 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:45.754416 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:45.754784 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.677191 22838 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:45.754807 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:45.761583 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.762471 22836 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.765219 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.680369 30323 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:45.766218 22758 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:45.743386 30326 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:45.775202 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:45.776329 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.776935 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.777593 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.769213 29846 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:45.780983 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:45.785655 30357 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:45.785907 30305 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:45.786547 30305 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:45.787084 30305 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.755971 30342 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:45.789863 30236 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:45.756690 30204 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:45.789921 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.792703 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:45.755379 30351 raft_consensus.cc:2804] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:45.728206 30316 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.797595 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.763716 30040 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:45.799358 22837 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:45.800277 22758 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 15: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:45.764238 30313 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:45.794054 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:45.802796 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.795989 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:45.810036 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.804831 22835 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.794615 30204 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:45.817921 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.804827 30236 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:45.805493 30354 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:45.806545 30040 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.807022 30342 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:45.825284 22911 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:45.804574 30351 raft_consensus.cc:697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:45.835229 30351 consensus_queue.cc:237] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.829843 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.837441 22913 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:45.814254 30313 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.816388 30204 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:45.838017 22879 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:45.825143 30372 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.314s	user 0.002s	sys 0.004s
I20260709 13:14:45.823006 29892 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:45.820888 30304 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:14:45.842286 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 204536us
I20260709 13:14:45.844579 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.844679 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:45.834060 22835 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
W20260709 13:14:45.846803 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.849184 22835 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:14:45.850495 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.850821 22835 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:45.839505 29874 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:14:45.852931 29653 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:45.855198 29653 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:45.856379 22764 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:45.856720 22764 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:45.857616 22761 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:45.859529 30040 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.838584 30354 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:45.862222 30354 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.853922 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.865612 30148 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:45.867846 30311 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:14:45.870005 30379 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:14:45.869572 22764 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.870704 22764 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:45.880229 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.843345 30275 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:14:45.845629 30236 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:45.843773 29892 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:45.898008 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:45.849308 30323 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:45.903121 22836 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:45.841717 30239 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:14:45.910502 22764 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:14:45.911017 22764 pending_rounds.cc:85] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:45.911271 22764 pending_rounds.cc:107] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:14:45.911516 22764 raft_consensus.cc:2889] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:45.919845 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.920710 22760 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:45.852299 30245 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:14:45.928722 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:45.900789 29892 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:45.887192 30204 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.933167 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:45.934674 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.941324 22764 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:45.934069 29933 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:14:45.926658 29992 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.923393 30323 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:45.898919 29823 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.926280 30278 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:45.932011 30358 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:45.968578 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.948798 30371 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:45.934402 29874 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
W20260709 13:14:45.981475 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.934538 30246 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:14:45.960335 30309 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:45.987872 22838 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 24: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:45.924924 30313 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:45.992409 30374 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:14:45.994232 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1294) took 1484 ms. Trace:
I20260709 13:14:45.994607 22839 rpcz_store.cc:276] 0709 13:14:44.509543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.773602 (+264059us) service_pool.cc:224] Handling call
0709 13:14:44.774661 (+  1059us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:44.783371 (+  8710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:44.783493 (+   122us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:44.839340 (+ 55847us) log.cc:844] Serialized 78 byte log entry
0709 13:14:45.515768 (+676428us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b180042e5c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:14:45.693089 (+177321us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:45.840232 (+147143us) log.cc:844] Serialized 14 byte log entry
0709 13:14:45.842049 (+  1817us) log.cc:844] Serialized 14 byte log entry
0709 13:14:45.847603 (+  5554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.847696 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.993942 (+146246us) raft_consensus.cc:1685] finished
0709 13:14:45.993967 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.994164 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9371,"prepare.queue_time_us":336447,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":115292,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":19370,"spinlock_wait_cycles":14598528,"thread_start_us":323684,"threads_started":2,"wal-append.queue_time_us":950648}
W20260709 13:14:46.013232 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:46.016331 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 85957us
W20260709 13:14:46.028703 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.024060 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.033042 23431 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:45.964521 29992 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:45.991065 30309 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.045630 30204 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:46.042853 30375 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:14:46.070427 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:46.071226 23458 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:46.035975 30213 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:14:46.046320 30323 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.048033 30309 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.010419 30356 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.014524 30175 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.038178 30354 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:46.054400 30305 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:45.970476 30278 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.037861 30337 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:46.057587 29992 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.036543 30247 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:46.059360 29823 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.013588 30245 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:46.061749 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.063853 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:46.074877 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.079550 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:45.933203 30342 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.072243 30040 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:46.092265 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.094744 30354 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.103952 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:46.114842 22760 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:46.115365 23458 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
W20260709 13:14:46.100885 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 245945us
I20260709 13:14:46.119408 29892 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.116617 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:46.097057 30309 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:46.116605 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:46.131274 22760 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:46.131841 22760 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.134092 30385 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.134779 30385 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.135215 30278 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.104887 30246 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:46.094967 30175 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.123273 30354 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:46.143986 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:46.139787 22879 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:46.102483 30247 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.129575 30275 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:46.141206 22760 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.132891 30239 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:46.146790 22879 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:46.150630 22760 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:46.105226 30245 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.138669 30351 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:46.125245 30334 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.133005 29823 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:14:46.197013 30247 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.193729 29892 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:46.149955 30148 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:46.198818 30148 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:46.199838 30148 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.122188 30040 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.203431 30040 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.196046 30245 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.197794 30304 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:46.209448 22760 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:14:46.209420 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:46.211542 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67721.2us
I20260709 13:14:46.213203 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.210448 22760 pending_rounds.cc:85] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 12
I20260709 13:14:46.214869 22760 pending_rounds.cc:107] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:14:46.215276 22760 raft_consensus.cc:2889] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:46.218456 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.219225 22913 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:46.181967 30239 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.147208 30385 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.225353 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.229039 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.228917 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.229945 22907 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 15: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:46.233011 22910 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:46.222347 30239 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:46.234936 30247 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:14:46.237460 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.237756 30247 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.238910 30247 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.226460 29892 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:14:46.242328 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.244486 30175 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.231163 30246 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:46.245457 30246 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:46.246122 30246 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:46.188547 30278 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:46.183350 30275 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:46.250609 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2353) took 1135 ms. Trace:
I20260709 13:14:46.251168 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.255998 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.255264 22835 rpcz_store.cc:276] 0709 13:14:45.114948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.058392 (+943444us) service_pool.cc:224] Handling call
0709 13:14:46.058865 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:46.071142 (+ 12277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.071176 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:46.071825 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 13:14:46.137598 (+ 65773us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:46.137665 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.137770 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:46.250360 (+112590us) raft_consensus.cc:1685] finished
0709 13:14:46.250381 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.250535 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11797,"raft.queue_time_us":13326,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":12026,"thread_start_us":65242,"threads_started":1,"wal-append.queue_time_us":171649}
I20260709 13:14:46.256786 22910 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:46.251739 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:46.257025 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.257333 22760 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:46.257907 22835 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:46.252393 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.253078 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.259321 22907 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:46.260218 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.262775 22912 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:46.188957 30351 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:46.253929 22732 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.583s (SENT)
I20260709 13:14:46.190227 30321 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:46.222390 29943 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:14:46.263749 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.264808 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.264887 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.270030 22914 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:46.266767 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.265281 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:46.265141 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.274015 22913 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:46.271175 22915 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:46.273928 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.278664 22835 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 25.
I20260709 13:14:46.279280 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.280051 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:46.264405 22730 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.266539 30247 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:14:46.282213 30305 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:46.282675 30387 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:14:46.286154 22915 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:46.287256 30324 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:46.287756 30324 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.288115 30324 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:46.310429 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.311769 22910 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:46.313959 22835 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:46.236138 30243 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:14:46.268313 30351 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:46.276674 30233 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.001s
I20260709 13:14:46.326054 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.327217 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2354) took 1204 ms. Trace:
I20260709 13:14:46.327617 22835 rpcz_store.cc:276] 0709 13:14:45.122491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.312873 (+1190382us) service_pool.cc:224] Handling call
0709 13:14:46.313403 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.314785 (+  1382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.327149 (+ 12364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.329994 22915 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:14:46.338667 22914 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:14:46.280260 30278 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:46.323282 30388 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
W20260709 13:14:46.346632 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.347947 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.348619 30358 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.352807 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.364578 22912 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:46.354265 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.355322 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.318208 30213 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:46.304566 30311 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:46.359412 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.359818 22906 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
W20260709 13:14:46.363040 30390 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:14:46.366161 22915 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:14:46.367973 22910 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:46.369676 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.358309 22835 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:14:46.371163 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.399515 22761 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:46.401626 22912 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:46.396751 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2355) took 1256 ms. Trace:
I20260709 13:14:46.387032 22766 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:46.402884 22835 rpcz_store.cc:276] 0709 13:14:45.140135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.357309 (+1217174us) service_pool.cc:224] Handling call
0709 13:14:46.358030 (+   721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.396325 (+ 38295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.396668 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.385161 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.404357 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.404606 22906 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:46.404953 22835 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:46.406898 22913 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
W20260709 13:14:46.385450 22732 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.707s (SENT)
I20260709 13:14:46.407711 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:46.325947 30342 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
W20260709 13:14:46.408984 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 144665us
I20260709 13:14:46.388118 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.412144 22909 raft_consensus.cc:3060] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:46.412544 22766 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:46.413609 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.415251 22766 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:46.397159 22757 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:14:46.409832 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:46.416594 22761 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:46.416987 22912 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:46.345165 30310 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:46.421420 22909 raft_consensus.cc:2468] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:46.372818 30356 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:46.342870 30233 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.366727 30321 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:46.429239 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.431787 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.393162 30385 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:46.407783 30296 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:46.433656 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.451311 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2350) took 1371 ms. Trace:
I20260709 13:14:46.442368 30385 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.451788 22836 rpcz_store.cc:276] 0709 13:14:45.079418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.910307 (+830889us) service_pool.cc:224] Handling call
0709 13:14:45.910692 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:45.924286 (+ 13594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.924321 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:45.939219 (+ 14898us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.184221 (+245002us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:46.184272 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.184354 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:46.451075 (+266721us) raft_consensus.cc:1685] finished
0709 13:14:46.451111 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.451248 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13159,"raft.queue_time_us":13713,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8328,"thread_start_us":208365,"threads_started":1,"wal-append.queue_time_us":222897}
I20260709 13:14:46.437554 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.433284 22906 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:46.399610 29933 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:46.454048 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:46.483178 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.462523 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.485704 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:46.377947 30358 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.396552 30311 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.491628 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:46.380127 30397 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:14:46.453275 30234 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:14:46.496241 22910 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
W20260709 13:14:46.540541 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.441375 30321 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.357654 30275 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.500121 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:46.545068 30337 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:46.453890 30369 consensus_queue.cc:1048] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:14:46.492286 30213 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:46.545377 30247 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:46.548239 30213 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.485954 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.552778 22909 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:46.553074 22835 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:46.434701 30310 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.555706 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.558519 22835 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:46.559741 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.560680 22835 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:46.563712 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.562328 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:46.567004 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.495596 29933 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.570960 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 1059) took 1710 ms. Trace:
W20260709 13:14:46.560784 30326 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.765s	user 0.014s	sys 0.000s
I20260709 13:14:46.572832 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.489711 30013 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 13:14:46.575076 22836 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:14:46.572192 23460 rpcz_store.cc:276] 0709 13:14:44.856856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.857183 (+   327us) service_pool.cc:224] Handling call
0709 13:14:44.857600 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.347594 (+489994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.567095 (+1219501us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b540068ef54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:46.567358 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125340928}
I20260709 13:14:46.579711 22835 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:46.587513 22839 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:14:46.588589 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2351) took 1508 ms. Trace:
I20260709 13:14:46.589008 22839 rpcz_store.cc:276] 0709 13:14:45.079705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.996697 (+916992us) service_pool.cc:224] Handling call
0709 13:14:45.997087 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.587151 (+590064us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b54000d50a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af31c0 00007f68a1af4231 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c
0709 13:14:46.588157 (+  1006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.588505 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5627776}
I20260709 13:14:46.593679 30233 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.596839 29933 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.598572 30321 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:46.601351 30401 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:14:46.584601 30326 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.602905 30310 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.574795 30234 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:46.603650 30234 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.603928 30326 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.575160 30342 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:46.606144 22759 raft_consensus.cc:2393] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:46.598997 29933 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:46.611357 30354 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.613665 30385 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.614511 22834 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:46.620553 30398 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:14:46.620821 30342 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:46.622010 30342 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.622807 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1307) took 1062 ms. Trace:
I20260709 13:14:46.623399 22838 rpcz_store.cc:276] 0709 13:14:45.560245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.795711 (+235466us) service_pool.cc:224] Handling call
0709 13:14:46.622571 (+826860us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54003219d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:46.622740 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409279744}
I20260709 13:14:46.624886 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.625885 30325 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.626516 30325 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.627015 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.628186 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:46.628574 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2808) took 2393 ms (client timeout 1881 ms). Trace:
W20260709 13:14:46.628991 22840 rpcz_store.cc:269] 0709 13:14:44.235404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.255805 (+ 20401us) service_pool.cc:224] Handling call
0709 13:14:46.628400 (+2372595us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54000c71d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:46.628503 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155705600}
I20260709 13:14:46.604878 30234 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.629485 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.630115 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.632092 22836 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:46.632352 22840 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:46.633005 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.633183 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.633810 22840 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:46.634701 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:46.635340 22840 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:46.636569 22840 raft_consensus.cc:1217] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Deduplicated request from leader. Original: 19.10->[21.11-24.13]   Dedup: 21.11->[22.12-24.13]
I20260709 13:14:46.637014 22840 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [21.11-24.13]
W20260709 13:14:46.604269 30394 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:14:46.628186 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.637939 30148 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:46.638219 22766 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:46.638916 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:46.639283 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.639734 30148 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:46.640017 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.640416 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.641068 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:46.634753 30325 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.641278 30311 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.566110 30246 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.655020 22766 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:46.643987 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:46.633826 22836 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:46.646032 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2356) took 1499 ms. Trace:
I20260709 13:14:46.674575 22840 rpcz_store.cc:276] 0709 13:14:45.146261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.635993 (+1489732us) service_pool.cc:224] Handling call
0709 13:14:46.636358 (+   365us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:46.637321 (+   963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.645971 (+  8650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.682688 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.684022 22839 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:46.686457 22836 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:46.687536 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2368) took 1063 ms. Trace:
I20260709 13:14:46.687958 22836 rpcz_store.cc:276] 0709 13:14:45.624149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.685777 (+1061628us) service_pool.cc:224] Handling call
0709 13:14:46.686224 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:46.687101 (+   877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.687442 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.690315 22836 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:46.692010 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48096.3us
W20260709 13:14:46.695781 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 388340us
I20260709 13:14:46.651481 22834 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:46.703905 22840 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:46.704770 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2369) took 1065 ms. Trace:
I20260709 13:14:46.706032 22836 rpcz_store.cc:276] 0709 13:14:45.626771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.688983 (+1062212us) service_pool.cc:224] Handling call
0709 13:14:46.689416 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:46.691472 (+  2056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.691769 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.604095 30334 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.710490 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2365) took 1114 ms. Trace:
I20260709 13:14:46.720522 22840 rpcz_store.cc:276] 0709 13:14:45.623028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.675651 (+1052623us) service_pool.cc:224] Handling call
0709 13:14:46.703582 (+ 27931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:46.709712 (+  6130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.710042 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.724033 22758 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:14:46.711123 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2812) took 2458 ms. Trace:
I20260709 13:14:46.129096 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:46.729898 22834 rpcz_store.cc:276] 0709 13:14:44.243921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.274313 (+ 30392us) service_pool.cc:224] Handling call
0709 13:14:44.274617 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.345434 (+1070817us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b4c000c72c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689fdbda20 00007f689fdcea29 00007f689fd94644 00007f689fdbbf2d 00007f689fdbbb93 00007f68a1aec18b 00007f68a1aeccb5 00007f68a1ae7f30 00007f68a1ae97de 00007f68a1af3b49 00007f68a1af4416
0709 13:14:46.614051 (+1268617us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5800062cd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a4d0aa 00007f68a1acf447 00007f68a1ae7fe7 00007f68a1ae97de 00007f68a1af3b49 00007f68a1af4416 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4
0709 13:14:46.700370 (+ 86319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.701749 (+  1379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2913764480}
W20260709 13:14:46.734860 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.652722 30354 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:14:46.661428 30326 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:46.637727 30304 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:14:46.776569 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:46.759465 22907 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:46.718518 30306 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:14:46.641098 30148 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:14:46.784194 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1315) took 1105 ms. Trace:
I20260709 13:14:46.784860 22840 rpcz_store.cc:276] 0709 13:14:45.678793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.754465 (+1075672us) service_pool.cc:224] Handling call
0709 13:14:46.783592 (+ 29127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:46.783787 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.783812 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:46.783860 (+    48us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:46.783890 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.783959 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.784126 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:46.787642 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:46.790108 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 334868us
I20260709 13:14:46.702993 30305 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:46.796692 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:46.789108 30324 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.003s
I20260709 13:14:46.785923 30326 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:46.781723 30276 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.805366 22840 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:46.800621 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:46.775460 30384 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 13:14:46.812542 30309 consensus_meta.cc:287] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.001s
I20260709 13:14:46.845108 30415 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:46.808379 30305 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:46.816893 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:46.805047 30324 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:46.857167 30351 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.006s
I20260709 13:14:46.822666 30416 raft_consensus.cc:2764] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:14:46.810133 30276 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.857165 30275 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:46.877075 30239 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.002s
I20260709 13:14:46.861723 30358 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:46.882929 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:46.896905 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73091us
W20260709 13:14:46.923892 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 381348us
I20260709 13:14:46.922017 30356 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:14:46.927846 29933 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:14:46.920258 30412 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 13:14:46.932255 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 139969us
W20260709 13:14:46.936322 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 119551us
I20260709 13:14:46.958858 23460 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:46.948503 23462 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
W20260709 13:14:46.948657 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.865615 30326 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.962150 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:46.972915 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:46.973941 22766 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:46.948518 30416 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:46.978183 30416 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:46.963100 30428 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.983148 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1311) took 1417 ms. Trace:
I20260709 13:14:46.983577 30428 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:46.983722 22835 rpcz_store.cc:276] 0709 13:14:45.565372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.645336 (+1079964us) service_pool.cc:224] Handling call
0709 13:14:46.645672 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.653037 (+  7365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.653076 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:46.682086 (+ 29010us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.924491 (+242405us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:46.924561 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.924682 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:46.982926 (+ 58244us) raft_consensus.cc:1685] finished
0709 13:14:46.982948 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.983083 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6856,"raft.queue_time_us":15878,"spinlock_wait_cycles":21034368,"thread_start_us":148173,"threads_started":1,"wal-append.queue_time_us":217931}
I20260709 13:14:46.964291 30213 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:46.986116 30416 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:46.986451 30246 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:46.986245 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:46.987267 30326 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:46.987433 22835 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:46.989846 22915 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:46.997927 22911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3759) took 1108 ms. Trace:
I20260709 13:14:46.941730 30275 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:47.005563 30405 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.600s	user 0.004s	sys 0.000s
I20260709 13:14:47.006114 30432 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:47.007259 30243 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:47.008072 30243 consensus_queue.cc:1243] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:14:47.007223 30275 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.008509 30432 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:46.938948 30351 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.009158 30432 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:47.010664 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.011562 30351 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.015208 30204 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:47.002158 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:46.999454 30305 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.017470 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:46.949169 29933 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:47.002879 30431 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:46.998871 30233 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:47.020169 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.019901 29933 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.020277 30233 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.022703 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:47.023423 23460 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:47.023658 30246 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:47.024595 30278 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.024644 30246 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:47.025214 30246 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:47.025979 22833 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2364) took 1870 ms (client timeout 1510 ms). Trace:
I20260709 13:14:47.026144 30246 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:47.026444 22833 rpcz_store.cc:269] 0709 13:14:45.155827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.175430 (+ 19603us) service_pool.cc:224] Handling call
0709 13:14:47.025746 (+1850316us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:47.025876 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771966720}
I20260709 13:14:47.027395 30246 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.027653 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:47.028491 22833 raft_consensus.cc:2376] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25. Re-sending same reply.
I20260709 13:14:47.028744 30236 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:47.029196 30236 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:47.029491 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:47.030146 22833 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:47.019902 30431 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:47.027647 30233 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:47.034518 30324 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.035178 22839 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:47.036819 30236 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.038271 30236 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.039135 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2347) took 1960 ms. Trace:
I20260709 13:14:47.039535 22837 rpcz_store.cc:276] 0709 13:14:45.078596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.766334 (+687738us) service_pool.cc:224] Handling call
0709 13:14:47.038947 (+1272613us) spinlock_profiling.cc:243] Waited 7.58 ms on lock 0x7b54000a5d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:47.039072 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17432960}
I20260709 13:14:47.040719 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:47.009069 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:47.042949 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:47.043135 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:47.044713 30233 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:47.045037 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3744) took 2302 ms. Trace:
I20260709 13:14:47.045472 22908 rpcz_store.cc:276] 0709 13:14:44.742389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.342759 (+600370us) service_pool.cc:224] Handling call
0709 13:14:45.343079 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:45.354376 (+ 11297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.354408 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:45.368858 (+ 14450us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.359742 (+990884us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b180044afa0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:14:46.406298 (+ 46556us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:46.406690 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 13:14:46.406945 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.407200 (+   255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.004859 (+597659us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:47.044831 (+ 39972us) raft_consensus.cc:1685] finished
0709 13:14:47.044855 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.044981 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":215,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":24095,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":3823,"spinlock_wait_cycles":19192832,"thread_start_us":46358,"threads_started":1,"wal-append.queue_time_us":1034712}
I20260709 13:14:47.046764 30233 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.011670 30432 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.048094 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.048943 30432 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:47.049145 30327 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:47.049800 30327 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.050827 30371 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.051358 30371 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.053753 30234 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:46.949717 30239 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.012100 23466 raft_consensus.cc:2368] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:47.055757 30327 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:47.058326 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:47.059814 30247 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.060310 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:47.060511 30247 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:47.061555 30247 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:47.061954 30247 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:47.062475 30421 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:14:47.062732 22766 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:47.062858 30247 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.064213 30247 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:47.065457 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.066229 22914 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:47.069861 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:47.070600 23457 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:47.070487 22879 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:47.060357 30321 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:47.015691 30204 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:47.075805 30204 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.076925 30204 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:47.078269 30403 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.680s	user 0.003s	sys 0.000s
W20260709 13:14:47.080370 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.081564 30428 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.053449 30371 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.085569 30371 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.086231 30371 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.087785 30316 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.090723 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1316) took 1236 ms. Trace:
I20260709 13:14:47.041316 22837 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:47.107697 30213 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.108322 30213 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.021322 29933 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.109095 23462 raft_consensus.cc:1240] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:47.109956 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:47.110839 23459 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:47.110749 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:47.112439 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:47.112844 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:47.113739 23463 raft_consensus.cc:2368] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:47.096242 30297 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:47.116498 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:47.003454 22911 rpcz_store.cc:276] 0709 13:14:45.889575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.889744 (+   169us) service_pool.cc:224] Handling call
0709 13:14:45.890070 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:45.930024 (+ 39954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.930063 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:45.930671 (+   608us) log.cc:844] Serialized 52 byte log entry
0709 13:14:46.151011 (+220340us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:46.151056 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.996950 (+845894us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b54001d2254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:46.997043 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:46.997118 (+    75us) raft_consensus.cc:1685] finished
0709 13:14:46.997138 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.997831 (+   693us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":871,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":90866944,"thread_start_us":199741,"threads_started":1,"wal-append.queue_time_us":196438}
I20260709 13:14:47.098299 30290 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:47.119371 23459 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:47.120296 22912 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:47.120600 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.121853 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.122208 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:47.044736 22834 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:47.124053 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:47.099283 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:47.100498 30423 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 13:14:47.101809 30400 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.704s	user 0.012s	sys 0.000s
I20260709 13:14:47.101341 30426 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:47.047130 30308 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:47.184494 30377 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.345s	user 0.000s	sys 0.005s
I20260709 13:14:47.168912 30290 raft_consensus.cc:2764] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:14:47.188526 30276 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.125097 30369 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:14:47.195036 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:47.196679 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:47.103890 30409 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:14:47.123732 30351 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:47.096499 30304 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:47.099082 30419 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.230s	user 0.003s	sys 0.000s
I20260709 13:14:47.125463 22834 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:47.104990 22839 rpcz_store.cc:276] 0709 13:14:45.854375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.734032 (+879657us) service_pool.cc:224] Handling call
0709 13:14:46.779726 (+ 45694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.035620 (+255894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.090439 (+ 54819us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400322dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:47.090659 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586918656}
I20260709 13:14:47.129642 22835 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:47.137403 22915 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 13:14:47.217697 22834 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:14:47.142920 22837 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:47.219115 22837 raft_consensus.cc:1240] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:14:47.199468 30344 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:14:47.220973 30344 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:47.143319 23460 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:47.221360 30344 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:47.201879 30233 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:47.198454 30384 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:14:47.182116 30404 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.244364 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:47.163175 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 64004.8us
W20260709 13:14:47.168216 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 107984us
I20260709 13:14:47.245345 22914 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:47.246749 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:47.171679 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:47.172302 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:47.255314 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.258728 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3519) took 1048 ms. Trace:
I20260709 13:14:47.079789 30239 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:47.207096 30431 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:47.212580 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:47.222220 30233 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:47.223804 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:47.149894 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:47.274085 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1310) took 1709 ms. Trace:
I20260709 13:14:47.274480 22915 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:47.238888 30290 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:47.225308 22835 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:14:47.175566 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.262290 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:47.259395 22762 rpcz_store.cc:276] 0709 13:14:46.210816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.210993 (+   177us) service_pool.cc:224] Handling call
0709 13:14:46.211474 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.211948 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.211990 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:46.226799 (+ 14809us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.360502 (+133703us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:46.360558 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.360656 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.101136 (+740480us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:47.258416 (+157280us) raft_consensus.cc:1685] finished
0709 13:14:47.258442 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.258655 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":705,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8170,"spinlock_wait_cycles":3782528,"thread_start_us":97739,"threads_started":1,"wal-append.queue_time_us":161381}
W20260709 13:14:47.279803 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67358.2us
W20260709 13:14:47.280316 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 130572us
I20260709 13:14:47.280580 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:47.283130 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.176832 30415 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:14:47.265847 22763 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:47.273180 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.286731 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.287658 22807 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:47.288338 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.141162 22833 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:47.290496 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:47.291219 22833 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:47.292884 22833 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:47.263777 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:47.294116 22912 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:47.295441 22879 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:47.274596 22838 rpcz_store.cc:276] 0709 13:14:45.565078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.643266 (+1078188us) service_pool.cc:224] Handling call
0709 13:14:46.643556 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.643928 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.643953 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:46.694558 (+ 50605us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.812580 (+118022us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:46.812646 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.812813 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.099896 (+287083us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:47.247608 (+147712us) raft_consensus.cc:1685] finished
0709 13:14:47.247643 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.273958 (+ 26315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":29578,"spinlock_wait_cycles":3081472,"thread_start_us":31095,"threads_started":1,"wal-append.queue_time_us":112294}
I20260709 13:14:47.300096 22838 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:47.302604 22834 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:14:47.264565 22761 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:47.306114 22833 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:47.314651 22833 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:47.307654 22838 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:14:47.312247 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.321619 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:47.326570 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.329047 22914 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:47.335258 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:47.337045 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.338519 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:47.338867 30296 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.339772 30296 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:47.340545 22914 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:14:47.342132 22914 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:47.342511 22914 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:47.343493 22914 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.344743 22914 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:47.347565 30399 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.959s	user 0.004s	sys 0.000s
I20260709 13:14:47.350664 30326 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:47.351177 30326 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:47.351558 30326 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:47.209712 30304 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.355599 30304 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:47.355980 22914 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:14:47.356509 22914 pending_rounds.cc:85] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 9
I20260709 13:14:47.356874 22914 pending_rounds.cc:107] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:14:47.357220 22914 raft_consensus.cc:2889] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:47.262467 30404 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:47.361554 22914 raft_consensus.cc:1275] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:47.364409 22915 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
W20260709 13:14:47.316097 30148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.004s
I20260709 13:14:47.279500 30431 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.370324 22913 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:47.374763 22913 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:47.279175 30233 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:47.373870 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.379029 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:47.277668 30290 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:47.383976 22833 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:14:47.384897 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63402us
I20260709 13:14:47.315706 30327 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:47.388571 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:47.329933 30334 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:47.392850 30407 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:47.396683 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:47.335321 30213 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:47.358034 30415 consensus_queue.cc:1243] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:14:47.323053 30413 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:47.387176 22913 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.416442 22913 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:47.371264 30148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.012s	sys 0.004s
I20260709 13:14:47.397703 22833 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:47.402523 22915 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:47.412809 22911 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:47.435289 30243 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:14:47.401342 30334 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.429039 30316 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:47.384104 30311 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:47.453711 22909 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.014s	sys 0.034s
I20260709 13:14:47.435658 30275 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:47.454420 22909 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.014s	sys 0.034s
W20260709 13:14:47.454686 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.383658 30431 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.390497 30276 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:47.388190 30369 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:14:47.461735 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 1065) took 1025 ms. Trace:
I20260709 13:14:47.447335 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:47.469487 23458 rpcz_store.cc:276] 0709 13:14:46.436007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.436249 (+   242us) service_pool.cc:224] Handling call
0709 13:14:46.436595 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:46.698319 (+261724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.698354 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:46.759106 (+ 60752us) log.cc:844] Serialized 52 byte log entry
0709 13:14:46.963435 (+204329us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:46.963469 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.963637 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.461354 (+497717us) raft_consensus.cc:1685] finished
0709 13:14:47.461379 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.461634 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64121,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":42933,"thread_start_us":409542,"threads_started":2,"wal-append.queue_time_us":619234}
W20260709 13:14:47.446355 30362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.008s	sys 0.050s
I20260709 13:14:47.453298 30334 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:47.563979 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.456028 30413 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:47.462821 30243 consensus_queue.cc:1243] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:47.462337 30148 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:47.443414 30327 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:47.394246 30416 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:14:47.394381 30294 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:14:47.393999 30342 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
W20260709 13:14:47.589983 30362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.008s	sys 0.059s
I20260709 13:14:47.571635 29846 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.577170 30428 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:47.594573 30422 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.626s	user 0.002s	sys 0.000s
I20260709 13:14:47.464900 30311 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.470992 30417 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.158s
W20260709 13:14:47.629693 30246 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.602s	user 0.017s	sys 0.008s
I20260709 13:14:47.627233 30369 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:47.628224 30276 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:47.623512 30309 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.527469 30290 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.626964 30327 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.690749 30417 consensus_queue.cc:1243] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:47.693684 30369 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:47.727254 30432 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.004s
W20260709 13:14:47.746846 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.671386 30428 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.749552 30428 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.752812 22840 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 13:14:47.760998 29892 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:47.761765 29892 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:47.762148 30351 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:47.762852 30351 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:47.763891 30351 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:47.764760 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 508148us
I20260709 13:14:47.765197 30351 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.657696 29846 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.734565 30309 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.743012 30344 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.622409 30148 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:47.746361 30297 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.001s
W20260709 13:14:47.774983 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 494485us
I20260709 13:14:47.747090 30369 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:47.786440 22913 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:47.717837 30276 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.812420 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:47.815132 22761 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:47.839471 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.823220 30309 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:47.823484 29846 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.824522 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1114) took 1066 ms. Trace:
I20260709 13:14:47.827997 30327 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.768198 30371 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:47.828902 22911 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:14:47.831321 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60930.6us
W20260709 13:14:47.793776 30305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.009s	sys 0.014s
W20260709 13:14:47.835083 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 539727us
I20260709 13:14:47.837409 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.840164 22762 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:47.843220 22759 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23.
I20260709 13:14:47.783459 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:47.852298 22841 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 15.
I20260709 13:14:47.853715 22909 rpcz_store.cc:276] 0709 13:14:46.758519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.758660 (+   141us) service_pool.cc:224] Handling call
0709 13:14:46.781403 (+ 22743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.799148 (+ 17745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.799189 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:46.807292 (+  8103us) log.cc:844] Serialized 26 byte log entry
0709 13:14:47.455138 (+647846us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:47.455209 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.455604 (+   395us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.824081 (+368477us) raft_consensus.cc:1685] finished
0709 13:14:47.824123 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.824408 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12800,"raft.queue_time_us":23497,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3426,"thread_start_us":601008,"threads_started":1,"wal-append.queue_time_us":899442}
W20260709 13:14:47.855651 22760 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.006s	sys 0.017s
W20260709 13:14:47.792951 22837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.012s	sys 0.024s
I20260709 13:14:47.730114 30413 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 23.9, Last appended by leader: 9, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.836290 30344 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:47.865382 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.896080 22879 leader_election.cc:400] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
W20260709 13:14:47.867228 22837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.012s	sys 0.026s
W20260709 13:14:47.865391 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82025.3us
W20260709 13:14:47.897378 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.866199 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:47.897854 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:47.899410 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:47.866369 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 419118us
I20260709 13:14:47.901036 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:47.901196 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.904170 22762 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:47.866961 22760 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.006s	sys 0.017s
I20260709 13:14:47.909335 22764 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:47.913043 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.905613 22730 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:47.862534 30305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.009s	sys 0.014s
I20260709 13:14:47.929342 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.866466 30359 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.943400 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:47.959022 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:47.954787 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.959319 22760 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:47.960256 22762 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:14:47.940187 23432 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.018s	sys 0.011s
I20260709 13:14:47.961850 22762 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:47.963120 22762 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:47.962070 23432 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.019s	sys 0.011s
W20260709 13:14:47.964581 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 839314us
W20260709 13:14:47.965845 23432 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.717s (SENT) [suppressed 12 similar messages]
I20260709 13:14:47.966602 22764 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.943198 30416 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:47.922156 30278 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:14:47.957901 30428 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:47.970662 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.996867 30416 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:47.971323 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:47.997892 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.998883 22759 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:47.997977 22761 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:47.999922 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:47.976322 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:48.000507 30416 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:47.978116 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:47.996871 30428 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:47.981964 22760 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:14:48.001688 30428 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:48.001984 22730 leader_election.cc:400] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:48.002552 30278 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:48.002944 22730 leader_election.cc:403] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:47.952687 30385 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:48.001905 22760 pending_rounds.cc:85] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 15
I20260709 13:14:48.004631 30385 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:48.005219 30148 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:48.004665 22760 pending_rounds.cc:107] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:14:48.006745 22760 raft_consensus.cc:2889] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:48.006409 30385 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:47.970685 30432 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:14:47.943567 30212 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.051s
I20260709 13:14:47.980971 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2836) took 1477 ms. Trace:
I20260709 13:14:48.010849 22837 rpcz_store.cc:276] 0709 13:14:46.503138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.219968 (+716830us) service_pool.cc:224] Handling call
0709 13:14:47.220301 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.220689 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.220719 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:47.221275 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:14:47.897056 (+675781us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:47.897287 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.897401 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.980712 (+ 83311us) raft_consensus.cc:1685] finished
0709 13:14:47.980739 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.980899 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25149,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3859,"thread_start_us":670192,"threads_started":1,"wal-append.queue_time_us":756326}
W20260709 13:14:47.982327 30308 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.791s	user 0.015s	sys 0.000s
I20260709 13:14:48.001108 23458 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:47.932627 30243 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:48.006924 30321 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.935s	user 0.010s	sys 0.004s
I20260709 13:14:48.008653 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:48.011998 30326 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.660s	user 0.013s	sys 0.000s
I20260709 13:14:48.046653 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.048662 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:48.012915 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:48.049808 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.003223 30359 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.003361 30278 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:48.000617 22759 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:47.902523 30309 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:47.944592 30334 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.017382 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2837) took 1514 ms. Trace:
I20260709 13:14:47.923506 30436 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:48.023319 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.029999 22761 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:14:48.024696 22762 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.092428 22762 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:48.051275 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.086366 22839 rpcz_store.cc:276] 0709 13:14:46.503376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.225038 (+721662us) service_pool.cc:224] Handling call
0709 13:14:47.268074 (+ 43036us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:47.268622 (+   548us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:47.335540 (+ 66918us) log.cc:844] Serialized 14 byte log entry
0709 13:14:47.706765 (+371225us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:47.729948 (+ 23183us) log.cc:844] Serialized 52 byte log entry
0709 13:14:47.730145 (+   197us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:47.730392 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 13:14:47.730752 (+   360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.730850 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.758653 (+ 27803us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b180018c580. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:47.819486 (+ 60833us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b180018c580. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f68a59c558f 00007f689ea04571 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd
0709 13:14:48.016900 (+197414us) raft_consensus.cc:1685] finished
0709 13:14:48.016938 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.017278 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119204,"prepare.queue_time_us":105834,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":63,"raft.queue_time_us":433,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":67580,"spinlock_wait_cycles":14700672,"thread_start_us":360085,"threads_started":1,"wal-append.queue_time_us":558117}
I20260709 13:14:48.095300 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.096724 22806 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23.
I20260709 13:14:48.058768 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:48.097013 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 73793.3us
I20260709 13:14:48.097766 22760 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
W20260709 13:14:48.012549 30233 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.006s
I20260709 13:14:48.062209 30449 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.062050 22832 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:48.067773 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:14:48.103443 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:48.103906 22837 raft_consensus.cc:606] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.104651 22837 raft_consensus.cc:614] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:48.048422 29933 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.939s	user 0.012s	sys 0.008s
I20260709 13:14:48.069738 30311 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.023458 30148 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.117899 23463 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:14:48.056365 22732 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.529s (SENT)
I20260709 13:14:48.118841 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:48.070544 30326 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.075245 30276 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.078088 30369 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.100983 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.097052 22806 leader_election.cc:403] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:14:48.140517 22762 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:14:48.141726 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45151.6us
I20260709 13:14:48.105621 23466 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:48.107834 22765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.003s	sys 0.052s
I20260709 13:14:48.081964 29846 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:48.114228 30308 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:48.114835 23459 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
W20260709 13:14:48.115805 30442 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:48.119001 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:48.167824 23460 raft_consensus.cc:2368] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19.
I20260709 13:14:48.093469 30246 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 16. Current term is 17.
I20260709 13:14:48.116477 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1115) took 1357 ms. Trace:
I20260709 13:14:48.090145 30212 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:48.123680 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.126258 23462 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:14:48.177330 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.177654 22839 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:48.060781 30293 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:14:48.094162 30243 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:48.187783 30293 consensus_queue.cc:1243] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:48.189787 30278 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.198303 30458 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.199226 30458 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.205595 30311 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:14:48.211283 30326 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:48.130743 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74427.1us
W20260709 13:14:48.150233 22765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.004s	sys 0.052s
I20260709 13:14:48.217103 30326 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:48.153039 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 52125.8us
I20260709 13:14:48.124939 30148 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.217768 30326 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.097144 30362 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:14:48.219062 30447 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:48.219822 30447 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:48.220041 30148 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.117985 30321 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.220372 30447 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:48.220700 30447 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:48.220782 30321 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.221678 30326 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.222349 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.188792 30243 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.223786 30321 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.224495 30447 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.160281 30417 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:48.226141 30447 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:48.160035 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.161684 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:48.226912 30334 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:48.228518 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68693.2us
I20260709 13:14:48.229620 30369 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:48.229934 30213 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:48.230590 30213 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:48.232170 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.229648 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:48.129733 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.000s	sys 0.021s
I20260709 13:14:48.172055 22910 rpcz_store.cc:276] 0709 13:14:46.758824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.780501 (+ 21677us) service_pool.cc:224] Handling call
0709 13:14:46.796623 (+ 16122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.279222 (+482599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.279273 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:47.373677 (+ 94404us) log.cc:844] Serialized 26 byte log entry
0709 13:14:47.843333 (+469656us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:47.843421 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.843537 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.116172 (+272635us) raft_consensus.cc:1685] finished
0709 13:14:48.116204 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.116392 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2471,"raft.queue_time_us":80316,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":881548,"threads_started":2,"wal-append.queue_time_us":712411}
I20260709 13:14:48.107981 30233 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.116616 30455 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:48.185525 22762 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:48.117709 30234 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:48.237949 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.000s	sys 0.022s
I20260709 13:14:48.231071 30213 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:48.230770 30369 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:14:48.262944 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2344) took 3185 ms (client timeout 2158 ms). Trace:
I20260709 13:14:48.256106 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:48.255934 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:48.264055 22832 rpcz_store.cc:269] 0709 13:14:45.077709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.520873 (+443164us) service_pool.cc:224] Handling call
0709 13:14:48.262697 (+2741824us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b540028aad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:48.262852 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5803875072}
I20260709 13:14:48.270695 22757 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:48.272993 22758 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:48.249491 22764 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:48.247874 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.111833 30449 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.171379 30276 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23.
I20260709 13:14:48.234061 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.281663 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 2999) took 1025 ms. Trace:
I20260709 13:14:48.264436 30234 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.242920 30334 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:48.244493 30311 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:48.292531 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.274322 30356 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.036s	sys 0.038s
I20260709 13:14:48.295882 22910 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:48.264700 30417 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:48.306218 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58490.8us
I20260709 13:14:48.323493 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.298956 22766 rpcz_store.cc:276] 0709 13:14:47.256709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.257702 (+   993us) service_pool.cc:224] Handling call
0709 13:14:47.258042 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.272773 (+ 14731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.272811 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:47.315875 (+ 43064us) log.cc:844] Serialized 26 byte log entry
0709 13:14:47.855410 (+539535us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:47.855471 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.855549 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.281365 (+425816us) raft_consensus.cc:1685] finished
0709 13:14:48.281388 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.281587 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14927,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":46625,"thread_start_us":463504,"threads_started":1,"wal-append.queue_time_us":822362}
I20260709 13:14:48.324281 22909 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:14:48.321992 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 88069.8us
I20260709 13:14:48.338167 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:48.320523 30431 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.836s	user 0.015s	sys 0.000s
I20260709 13:14:48.349452 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.240330 30212 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:48.322953 30356 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.037s	sys 0.039s
I20260709 13:14:48.306149 30455 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:48.364315 22758 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:48.324596 30308 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:48.243577 30428 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.300356 30148 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:48.368628 22907 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:14:48.364467 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.320060 30234 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:48.319762 30436 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:48.371923 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.372714 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.335116 30407 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:48.385347 22909 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 27.
I20260709 13:14:48.387651 30148 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.388774 30334 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.389429 30334 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.393301 30212 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:48.394035 30334 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.393729 30436 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.370608 30431 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.397872 22760 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:14:48.400262 30436 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.402252 30431 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.403322 22763 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 24. Ops: []
I20260709 13:14:48.404539 30428 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.390072 30148 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.386761 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:48.407377 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.410355 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.411768 22907 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:48.413169 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:48.414009 22909 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:48.414965 30234 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:48.418159 22912 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 1121) took 1317 ms (client timeout 1687 ms). Trace:
W20260709 13:14:48.418828 22912 rpcz_store.cc:269] 0709 13:14:47.100599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.112152 (+ 11553us) service_pool.cc:224] Handling call
0709 13:14:48.417877 (+1305725us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b54001e3cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:48.418073 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16424320}
I20260709 13:14:48.410598 22758 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:48.424811 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3005) took 1159 ms. Trace:
I20260709 13:14:48.368943 30321 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:48.426011 30321 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.425311 22757 rpcz_store.cc:276] 0709 13:14:47.265135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.273375 (+  8240us) service_pool.cc:224] Handling call
0709 13:14:47.316350 (+ 42975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.273947 (+957597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.424498 (+150551us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b540037f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:48.424731 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194259200}
I20260709 13:14:48.428512 30321 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.430109 29892 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.431010 30461 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.431788 30461 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.433384 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.431062 30234 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.434299 22764 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:48.435050 30233 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.435436 30461 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.435483 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.386073 30455 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.437786 30369 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.002s
I20260709 13:14:48.437558 30458 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:48.442798 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.441454 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.448084 22757 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:48.454238 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.382592 30308 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:48.385901 30327 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:48.436369 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.347921 30362 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:14:48.450181 30310 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:48.457183 29892 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:48.461822 30344 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.003s
W20260709 13:14:48.497534 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.465068 30308 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:48.489054 22909 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:48.508955 22764 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:14:48.498667 30306 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.466877 30461 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:48.541972 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 201561us
I20260709 13:14:48.492097 30324 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.494789 30310 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.490356 30334 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:48.490926 30327 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.526226 22758 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
W20260709 13:14:48.528496 30434 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.073s	user 0.003s	sys 0.000s
I20260709 13:14:48.495730 29933 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:48.467026 30293 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.558902 30293 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.547184 30256 consensus_queue.cc:1048] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:14:48.518472 30455 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.506419 30276 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.562647 30276 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:48.563663 22841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2343) took 3486 ms (client timeout 1674 ms). Trace:
W20260709 13:14:48.564133 22841 rpcz_store.cc:269] 0709 13:14:45.077371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.104464 (+ 27093us) service_pool.cc:224] Handling call
0709 13:14:48.563422 (+3458958us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b54001bc954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:48.563584 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6163890432}
I20260709 13:14:48.564981 30276 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.490629 30371 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.565732 30371 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.565343 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:48.567625 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.568039 30371 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.547143 30306 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.464093 30417 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.568297 22841 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:48.569352 30385 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:48.570003 30385 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.570691 30306 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.571323 29846 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:48.572170 30385 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.573015 30293 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:48.573101 22906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51142 (request call id 1116) took 1786 ms (client timeout 1514 ms). Trace:
W20260709 13:14:48.573638 22906 rpcz_store.cc:269] 0709 13:14:46.787220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.787358 (+   138us) service_pool.cc:224] Handling call
0709 13:14:48.572862 (+1785504us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400043654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:48.573013 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2674681600}
I20260709 13:14:48.575664 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.577997 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1314) took 2903 ms. Trace:
I20260709 13:14:48.578536 22836 rpcz_store.cc:276] 0709 13:14:45.674831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.735788 (+1060957us) service_pool.cc:224] Handling call
0709 13:14:46.736238 (+   450us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:46.736854 (+   616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.736904 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:46.786855 (+ 49951us) log.cc:844] Serialized 52 byte log entry
0709 13:14:47.166774 (+379919us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:47.189755 (+ 22981us) log.cc:844] Serialized 14 byte log entry
0709 13:14:47.190021 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.190106 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.527586 (+1337480us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:48.577680 (+ 50094us) raft_consensus.cc:1685] finished
0709 13:14:48.577799 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.577939 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7702,"prepare.queue_time_us":23508,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":8646,"raft.queue_time_us":16681,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":13526,"spinlock_wait_cycles":12415872,"thread_start_us":343644,"threads_started":1,"wal-append.queue_time_us":564823}
I20260709 13:14:48.582929 22836 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:48.583508 30327 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.584833 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2832) took 2085 ms. Trace:
I20260709 13:14:48.584729 22841 raft_consensus.cc:1217] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Deduplicated request from leader. Original: 19.10->[21.11-25.14]   Dedup: 21.11->[22.12-25.14]
I20260709 13:14:48.585464 22835 rpcz_store.cc:276] 0709 13:14:46.499843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.217124 (+717281us) service_pool.cc:224] Handling call
0709 13:14:47.225113 (+  7989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.277880 (+ 52767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.584581 (+1306701us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b54002b3250. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:48.584760 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17708416}
I20260709 13:14:48.586833 30290 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:48.528515 30309 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:48.590948 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.593349 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2835) took 2090 ms. Trace:
W20260709 13:14:48.552785 30351 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.779s	user 0.010s	sys 0.006s
I20260709 13:14:48.550568 30310 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:48.593843 22834 rpcz_store.cc:276] 0709 13:14:46.502872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.218478 (+715606us) service_pool.cc:224] Handling call
0709 13:14:47.302255 (+ 83777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.303236 (+   981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.593119 (+1289883us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b540028b4d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:48.593281 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191781376}
I20260709 13:14:48.585713 22841 pending_rounds.cc:85] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 11
I20260709 13:14:48.599313 22838 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:48.599709 22841 pending_rounds.cc:107] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:14:48.605165 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.608078 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.610302 22909 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:14:48.611189 22841 raft_consensus.cc:2889] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:48.611614 22908 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:14:48.612047 22908 pending_rounds.cc:85] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 11
I20260709 13:14:48.612277 22908 pending_rounds.cc:107] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:14:48.612511 22908 raft_consensus.cc:2889] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:48.549095 30461 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.549844 30324 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.599747 30278 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.615833 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.598358 30385 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.554417 30334 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.636026 22909 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
W20260709 13:14:48.670713 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.670399 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.674685 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.682626 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.683808 22909 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 13:14:48.690884 30013 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 13:14:48.692327 30449 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:48.697151 30478 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:14:48.701951 30233 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:48.702848 30233 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.704447 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:48.705116 30233 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.706699 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.705124 22838 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:14:48.649083 30278 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.708536 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.745075 22840 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:48.745400 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:48.746129 22762 raft_consensus.cc:2376] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20. Re-sending same reply.
I20260709 13:14:48.708829 30483 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.752192 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.710752 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3777) took 1499 ms. Trace:
I20260709 13:14:48.713532 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3002) took 1453 ms. Trace:
I20260709 13:14:48.754402 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.753679 22908 rpcz_store.cc:276] 0709 13:14:47.211098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.844715 (+633617us) service_pool.cc:224] Handling call
0709 13:14:47.845159 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.612928 (+767769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.710409 (+ 97481us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54003c96d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:48.710672 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751417600}
W20260709 13:14:48.758315 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.719724 30324 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:48.721853 30461 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.763240 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.763998 30278 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.723449 30213 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.664085 30385 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:48.731833 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.748167 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.748220 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.755529 22909 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:48.719518 30334 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:48.753919 22765 rpcz_store.cc:276] 0709 13:14:47.260013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.260179 (+   166us) service_pool.cc:224] Handling call
0709 13:14:47.323162 (+ 62983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.832142 (+508980us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b48002ac0d8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:14:48.216760 (+384618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.216795 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.217352 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.493861 (+276509us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:48.493917 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.494037 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.713096 (+219059us) raft_consensus.cc:1685] finished
0709 13:14:48.713131 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.713429 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2383,"raft.queue_time_us":22233,"raft.run_cpu_time_us":2399,"raft.run_wall_time_us":486506,"spinlock_wait_cycles":33473280,"thread_start_us":1166698,"threads_started":2,"wal-append.queue_time_us":488192}
I20260709 13:14:48.777235 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.752625 22838 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:14:48.732448 30309 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:48.760231 30483 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.753167 30351 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.764082 22837 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
W20260709 13:14:48.766033 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.767027 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:48.730353 30276 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:48.778262 22909 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:48.779585 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.776135 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.781989 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.796837 22838 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:48.799942 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.800745 22837 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:48.801889 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:48.806054 22837 raft_consensus.cc:2393] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:48.776196 30278 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:48.781261 30309 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.775437 30461 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:48.765110 30256 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:14:48.809238 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:48.829646 22909 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:14:48.810729 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:48.785861 30246 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.814417 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:48.803536 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.790766 30204 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:48.820485 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.789639 30013 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:48.840202 22837 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:48.841980 30432 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:48.812158 30344 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.816911 30309 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:48.799557 30276 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:48.864181 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.812597 30324 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:48.843114 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.791821 30351 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:48.875919 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 65406.6us
W20260709 13:14:48.876044 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 298169us
I20260709 13:14:48.918967 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3771) took 1792 ms. Trace:
I20260709 13:14:48.925854 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.926594 22758 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:48.927011 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:14:48.927716 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.927973 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:48.925441 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 82524.3us
I20260709 13:14:48.862082 30432 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.818943 30278 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.921305 22914 rpcz_store.cc:276] 0709 13:14:47.127005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.371497 (+244492us) service_pool.cc:224] Handling call
0709 13:14:47.372411 (+   914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.372701 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:47.856363 (+483662us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.331504 (+475141us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.535897 (+204393us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b18000a2400. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:48.918418 (+382521us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18000a2400. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:14:48.918601 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.918655 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:48.918697 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.918779 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.918906 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39993,"prepare.queue_time_us":457698,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":122,"raft.queue_time_us":7019,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":25897,"spinlock_wait_cycles":50164224,"thread_start_us":915187,"threads_started":2,"wal-append.queue_time_us":644528}
I20260709 13:14:48.928915 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.942646 22837 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:14:48.915946 30334 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:48.959329 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.862704 30246 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.835793 30461 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:48.973346 22758 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:48.940030 30321 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:48.872579 30013 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.972157 22807 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:48.873469 30344 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:48.984380 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.001339 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.018522 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 241238us
I20260709 13:14:48.891428 30324 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:48.952138 30432 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:48.977452 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:49.041426 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.960357 30334 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:49.037480 22807 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:49.042861 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 65497.9us
I20260709 13:14:49.034614 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:48.980557 30321 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.979977 30369 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:14:49.049189 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87943.5us
I20260709 13:14:48.957901 30306 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:14:48.960114 30293 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.083s
I20260709 13:14:49.032759 30327 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:49.061288 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:49.060447 30213 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.067270 30324 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.067929 30324 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:48.978475 30455 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:14:49.069818 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3528) took 1282 ms. Trace:
I20260709 13:14:49.070269 22763 rpcz_store.cc:276] 0709 13:14:47.787639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.787816 (+   177us) service_pool.cc:224] Handling call
0709 13:14:48.197299 (+409483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.403776 (+206477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.588442 (+184666us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:49.069571 (+481129us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540002e9d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:49.069747 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413931008}
I20260709 13:14:49.071738 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:49.072425 30305 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:49.062207 22758 raft_consensus.cc:2410] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:14:49.048909 30334 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:49.048869 30311 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:49.051642 30148 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.050683 30483 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.055604 30321 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:49.070079 30489 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:14:49.060699 30306 consensus_queue.cc:1243] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:49.057379 30369 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:49.091204 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 925) took 1009 ms. Trace:
I20260709 13:14:49.064067 30327 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:49.110149 23458 rpcz_store.cc:276] 0709 13:14:48.082694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.083534 (+   840us) service_pool.cc:224] Handling call
0709 13:14:48.084407 (+   873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.085190 (+   783us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:48.455187 (+369997us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.090667 (+635480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.090762 (+    95us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:49.090830 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.090927 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.091113 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":105388,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":198930,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11208,"thread_start_us":982179,"threads_started":3,"wal-append.queue_time_us":605920}
W20260709 13:14:49.107569 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 311715us
I20260709 13:14:49.019428 30276 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:49.112586 23431 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:49.118716 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:49.092307 30311 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:49.146749 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.095912 30416 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:49.170778 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58282.5us
I20260709 13:14:49.185480 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.185741 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:49.186266 23458 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:49.186334 23457 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:49.107728 30385 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.099949 30371 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:49.113660 30369 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:49.113335 30327 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:49.089044 30204 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:49.222954 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:49.229053 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.230168 23463 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:49.183691 30311 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:49.230361 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:49.198817 30489 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:49.098205 30148 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.239729 30351 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:49.192163 30486 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 13:14:49.239025 22762 pending_rounds.cc:85] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:49.265774 22762 pending_rounds.cc:107] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:14:49.266160 22762 raft_consensus.cc:2889] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:49.216327 30305 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:49.258911 30493 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 13:14:49.262373 30344 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:49.271847 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2841) took 1352 ms. Trace:
I20260709 13:14:49.392475 22839 rpcz_store.cc:276] 0709 13:14:47.919521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.197218 (+277697us) service_pool.cc:224] Handling call
0709 13:14:48.197574 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.277552 (+ 79978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.277612 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.277678 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:48.277730 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.271529 (+993799us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b54001ff4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:49.271595 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.271762 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183182720}
I20260709 13:14:49.394686 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:49.329861 22732 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.555s (SENT)
I20260709 13:14:49.396296 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:49.401882 22839 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 13:14:49.265513 30488 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.335s	user 0.002s	sys 0.000s
I20260709 13:14:49.263985 30510 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:49.226486 30371 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:49.420873 30308 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.880s	user 0.014s	sys 0.000s
W20260709 13:14:49.424018 30310 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.817s	user 0.014s	sys 0.000s
I20260709 13:14:49.436543 22839 raft_consensus.cc:1275] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:14:49.441705 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 372358us
W20260709 13:14:49.442799 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 383088us
I20260709 13:14:49.443794 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:49.441088 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.444855 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.447000 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.449360 22807 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.467381 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.423015 30246 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:49.471537 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 296255us
I20260709 13:14:49.391654 30305 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.427518 30013 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.433310 30278 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.430851 30510 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:49.434092 30518 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:49.438876 30496 log.cc:927] Time spent T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
W20260709 13:14:49.439257 30492 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.392s	user 0.001s	sys 0.001s
I20260709 13:14:49.433003 30359 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:49.488724 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3551) took 1054 ms. Trace:
I20260709 13:14:49.505470 22764 rpcz_store.cc:276] 0709 13:14:48.434718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.435128 (+   410us) service_pool.cc:224] Handling call
0709 13:14:49.488468 (+1053340us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b5400393354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:49.488652 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166136576}
I20260709 13:14:49.499189 30517 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:14:49.474233 30290 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.887s	user 0.010s	sys 0.007s
I20260709 13:14:49.502193 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.460498 30306 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:14:49.504280 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2386) took 1001 ms. Trace:
I20260709 13:14:49.504212 22838 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:49.515268 22835 rpcz_store.cc:276] 0709 13:14:48.503481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.586732 (+ 83251us) service_pool.cc:224] Handling call
0709 13:14:48.587060 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.587296 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:48.588113 (+   817us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.503725 (+915612us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.503905 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.503959 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:49.504009 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.504078 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.504218 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28849,"prepare.queue_time_us":38391,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":11096,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":15029,"thread_start_us":884178,"threads_started":2}
W20260709 13:14:49.490803 30495 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
W20260709 13:14:49.511857 30501 log.cc:927] Time spent T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:14:49.519553 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:49.448654 30308 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:49.462989 30310 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:49.500909 30520 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:49.433085 30204 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.517225 30293 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:49.494755 30456 consensus_queue.cc:1048] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:14:49.522791 30239 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.520107 30461 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:49.520569 30306 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:49.561556 30499 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.442s	user 0.001s	sys 0.001s
I20260709 13:14:49.554481 30290 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:49.579846 22806 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.003s	sys 0.012s
I20260709 13:14:49.516019 30517 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:49.588042 22806 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.003s	sys 0.012s
I20260709 13:14:49.605089 30519 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.605777 30519 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.573185 30520 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:49.609102 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3789) took 1278 ms. Trace:
I20260709 13:14:49.609661 22913 rpcz_store.cc:276] 0709 13:14:48.330659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.330809 (+   150us) service_pool.cc:224] Handling call
0709 13:14:48.331306 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:48.351999 (+ 20693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.352050 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:48.388129 (+ 36079us) log.cc:844] Serialized 52 byte log entry
0709 13:14:48.595334 (+207205us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:48.621276 (+ 25942us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.042410 (+421134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.042539 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:49.608775 (+566236us) raft_consensus.cc:1685] finished
0709 13:14:49.608798 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.609019 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":923517,"prepare.queue_time_us":25666,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":5062,"raft.queue_time_us":450042,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":15256,"thread_start_us":206788,"threads_started":1,"wal-append.queue_time_us":401748}
I20260709 13:14:49.512804 30518 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.617340 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1168) took 1384 ms. Trace:
I20260709 13:14:49.617885 22761 rpcz_store.cc:276] 0709 13:14:48.233094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.562024 (+328930us) service_pool.cc:224] Handling call
0709 13:14:48.562462 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.562834 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.562867 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.563388 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.916578 (+353190us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:48.916645 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.916753 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:49.617047 (+700294us) raft_consensus.cc:1685] finished
0709 13:14:49.617078 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.617258 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2879,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":352647,"threads_started":1,"wal-append.queue_time_us":625357}
I20260709 13:14:49.566756 30515 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:49.624965 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 105453us
W20260709 13:14:49.634171 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.587044 30306 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:49.569320 30455 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:14:49.603065 30461 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:49.623611 30506 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 13:14:49.645870 23432 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.008s	sys 0.008s
W20260709 13:14:49.646246 23432 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.008s	sys 0.008s
I20260709 13:14:49.636725 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.614318 30517 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:49.647689 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3550) took 1218 ms. Trace:
W20260709 13:14:49.640177 30502 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:14:49.657691 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1335) took 1635 ms. Trace:
I20260709 13:14:49.659727 30519 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:49.658489 22833 rpcz_store.cc:276] 0709 13:14:48.022275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.576843 (+554568us) service_pool.cc:224] Handling call
0709 13:14:48.577224 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.596540 (+ 19316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.596584 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.602142 (+  5558us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.910564 (+308422us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:48.910635 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.910751 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:49.657290 (+746539us) raft_consensus.cc:1685] finished
0709 13:14:49.657326 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.657590 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18888,"raft.queue_time_us":21125,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5928,"spinlock_wait_cycles":1458560,"thread_start_us":289266,"threads_started":1,"wal-append.queue_time_us":479648}
W20260709 13:14:49.660744 30514 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 13:14:49.646528 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 725820us
I20260709 13:14:49.649371 22761 rpcz_store.cc:276] 0709 13:14:48.429297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.635213 (+1205916us) service_pool.cc:224] Handling call
0709 13:14:49.647170 (+ 11957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.647358 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:49.647386 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.647429 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:49.647461 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.647523 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.647632 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.648059 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.648054 23457 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:49.615025 30290 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.590245 30239 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.671825 30530 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:49.672487 30530 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:49.672988 30530 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:49.646108 30461 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.666884 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:49.668432 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:49.688113 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3552) took 1246 ms. Trace:
I20260709 13:14:49.685775 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2387) took 1181 ms. Trace:
I20260709 13:14:49.691089 22761 rpcz_store.cc:276] 0709 13:14:48.441912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.663523 (+1221611us) service_pool.cc:224] Handling call
0709 13:14:49.685051 (+ 21528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.685249 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:49.685281 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.685317 (+    36us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:49.685351 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.685409 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.688049 (+  2640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.707897 22761 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:49.708878 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3007) took 1115 ms. Trace:
I20260709 13:14:49.709218 22761 rpcz_store.cc:276] 0709 13:14:48.593494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.694038 (+1100544us) service_pool.cc:224] Handling call
0709 13:14:49.694526 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.708458 (+ 13932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.708813 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.710847 22761 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:49.711585 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3008) took 1117 ms. Trace:
I20260709 13:14:49.711969 22761 rpcz_store.cc:276] 0709 13:14:48.593996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.710139 (+1116143us) service_pool.cc:224] Handling call
0709 13:14:49.710597 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.711272 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.711512 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.715903 22913 raft_consensus.cc:2393] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:49.691344 22834 rpcz_store.cc:276] 0709 13:14:48.504282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.599051 (+ 94769us) service_pool.cc:224] Handling call
0709 13:14:48.599425 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.685219 (+ 85794us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:48.973111 (+287892us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.217400 (+244289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.217540 (+   140us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:49.217612 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.685460 (+467848us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b540027f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:49.685536 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.685697 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3790,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":48961,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":116681,"spinlock_wait_cycles":196662144,"thread_start_us":525408,"threads_started":2,"wal-append.queue_time_us":453131}
W20260709 13:14:49.686029 30509 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 13:14:49.682202 30500 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.515s	user 0.010s	sys 0.000s
I20260709 13:14:49.723523 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:49.729391 22914 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20. Re-sending same reply.
W20260709 13:14:49.677457 30487 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.751s	user 0.001s	sys 0.009s
I20260709 13:14:49.734297 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1332) took 1792 ms. Trace:
I20260709 13:14:49.734066 30530 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:49.704463 30461 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:49.734755 22832 rpcz_store.cc:276] 0709 13:14:47.941835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.277937 (+336102us) service_pool.cc:224] Handling call
0709 13:14:48.280509 (+  2572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.291026 (+ 10517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.291069 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.320287 (+ 29218us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.619685 (+299398us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:48.619731 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.619813 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:49.676587 (+1056774us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:49.733948 (+ 57361us) raft_consensus.cc:1685] finished
0709 13:14:49.733975 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.734226 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25206,"raft.queue_time_us":10330,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":36617,"spinlock_wait_cycles":11921152,"thread_start_us":235863,"threads_started":1,"wal-append.queue_time_us":466728}
I20260709 13:14:49.724957 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:49.738302 22909 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:49.725615 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:49.739037 30505 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.467s	user 0.000s	sys 0.001s
I20260709 13:14:49.739485 22910 raft_consensus.cc:2376] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21. Re-sending same reply.
W20260709 13:14:49.741734 30369 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.013s
I20260709 13:14:49.742336 30369 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.721068 30515 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:49.762240 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:49.765836 22908 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:49.771184 23458 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 16.
I20260709 13:14:49.772769 30371 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:49.743193 30523 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:14:49.726095 30148 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:49.638373 30510 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.784940 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:49.818434 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.835646 22906 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:49.768905 30515 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.767568 30305 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:49.845701 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:49.846118 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.846480 22915 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:49.848142 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.848495 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.836320 30212 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:49.848577 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.849339 23459 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:49.850096 30212 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.850863 30369 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:49.852392 30212 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:49.854274 30516 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:49.855854 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2391) took 1305 ms. Trace:
I20260709 13:14:49.849334 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.856244 22837 rpcz_store.cc:276] 0709 13:14:48.550956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.956078 (+405122us) service_pool.cc:224] Handling call
0709 13:14:48.956460 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.978771 (+ 22311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.978807 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.979360 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:14:49.442410 (+463050us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:49.442454 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.442518 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:49.855485 (+412967us) raft_consensus.cc:1685] finished
0709 13:14:49.855512 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.855774 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":407,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":54176,"thread_start_us":386576,"threads_started":1,"wal-append.queue_time_us":622044}
I20260709 13:14:49.860723 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.861583 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
W20260709 13:14:49.872080 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 147635us
W20260709 13:14:49.872150 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 206667us
I20260709 13:14:49.878036 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:49.879732 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.837723 30455 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:49.894203 22805 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.842118 30520 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:49.896795 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:49.905038 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:49.858379 30308 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.843914 30213 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.905875 22915 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:49.846006 30515 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:49.906986 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:49.842993 30531 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.858081 30310 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:49.865975 30276 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:49.933218 30542 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:49.935288 30371 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:14:49.940348 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1163) took 1968 ms. Trace:
I20260709 13:14:49.908109 30369 raft_consensus.cc:2764] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:14:49.954761 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:49.957259 22760 rpcz_store.cc:276] 0709 13:14:47.972354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.098718 (+126364us) service_pool.cc:224] Handling call
0709 13:14:48.099219 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.398373 (+299154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.939997 (+1541624us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b540002e4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:49.940268 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686038400}
W20260709 13:14:49.947264 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.966526 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.917186 30204 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:49.966282 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1173) took 1270 ms. Trace:
I20260709 13:14:49.972091 22760 rpcz_store.cc:276] 0709 13:14:48.695880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.965098 (+1269218us) service_pool.cc:224] Handling call
0709 13:14:49.965577 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.965839 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:49.965881 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.965945 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:49.965993 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.966059 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.966212 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.975791 30456 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:49.976320 30456 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.976686 30456 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:49.979343 29846 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:49.979961 29846 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:49.982048 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 92050.8us
I20260709 13:14:49.937914 30520 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.984794 22833 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:49.974334 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:49.982885 30308 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:50.012288 30513 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:49.982136 30457 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:50.013293 30457 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.013823 30457 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:50.015105 30334 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.921s	user 0.010s	sys 0.012s
I20260709 13:14:50.015316 30306 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.012840 30513 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:50.017786 30513 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:50.008219 30520 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:49.956630 30342 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:14:50.020016 30520 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:49.945463 30213 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:49.907991 30455 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.026141 30306 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.029052 30276 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.029713 30276 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.031975 30276 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:50.033010 30321 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.918s	user 0.010s	sys 0.009s
I20260709 13:14:50.033529 30321 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.034261 29846 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.035884 30321 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.036088 29892 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:50.036485 29892 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:50.037796 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.038977 22879 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.936s	user 0.027s	sys 0.025s
W20260709 13:14:50.039390 22879 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.937s	user 0.028s	sys 0.025s
W20260709 13:14:50.039649 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 2.01051e+06us
W20260709 13:14:49.997232 30327 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.011s
I20260709 13:14:50.040681 30327 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.043763 30327 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.043848 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.044677 22906 raft_consensus.cc:2376] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22. Re-sending same reply.
I20260709 13:14:50.044996 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:50.045473 30537 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:14:50.045701 22915 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:50.045755 22835 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:49.986672 30204 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.047825 30204 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:50.033950 30309 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.142s	user 0.015s	sys 0.001s
I20260709 13:14:50.050467 30309 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.052011 22765 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:50.052608 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3541) took 2064 ms. Trace:
I20260709 13:14:49.959941 30531 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:50.054831 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3554) took 1383 ms (client timeout 1775 ms). Trace:
W20260709 13:14:50.060693 22765 rpcz_store.cc:269] 0709 13:14:48.671742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.863877 (+192135us) service_pool.cc:224] Handling call
0709 13:14:50.054630 (+1190753us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400311354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:50.054772 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2714671360}
I20260709 13:14:50.056676 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:50.062227 23460 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:50.056918 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.062906 22912 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:50.067360 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:50.068261 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:50.069585 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:50.069814 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:50.070214 22837 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:50.070366 22838 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:50.053247 22766 rpcz_store.cc:276] 0709 13:14:47.987726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.345287 (+357561us) service_pool.cc:224] Handling call
0709 13:14:48.345895 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.346224 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:48.614236 (+268012us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.929741 (+315505us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.051697 (+1121956us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b18003570a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:14:50.051895 (+   198us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:50.051952 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:50.052286 (+   334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.052378 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.052532 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48962,"prepare.queue_time_us":200732,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":2144,"raft.queue_time_us":14597,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":15957,"spinlock_wait_cycles":59090816,"thread_start_us":552431,"threads_started":2,"wal-append.queue_time_us":492777}
I20260709 13:14:50.074679 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.068904 22840 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:50.077764 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:50.078444 22832 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:50.078895 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:50.080127 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 186053us
I20260709 13:14:50.082818 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.083796 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.085398 22912 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:50.015827 30334 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.085465 23462 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:50.108052 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.109633 22912 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:14:50.110860 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.110901 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.113780 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.037335 29892 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:50.137166 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 53429.8us
I20260709 13:14:50.137719 29892 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:50.138098 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:50.139285 30432 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.085s	user 0.027s	sys 0.006s
I20260709 13:14:50.144253 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.145015 22910 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:50.146006 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.147449 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.139954 30432 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.152351 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.160341 30455 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.176055 22910 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:50.160974 22832 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:50.190872 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.191684 22912 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:14:50.164064 30531 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:50.166851 30485 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.323s	user 0.003s	sys 0.008s
I20260709 13:14:50.167567 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.169023 30334 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.151026 30309 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.089022 30212 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 13:14:50.222390 30212 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:50.222913 30212 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:50.223891 30212 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.176364 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.226502 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:50.227154 30311 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.973s	user 0.022s	sys 0.004s
W20260709 13:14:50.227044 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 85568.2us
I20260709 13:14:50.227768 30311 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.228434 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.164178 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1344) took 1083 ms. Trace:
I20260709 13:14:50.229564 22835 rpcz_store.cc:276] 0709 13:14:49.080474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.610084 (+529610us) service_pool.cc:224] Handling call
0709 13:14:49.638633 (+ 28549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.046569 (+407936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.163819 (+117250us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54000c71d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:50.164093 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935416704}
I20260709 13:14:50.231149 30520 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:50.231652 30520 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.232009 30520 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:50.233750 30446 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:50.235634 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.238615 30311 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.243230 22906 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:50.203513 22911 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:50.208300 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.244350 22910 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:50.244791 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.209204 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.246040 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:50.192338 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 133960us
I20260709 13:14:50.246840 22911 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:50.247085 30456 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.248725 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.241570 22912 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:50.241780 30446 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:50.250689 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.252656 30542 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:50.195339 30306 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:50.253193 30542 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:50.253791 30461 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.254132 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.253929 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:50.254258 30542 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.254995 22913 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:50.255273 22832 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:50.255956 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.256227 30461 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.256486 30311 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:50.245239 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.134426 30517 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.225323 30212 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:50.256605 22913 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:50.258162 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.258982 30517 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.259301 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.259961 22913 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:50.260335 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.260874 30204 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.261687 30311 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.262235 30311 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:50.263288 30204 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.264243 22763 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:50.265435 30432 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.265838 22907 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:50.242372 30551 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:50.269639 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:50.267787 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.275926 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.249473 22911 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 14.
I20260709 13:14:50.260900 22912 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:50.268694 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.236986 30515 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.271623 22835 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 27. Current term is 28. Ops: []
I20260709 13:14:50.273903 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.249287 30456 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.285408 22913 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:50.289695 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:50.291076 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.295074 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:50.300253 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.301003 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.301486 22913 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 14.
I20260709 13:14:50.276131 30446 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:50.274709 30310 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.309652 22912 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:50.305938 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.311389 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:50.311457 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.312820 22912 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:50.306257 30544 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:50.313838 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:50.314090 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.316402 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:50.310953 22913 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:50.306634 30148 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:50.299244 29892 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:50.322491 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:50.325518 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.326388 22911 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:50.310602 30305 raft_consensus.cc:2764] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:14:50.330276 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:50.306896 30551 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.318146 30544 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.343796 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.344843 22909 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 14.
I20260709 13:14:50.333461 23464 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:50.334734 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:50.308377 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.348265 22805 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:50.349725 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.338456 22912 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:50.351577 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.352068 22912 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:50.328939 30148 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.355970 22879 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.360394 23458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 1081) took 1164 ms. Trace:
I20260709 13:14:50.364214 23458 rpcz_store.cc:276] 0709 13:14:49.195624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.228643 (+ 33019us) service_pool.cc:224] Handling call
0709 13:14:50.360121 (+1131478us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54002c6d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:50.360274 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219305728}
I20260709 13:14:50.319309 30310 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.336651 30517 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:50.372762 22913 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:50.375928 22913 raft_consensus.cc:1217] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 12.10->[13.11-18.14]   Dedup: 13.11->[14.12-18.14]
I20260709 13:14:50.376523 22913 pending_rounds.cc:85] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 11
I20260709 13:14:50.376506 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:50.377333 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.377679 22913 pending_rounds.cc:107] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:14:50.355225 30541 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:14:50.362039 30458 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:50.382359 30515 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.370455 30554 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:50.368134 30148 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:50.419320 30290 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.348229 30551 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.424114 30311 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.424810 30551 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.426350 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1334) took 2468 ms. Trace:
I20260709 13:14:50.426733 22836 raft_consensus.cc:1217] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Deduplicated request from leader. Original: 19.10->[21.11-24.13]   Dedup: 24.13->[]
I20260709 13:14:50.427426 22836 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [21.11-24.13]
I20260709 13:14:50.428337 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1340) took 1723 ms. Trace:
I20260709 13:14:50.429054 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2389) took 1924 ms. Trace:
I20260709 13:14:50.429476 22836 rpcz_store.cc:276] 0709 13:14:48.504983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.603435 (+ 98452us) service_pool.cc:224] Handling call
0709 13:14:50.426597 (+1823162us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:14:50.427778 (+  1181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.428815 (+  1037us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54001bb550. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:50.428982 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4157151616}
W20260709 13:14:50.434387 30511 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.877s	user 0.003s	sys 0.000s
I20260709 13:14:50.378129 22915 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:50.348625 30544 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:50.392601 30457 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.397842 30542 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.361642 30446 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:50.426791 22841 rpcz_store.cc:276] 0709 13:14:47.958409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.576824 (+618415us) service_pool.cc:224] Handling call
0709 13:14:48.577292 (+   468us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:48.614760 (+ 37468us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:48.641113 (+ 26353us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.922067 (+280954us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:48.923060 (+   993us) log.cc:844] Serialized 78 byte log entry
0709 13:14:48.923228 (+   168us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:48.923507 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.938024 (+ 14517us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.947129 (+  9105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.259978 (+312849us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b18003e6ce0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:49.738266 (+478288us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:49.947994 (+209728us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b4800130158. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
0709 13:14:50.164899 (+216905us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b18003e6ce0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:14:50.425977 (+261078us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b54001bb554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:50.426036 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:50.426083 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:50.426097 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.426283 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82557,"prepare.queue_time_us":379182,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":144,"raft.queue_time_us":13066,"raft.run_cpu_time_us":14155,"raft.run_wall_time_us":1065354,"spinlock_wait_cycles":136017920,"thread_start_us":170178,"threads_started":1,"wal-append.queue_time_us":496466}
I20260709 13:14:50.402318 22911 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:50.396888 30520 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.403100 30384 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:50.408138 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.446048 22908 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:50.447943 30290 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:50.412293 22913 raft_consensus.cc:2889] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.413897 29892 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:50.428694 22839 rpcz_store.cc:276] 0709 13:14:48.704524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.441930 (+737406us) service_pool.cc:224] Handling call
0709 13:14:49.443304 (+  1374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.448817 (+  5513us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:49.482224 (+ 33407us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.705312 (+223088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.705378 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:49.705427 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.428094 (+722667us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:50.428148 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.428273 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16475,"prepare.queue_time_us":39783,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":77233,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":19500,"spinlock_wait_cycles":12014336,"thread_start_us":222587,"threads_started":2,"wal-append.queue_time_us":167900}
I20260709 13:14:50.432288 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:50.436820 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.489914 23457 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: [24.19-24.19]
W20260709 13:14:50.490172 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53458.3us
I20260709 13:14:50.490064 22906 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
W20260709 13:14:50.462951 30490 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.350s	user 0.007s	sys 0.004s
W20260709 13:14:50.468427 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.491178 22807 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.470899 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3543) took 2482 ms. Trace:
I20260709 13:14:50.494951 22910 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:14:50.472004 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.495223 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.472697 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:50.476995 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.477942 22913 pending_rounds.cc:107] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:14:50.445174 23464 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:50.387737 30513 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.468257 30447 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.497401 22913 raft_consensus.cc:2889] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:50.505769 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.505477 22915 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
W20260709 13:14:50.512606 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89171.1us
I20260709 13:14:50.519236 22807 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.494730 22759 rpcz_store.cc:276] 0709 13:14:47.988440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.532032 (+543592us) service_pool.cc:224] Handling call
0709 13:14:48.532488 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.532756 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:48.810785 (+278029us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.470481 (+1659696us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:50.470548 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:50.470602 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.470676 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.470829 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1505709,"prepare.queue_time_us":87598,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"thread_start_us":427959,"threads_started":2,"wal-append.queue_time_us":621829}
I20260709 13:14:50.521766 30517 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:50.522522 30517 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:50.523761 30517 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.513360 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:50.545838 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:14:50.548092 22805 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:50.499902 30551 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:50.615453 30362 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:50.540596 30304 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:50.493408 30449 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:50.508314 30515 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:50.630802 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 82836.1us
I20260709 13:14:50.446470 30426 consensus_queue.cc:1059] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:14:50.508098 30541 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:50.530716 30461 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.616269 30536 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.606784 30278 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:50.650477 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 1.32062e+06us
I20260709 13:14:50.446820 30542 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.649439 22908 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 26.
I20260709 13:14:50.660696 22807 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:50.669921 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.537668 30447 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.639914 30276 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:50.633134 30551 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:50.647778 30515 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.698899 30458 raft_consensus.cc:3055] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:14:50.712710 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.667806 30554 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:50.655735 30362 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:50.717991 30458 raft_consensus.cc:740] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:50.654170 30449 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.733946 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:50.679235 30541 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:50.697654 30551 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:50.664336 30461 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.687012 30426 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:50.699345 30013 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:50.744602 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.715772 30515 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:50.738219 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:50.759610 22765 raft_consensus.cc:1240] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:50.763759 22760 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:50.774669 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.758692 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.719352 30569 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.676560 30536 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.730643 30309 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.675103 30278 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.691619 29892 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:14:50.722204 30276 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:50.774745 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.739660 30234 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.778635 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.781308 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:50.770270 30212 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.006s
I20260709 13:14:50.741130 30554 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:50.739962 29933 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:50.787513 22760 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:50.783509 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.769397 30311 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.793653 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.799142 22765 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:50.800076 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.801117 22765 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:50.766635 30385 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.770151 30426 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:50.797875 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.777182 30555 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.803965 22760 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:50.806833 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.806684 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.810952 22759 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:50.808219 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:50.811504 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:50.812211 22760 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:50.805881 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.814930 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.829989 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.797189 30457 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.798687 30557 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:50.776089 30013 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.803146 30520 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:50.807770 30426 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:50.805272 30290 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:50.855263 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.792557 30309 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.796427 30234 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.822871 30572 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:50.853989 30446 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.816923 30385 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.798111 30296 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:14:50.785342 30569 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.802630 29933 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.855940 30344 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:50.888906 22880 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:50.856812 30513 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.782434 30575 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
W20260709 13:14:50.958791 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 143955us
W20260709 13:14:50.906970 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.879786 30565 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:14:50.779026 30567 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:14:50.878441 30561 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:50.871312 30290 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:50.814401 30458 consensus_queue.cc:260] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.873200 30557 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:50.935640 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.822542 30555 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:50.890583 30530 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.906371 30520 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:50.805900 30362 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:50.943974 30563 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 13:14:50.878407 30568 consensus_queue.cc:1059] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.152s
W20260709 13:14:50.971753 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 82946.5us
W20260709 13:14:50.971992 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 472713us
I20260709 13:14:50.929335 30573 raft_consensus.cc:2764] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:14:50.942629 30344 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.965240 30461 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:50.970685 30572 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:50.988879 30574 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:50.905578 30446 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.968647 30447 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.866671 30541 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:50.994283 30344 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:50.999842 30570 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:50.909677 30276 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:51.041239 30571 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:14:50.911933 30554 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 23.9, Last appended by leader: 9, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.043928 30569 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.044703 30234 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.045569 30309 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.984215 30520 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.048036 30520 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:51.048666 30385 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.007793 30574 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:50.984639 30449 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:50.981371 30581 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:51.049849 30574 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:51.049628 30290 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.050150 30581 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:51.050664 30581 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:51.050848 30290 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:50.990216 30296 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:50.966933 30561 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:50.998108 30458 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:51.013128 30572 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:51.033428 30536 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.059303 30542 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:51.055603 30583 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
I20260709 13:14:51.051956 30557 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.051895 30570 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.056926 30561 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:51.064396 30579 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:14:51.071981 30570 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:51.077557 30572 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.084126 30561 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.087378 30561 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.089134 30574 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.091204 30581 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.092810 30574 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:50.940637 22760 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:50.987483 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:51.038002 30447 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:51.132035 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:51.137466 22757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.006s	sys 0.006s
W20260709 13:14:51.397809 22757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.018s	sys 0.040s
I20260709 13:14:51.399158 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3542) took 3411 ms. Trace:
I20260709 13:14:51.247211 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3807) took 1185 ms. Trace:
I20260709 13:14:51.400477 30446 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:51.400732 22913 rpcz_store.cc:276] 0709 13:14:50.062101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.375244 (+313143us) service_pool.cc:224] Handling call
0709 13:14:50.375684 (+   440us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:50.642071 (+266387us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:50.642126 (+    55us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:50.643105 (+   979us) log.cc:844] Serialized 78 byte log entry
0709 13:14:50.809752 (+166647us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:50.809877 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.810041 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.246243 (+436202us) raft_consensus.cc:1685] finished
0709 13:14:51.246372 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.246978 (+   606us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":191830,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":144,"thread_start_us":300394,"threads_started":2,"wal-append.queue_time_us":341440}
I20260709 13:14:51.402168 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:51.403437 30310 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.403723 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:51.405011 22913 raft_consensus.cc:2410] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:51.148149 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1170) took 2773 ms. Trace:
I20260709 13:14:51.405706 22764 rpcz_store.cc:276] 0709 13:14:48.374927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.567363 (+1192436us) service_pool.cc:224] Handling call
0709 13:14:49.567889 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.607283 (+1039394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.607316 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.620012 (+ 12696us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.918079 (+298067us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:50.921024 (+  2945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.921197 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.147317 (+226120us) raft_consensus.cc:1685] finished
0709 13:14:51.147399 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.147871 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1099326,"raft.queue_time_us":1052494,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":9256,"thread_start_us":297625,"threads_started":1,"wal-append.queue_time_us":378591}
I20260709 13:14:51.408402 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.409919 30243 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:51.410808 30243 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.399621 22757 rpcz_store.cc:276] 0709 13:14:47.988102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.501742 (+513640us) service_pool.cc:224] Handling call
0709 13:14:48.508331 (+  6589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.508622 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:48.833913 (+325291us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.433489 (+1599576us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:51.398416 (+964927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:51.398546 (+   130us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:51.398701 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.398848 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.399084 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1433732,"prepare.queue_time_us":88480,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":107,"raft.queue_time_us":1474233,"raft.run_cpu_time_us":3024,"raft.run_wall_time_us":872657,"spinlock_wait_cycles":169897088,"thread_start_us":1434399,"threads_started":3,"wal-append.queue_time_us":658720}
I20260709 13:14:51.414601 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:51.415380 30236 raft_consensus.cc:3055] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:51.415618 22757 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:51.199649 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.147964 22765 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:51.275455 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:51.417785 23464 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:51.283901 30013 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.199606 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.316680 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:51.398037 30447 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.034204 30278 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.421554 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.219209 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.277243 23458 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:51.337891 30517 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.338047 30530 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:51.345114 22908 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:14:51.094570 30581 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:51.315447 22834 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:51.369304 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3546) took 3040 ms. Trace:
I20260709 13:14:51.053743 30296 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:51.191315 22730 connection.cc:256] RPC call timeout handler was delayed by -1.0201028250000945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:14:51.203810 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.378515 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:51.384785 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1331) took 1089 ms. Trace:
W20260709 13:14:51.389245 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:51.389889 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:51.390808 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1327) took 1542 ms (client timeout 1983 ms). Trace:
I20260709 13:14:51.393882 30578 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:51.078390 30555 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.396727 23460 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1334) took 1099 ms. Trace:
I20260709 13:14:51.415923 30236 raft_consensus.cc:740] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:51.418201 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:51.236536 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3806) took 1176 ms. Trace:
I20260709 13:14:51.409201 22764 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:51.181764 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.263537 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2457) took 1186 ms. Trace:
I20260709 13:14:51.214406 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.213606 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.287261 30276 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:51.229622 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3812) took 1162 ms. Trace:
I20260709 13:14:51.437469 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:51.437738 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:51.429411 22908 pending_rounds.cc:85] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 10
I20260709 13:14:51.280241 29933 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.417305 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:51.440726 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.161160 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.434542 23457 rpcz_store.cc:276] 0709 13:14:50.295032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.295255 (+   223us) service_pool.cc:224] Handling call
0709 13:14:50.295677 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.490473 (+194796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.383873 (+893400us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540063f954. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:51.384472 (+   599us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446021504}
I20260709 13:14:51.197283 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:51.438383 22914 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:51.182241 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:51.213486 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.409196 30239 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:51.434801 23462 rpcz_store.cc:269] 0709 13:14:49.847727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.847867 (+   140us) service_pool.cc:224] Handling call
0709 13:14:51.390191 (+1542324us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400740c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:51.390506 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544106752}
I20260709 13:14:51.439177 23460 rpcz_store.cc:276] 0709 13:14:50.302086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.309988 (+  7902us) service_pool.cc:224] Handling call
0709 13:14:50.310418 (+   430us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:50.319804 (+  9386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.319843 (+    39us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:14:50.320705 (+   862us) log.cc:844] Serialized 104 byte log entry
0709 13:14:50.447119 (+126414us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:50.447161 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.447235 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.300221 (+852986us) raft_consensus.cc:1685] finished
0709 13:14:51.300339 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.396489 (+ 96150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4406,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":478,"spinlock_wait_cycles":753920,"thread_start_us":117927,"threads_started":1,"wal-append.queue_time_us":291542}
I20260709 13:14:51.287729 30297 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:51.441144 22909 rpcz_store.cc:276] 0709 13:14:50.060093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.358879 (+298786us) service_pool.cc:224] Handling call
0709 13:14:50.359351 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.367281 (+  7930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.367315 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.373815 (+  6500us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.510365 (+136550us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:14:50.510432 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.510508 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.235776 (+725268us) raft_consensus.cc:1685] finished
0709 13:14:51.235835 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.236270 (+   435us) inbound_call.cc:177] Queueing success response
0709 13:14:51.399294 (+163024us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b48000b0c58. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a6a1aaa1 00007f68a6a11b37 00007f68a6a119ca 00007f68a6b0926f 00007f68a6b08f37 00007f68a6b1d55c 00007f68a6b1d508 00007f68a6b1d488 00007f68a6b1d443 00007f68a6b1c6ec 00007f68a5a46a96
Metrics: {"mutex_wait_us":7396,"raft.queue_time_us":7505,"raft.run_cpu_time_us":4238,"raft.run_wall_time_us":1029314,"spinlock_wait_cycles":9237760,"thread_start_us":136110,"threads_started":1,"wal-append.queue_time_us":280805}
I20260709 13:14:51.454115 30517 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.436103 30578 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:51.466231 22913 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:51.466411 30297 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.461812 22908 pending_rounds.cc:107] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:14:51.471203 22908 raft_consensus.cc:2889] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:51.472975 30297 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.474049 22841 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:51.441689 22915 rpcz_store.cc:276] 0709 13:14:50.076578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.518818 (+442240us) service_pool.cc:224] Handling call
0709 13:14:50.519427 (+   609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.712799 (+193372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.712842 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.713403 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.814204 (+100801us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:50.814266 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.814382 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.262703 (+448321us) raft_consensus.cc:1685] finished
0709 13:14:51.262762 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.263203 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178682,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":288542,"threads_started":2,"wal-append.queue_time_us":259065}
I20260709 13:14:51.478456 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:51.478924 22915 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:51.479700 22909 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 24 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:14:51.479838 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2461) took 1342 ms. Trace:
I20260709 13:14:51.488049 30530 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.489744 22840 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:51.480270 22915 rpcz_store.cc:276] 0709 13:14:50.137232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.477887 (+1340655us) service_pool.cc:224] Handling call
0709 13:14:51.478496 (+   609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.479472 (+   976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.479740 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8576}
I20260709 13:14:51.491322 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.469033 22760 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:51.496083 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:51.501607 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:51.492086 30530 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.506333 22837 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:51.442674 22906 rpcz_store.cc:276] 0709 13:14:50.066908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.500950 (+434042us) service_pool.cc:224] Handling call
0709 13:14:50.501286 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.733374 (+232088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.733408 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.734005 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.984105 (+250100us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:50.984146 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.984235 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.228797 (+244562us) raft_consensus.cc:1685] finished
0709 13:14:51.228854 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.229311 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187723,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":153,"thread_start_us":481379,"threads_started":2,"wal-append.queue_time_us":303451}
I20260709 13:14:51.454819 30236 consensus_queue.cc:260] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.431543 22762 rpcz_store.cc:276] 0709 13:14:48.328529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.238074 (+909545us) service_pool.cc:224] Handling call
0709 13:14:49.238687 (+   613us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:50.493605 (+1254918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.493641 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:50.494293 (+   652us) log.cc:844] Serialized 52 byte log entry
0709 13:14:51.356437 (+862144us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b18003217c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1
0709 13:14:51.357053 (+   616us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:51.357140 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.357273 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.357351 (+    78us) raft_consensus.cc:1685] finished
0709 13:14:51.357366 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.369014 (+ 11648us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1293096,"raft.queue_time_us":1271219,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":3956,"spinlock_wait_cycles":51586944,"thread_start_us":112601,"threads_started":1,"wal-append.queue_time_us":407728}
I20260709 13:14:51.518098 22838 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:51.525429 22833 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:51.499006 30296 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:51.547084 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 559514us
W20260709 13:14:51.550307 30526 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.908s	user 0.003s	sys 0.000s
W20260709 13:14:51.554987 30534 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.782s	user 0.000s	sys 0.004s
I20260709 13:14:51.562876 23464 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:51.562851 22764 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:51.564445 29933 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:51.569964 29933 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.572921 29933 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:51.567932 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62988.6us
I20260709 13:14:51.576869 22841 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:51.582468 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.517714 30276 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.578792 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:51.583652 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:51.588981 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3556) took 2916 ms. Trace:
I20260709 13:14:51.590067 22880 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:51.591276 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58976.1us
W20260709 13:14:51.592909 30544 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.145s	user 0.010s	sys 0.009s
I20260709 13:14:51.593876 30591 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:51.594045 30567 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:51.595268 30567 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:51.595924 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2453) took 1538 ms. Trace:
I20260709 13:14:51.596393 22907 rpcz_store.cc:276] 0709 13:14:50.057664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.057840 (+   176us) service_pool.cc:224] Handling call
0709 13:14:50.265626 (+207786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.266289 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.595298 (+1329009us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54002116d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:51.595611 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476800768}
I20260709 13:14:51.600404 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:51.605772 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:51.602921 22763 rpcz_store.cc:276] 0709 13:14:48.672387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.071414 (+399027us) service_pool.cc:224] Handling call
0709 13:14:51.588471 (+2517057us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400311d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:51.588741 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2740651904}
I20260709 13:14:51.564976 30447 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.559758 30578 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:51.611176 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.615345 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:51.608889 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:51.612229 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:51.594820 30591 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:51.622821 30148 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.205s	user 0.011s	sys 0.014s
I20260709 13:14:51.623489 30243 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.623453 30148 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.624328 22763 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:51.624979 30293 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:51.627036 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3827) took 1190 ms. Trace:
I20260709 13:14:51.649998 22907 rpcz_store.cc:276] 0709 13:14:50.436443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.625090 (+1188647us) service_pool.cc:224] Handling call
0709 13:14:51.625778 (+   688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.626687 (+   909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:51.626716 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:51.626765 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:51.626808 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.626867 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.626974 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:51.651141 30293 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:51.652607 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1142) took 1176 ms. Trace:
I20260709 13:14:51.631825 30518 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:51.653115 30554 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.618202 30296 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:51.630604 30407 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.738s	user 0.015s	sys 0.009s
I20260709 13:14:51.656606 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:51.659739 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45600.9us
I20260709 13:14:51.653053 22907 rpcz_store.cc:276] 0709 13:14:50.476548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.651258 (+1174710us) service_pool.cc:224] Handling call
0709 13:14:51.651837 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.652191 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:51.652218 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:51.652283 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:51.652336 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.652411 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.652546 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:51.664606 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:51.602751 30544 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.668519 22757 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:51.664886 22764 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:51.670547 22911 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:51.658556 30517 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:51.662132 30578 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:51.664518 30407 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.889s	user 0.017s	sys 0.009s
I20260709 13:14:51.684937 22840 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:51.684741 30239 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:51.684962 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:51.687319 30564 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.952s	user 0.003s	sys 0.000s
I20260709 13:14:51.693394 22914 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:51.698645 22909 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:51.709848 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1141) took 1222 ms. Trace:
I20260709 13:14:51.710445 22906 rpcz_store.cc:276] 0709 13:14:50.476229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.656085 (+1179856us) service_pool.cc:224] Handling call
0709 13:14:51.676597 (+ 20512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.697964 (+ 21367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:51.697999 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:51.698039 (+    40us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:14:51.698060 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.698111 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.698653 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:51.700763 23458 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:51.713196 22766 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:51.689237 30239 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:51.753455 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.936672 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:51.777136 30239 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:51.962707 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:51.712671 30554 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:51.993248 23464 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
W20260709 13:14:51.996905 30570 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.886s	user 0.017s	sys 0.003s
I20260709 13:14:52.001538 30570 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:51.775275 30567 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.005128 30570 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:52.009653 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 242585us
I20260709 13:14:52.014060 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:52.028206 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 418205us
W20260709 13:14:52.037966 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 281134us
W20260709 13:14:52.038663 22732 connection.cc:256] RPC call timeout handler was delayed by -1.2452065110001058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 13:14:52.039247 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.039611 22732 connection.cc:256] RPC call timeout handler was delayed by -1.1872663490000832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.165s for a response
W20260709 13:14:52.040087 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.040566 22732 connection.cc:256] RPC call timeout handler was delayed by -1.0679065110000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
I20260709 13:14:52.042060 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.043687 30517 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:52.044606 30455 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:52.045358 30455 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:52.046083 30455 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:52.047645 30581 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:52.048843 22766 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:52.048750 30581 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.049878 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3025) took 1278 ms. Trace:
I20260709 13:14:52.050452 22766 rpcz_store.cc:276] 0709 13:14:50.771562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.923112 (+1151550us) service_pool.cc:224] Handling call
0709 13:14:51.987231 (+ 64119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.049395 (+ 62164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.049746 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.051551 30581 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:52.052407 22766 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:52.053943 30290 consensus_meta.cc:287] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.982s	user 0.018s	sys 0.003s
I20260709 13:14:52.044872 30517 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:52.061064 30515 consensus_meta.cc:287] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.277s	user 0.013s	sys 0.010s
W20260709 13:14:52.066648 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.053057 23464 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:14:52.071103 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45164.4us
I20260709 13:14:52.070233 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1144) took 1555 ms. Trace:
I20260709 13:14:52.078226 22914 rpcz_store.cc:276] 0709 13:14:50.477052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.711744 (+1234692us) service_pool.cc:224] Handling call
0709 13:14:51.951865 (+240121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.027250 (+ 75385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.027297 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.027406 (+   109us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:52.032378 (+  4972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.032453 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.032588 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:51.975734 30591 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:52.086858 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.085224 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3027) took 1311 ms. Trace:
I20260709 13:14:52.088902 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:52.099790 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.099540 22766 rpcz_store.cc:276] 0709 13:14:50.786137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.051630 (+1265493us) service_pool.cc:224] Handling call
0709 13:14:52.052134 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.068959 (+ 16825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.085119 (+ 16160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.087672 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:52.095341 30213 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.087369 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3804) took 2028 ms. Trace:
I20260709 13:14:52.066987 30148 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.113281 22835 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 28: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:52.087060 30544 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.118259 30234 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.122138 30234 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.124619 30234 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.125538 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.127053 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.101727 30276 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.127987 22838 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:52.092152 30515 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.131451 22836 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:52.135329 30515 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.149677 30570 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.151315 30276 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:52.152665 22914 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:52.152467 30276 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.153803 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3828) took 1619 ms. Trace:
I20260709 13:14:52.154356 22914 rpcz_store.cc:276] 0709 13:14:50.534615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.103830 (+1569215us) service_pool.cc:224] Handling call
0709 13:14:52.152330 (+ 48500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.153307 (+   977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.153681 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:52.159633 30426 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.280s	user 0.008s	sys 0.011s
I20260709 13:14:52.160240 30426 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.161005 30234 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:52.108178 22912 rpcz_store.cc:276] 0709 13:14:50.059181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.352821 (+293640us) service_pool.cc:224] Handling call
0709 13:14:50.353081 (+   260us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:50.364464 (+ 11383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.364509 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:50.370054 (+  5545us) log.cc:844] Serialized 52 byte log entry
0709 13:14:50.499431 (+129377us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:50.504140 (+  4709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.504226 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.686287 (+1182061us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:52.067580 (+381293us) raft_consensus.cc:1685] finished
0709 13:14:52.067610 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.087115 (+ 19505us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10321,"raft.queue_time_us":12905,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":4932,"spinlock_wait_cycles":5234560,"thread_start_us":120389,"threads_started":1,"wal-append.queue_time_us":255426}
I20260709 13:14:52.161836 30234 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:52.163908 30344 consensus_meta.cc:287] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.124s	user 0.016s	sys 0.002s
I20260709 13:14:52.164440 30344 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.164790 30234 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:52.180366 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 99053.2us
I20260709 13:14:52.176036 22914 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:52.149318 30517 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:52.197399 30520 consensus_meta.cc:287] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.134s	user 0.012s	sys 0.007s
I20260709 13:14:52.198086 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3829) took 1649 ms. Trace:
I20260709 13:14:52.206449 22914 rpcz_store.cc:276] 0709 13:14:50.537000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.155719 (+1618719us) service_pool.cc:224] Handling call
0709 13:14:52.156345 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.185024 (+ 28679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.186720 (+  1696us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.151062 30570 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.200784 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.214064 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:52.199208 30293 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.003s
I20260709 13:14:52.219548 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:52.236428 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.236884 22836 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:52.199213 30426 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:52.239835 22730 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:52.240597 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 196967us
I20260709 13:14:52.246665 22766 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.244551 23458 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:52.254254 22807 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:52.255689 22910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2474) took 2114 ms (client timeout 1889 ms). Trace:
W20260709 13:14:52.256402 22910 rpcz_store.cc:269] 0709 13:14:50.140860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.244959 (+104099us) service_pool.cc:224] Handling call
0709 13:14:52.255016 (+2010057us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54000cf654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:52.255380 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546101120}
I20260709 13:14:52.270182 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2420) took 1056 ms. Trace:
I20260709 13:14:52.270848 22840 rpcz_store.cc:276] 0709 13:14:51.213490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.490845 (+277355us) service_pool.cc:224] Handling call
0709 13:14:52.269817 (+778972us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540028aad4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:52.270044 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400956672}
W20260709 13:14:52.272877 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.273770 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.275292 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:52.276770 30290 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:52.277594 30290 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.280431 30290 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.281803 30148 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.282646 30148 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:52.292151 30546 log.cc:927] Time spent T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.321s	user 0.008s	sys 0.005s
I20260709 13:14:52.295102 30324 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:14:52.311251 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 120745us
I20260709 13:14:52.297071 30324 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:52.281087 30290 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:52.334028 22764 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.005s
I20260709 13:14:52.336052 22764 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:52.341812 30512 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.342374 30554 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.342857 30512 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:52.345603 30551 consensus_meta.cc:287] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 1.417s	user 0.003s	sys 0.024s
I20260709 13:14:52.346338 30551 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:52.349722 30458 consensus_meta.cc:287] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.122s	user 0.009s	sys 0.009s
I20260709 13:14:52.350793 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.351423 30344 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.351716 22766 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.352200 30567 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.353060 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.319415 30555 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.354581 22832 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:52.355014 30234 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:52.356396 22836 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:52.357009 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 129300us
I20260709 13:14:52.357190 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.234920 30293 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.349819 22838 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:52.364976 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.359649 30290 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.372391 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.220709 30520 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.365962 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.285406 30148 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:52.369340 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.363044 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:52.376922 30555 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.377825 30290 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:52.361397 30239 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.378258 22835 raft_consensus.cc:2368] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:52.378675 22766 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 23: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.378857 22764 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.382552 22914 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:52.349289 30551 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.364324 30324 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:52.386828 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3832) took 1606 ms. Trace:
I20260709 13:14:52.388052 22914 rpcz_store.cc:276] 0709 13:14:50.776850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.232675 (+1455825us) service_pool.cc:224] Handling call
0709 13:14:52.329816 (+ 97141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.383021 (+ 53205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.383268 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.363684 30554 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:14:52.391420 22757 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.011s
I20260709 13:14:52.394179 22912 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:52.353221 30567 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.396922 22757 raft_consensus.cc:2468] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:52.377830 30234 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:52.399057 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.403612 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.420475 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.430490 30324 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.435318 30239 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:52.436506 30570 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.439343 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.440675 22763 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.441187 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:52.442472 30601 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:14:52.443238 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3822) took 2184 ms. Trace:
W20260709 13:14:52.443470 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 123471us
I20260709 13:14:52.443689 22915 rpcz_store.cc:276] 0709 13:14:50.258739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.442089 (+2183350us) service_pool.cc:224] Handling call
0709 13:14:52.442516 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.442840 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.442879 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.442940 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:52.442981 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.443046 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.443168 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.448000 30455 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:52.448773 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 143087us
I20260709 13:14:52.449414 22841 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:52.450768 22805 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:52.452353 30432 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:52.452680 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.453356 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:52.454011 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.468523 30344 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:52.471823 30344 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:52.474797 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.474886 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2471) took 2334 ms. Trace:
I20260709 13:14:52.475754 22759 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:52.475651 22909 rpcz_store.cc:276] 0709 13:14:50.139993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.480522 (+1340529us) service_pool.cc:224] Handling call
0709 13:14:51.698419 (+217897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.699080 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.474597 (+775517us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540010e0d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:52.474809 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499863936}
I20260709 13:14:52.477061 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.477761 22909 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:14:52.431134 30293 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.493026 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3838) took 1074 ms. Trace:
I20260709 13:14:52.493340 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.493937 22909 rpcz_store.cc:276] 0709 13:14:51.408380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.476867 (+1068487us) service_pool.cc:224] Handling call
0709 13:14:52.477221 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.482813 (+  5592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.483121 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.498713 22764 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:52.498047 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 243926us
W20260709 13:14:52.505661 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.506520 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.506914 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.508283 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57597.8us
I20260709 13:14:52.511385 30310 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:52.424271 30512 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.395470 30554 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:52.521808 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.523196 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:52.533463 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.531613 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.518728 30293 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:52.523226 30551 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:14:52.517060 30591 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.550472 22763 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.554584 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.558418 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.545148 30515 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:52.560324 30515 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:52.560853 30515 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:52.562810 30236 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:52.542126 30293 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.563735 30293 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.565176 30608 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:52.565976 30608 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:52.573909 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.497817 30508 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:14:52.530169 30276 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.560415 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:52.579149 22757 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:52.581108 30551 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.582970 30544 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.586771 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3830) took 1965 ms. Trace:
W20260709 13:14:52.578678 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.548s (SENT)
I20260709 13:14:52.595417 22912 rpcz_store.cc:276] 0709 13:14:50.621772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.162555 (+1540783us) service_pool.cc:224] Handling call
0709 13:14:52.162954 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.398287 (+235333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.585415 (+187128us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540010e5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:52.586685 (+  1270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531295488}
I20260709 13:14:52.512228 30310 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.596741 22759 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:52.603282 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.604141 22763 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 15.
I20260709 13:14:52.605211 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.605988 22763 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:52.602524 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:52.633538 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3021) took 1844 ms. Trace:
I20260709 13:14:52.634032 22762 rpcz_store.cc:276] 0709 13:14:50.760778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.658416 (+897638us) service_pool.cc:224] Handling call
0709 13:14:51.668040 (+  9624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.886624 (+218584us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:52.241428 (+354804us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.605160 (+363732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.605219 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:52.605282 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.605350 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.605463 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":644342,"prepare.run_cpu_time_us":242,"prepare.run_wall_time_us":2246,"raft.queue_time_us":18340,"thread_start_us":496673,"threads_started":2}
I20260709 13:14:52.657167 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.585922 30608 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:52.657963 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:52.658615 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56444.9us
I20260709 13:14:52.659888 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.657976 22764 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:52.596593 30276 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.662273 23466 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:14:52.664083 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.666615 30555 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.667358 30520 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.668289 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:52.669327 22909 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:14:52.670039 30304 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.670689 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:52.670804 30304 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.671160 30234 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.673475 22909 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:14:52.593693 30213 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.069s
I20260709 13:14:52.678265 30213 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:14:52.678673 30213 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:52.679803 30213 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:52.657405 22765 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 3022) took 1889 ms (client timeout 2204 ms). Trace:
W20260709 13:14:52.681754 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48313.8us
W20260709 13:14:52.682983 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.686942 22763 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:14:52.682723 22765 rpcz_store.cc:269] 0709 13:14:50.761005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.807671 (+ 46666us) service_pool.cc:224] Handling call
0709 13:14:52.592438 (+1784767us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002233d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:52.649899 (+ 57461us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771048192}
I20260709 13:14:52.579953 30591 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:52.695756 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:52.700811 22760 consensus_meta.cc:287] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.179s	user 0.009s	sys 0.005s
I20260709 13:14:52.632781 30290 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.704690 22760 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:52.712448 22759 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:52.714810 22760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 1184) took 1573 ms (client timeout 2077 ms). Trace:
W20260709 13:14:52.715382 22760 rpcz_store.cc:269] 0709 13:14:51.145758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.421091 (+275333us) service_pool.cc:224] Handling call
0709 13:14:52.714728 (+1293637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.717612 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.723129 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.681491 30213 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:52.722138 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.738157 22757 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:52.742000 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.647643 30551 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:52.710212 30578 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.011s	user 0.011s	sys 0.004s
W20260709 13:14:52.740545 30311 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.000s	sys 0.033s
I20260709 13:14:52.673324 30304 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:52.746331 22765 raft_consensus.cc:2393] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:52.748620 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 280873us
I20260709 13:14:52.650460 30544 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.780550 22762 raft_consensus.cc:2393] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:52.786075 22763 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:14:52.776546 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.793897 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:52.775501 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.798880 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:52.799782 22763 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:52.801359 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:52.786103 30311 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.000s	sys 0.034s
W20260709 13:14:52.803203 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60071.6us
I20260709 13:14:52.797631 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.787496 30455 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:52.725996 30290 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.723402 30608 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.070s
I20260709 13:14:52.724350 30276 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:52.823527 22764 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:52.823992 22759 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:52.833519 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1178) took 1688 ms. Trace:
I20260709 13:14:52.834170 22764 rpcz_store.cc:276] 0709 13:14:51.139407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.806619 (+1667212us) service_pool.cc:224] Handling call
0709 13:14:52.823167 (+ 16548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.825306 (+  2139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.827672 (+  2366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.824285 22760 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:52.815016 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:52.843240 22757 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:52.844105 22766 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:52.835767 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1177) took 1696 ms. Trace:
I20260709 13:14:52.851294 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1147) took 1469 ms. Trace:
I20260709 13:14:52.851850 22914 rpcz_store.cc:276] 0709 13:14:51.381776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.392476 (+1010700us) service_pool.cc:224] Handling call
0709 13:14:52.417824 (+ 25348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.445826 (+ 28002us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:52.650882 (+205056us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.832113 (+181231us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.850877 (+ 18764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.850963 (+    86us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:52.851025 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.851104 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.851231 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":370199,"raft.queue_time_us":393,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":361891,"threads_started":2}
I20260709 13:14:52.847488 22759 rpcz_store.cc:276] 0709 13:14:51.139001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.806734 (+1667733us) service_pool.cc:224] Handling call
0709 13:14:52.819720 (+ 12986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.828292 (+  8572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.835654 (+  7362us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:52.862252 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.876973 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 84682.3us
W20260709 13:14:52.886024 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 348359us
W20260709 13:14:52.850389 30296 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.180s	user 0.013s	sys 0.002s
I20260709 13:14:52.934135 22759 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:52.932358 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:52.962503 22766 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:14:52.963800 22766 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3036) took 1436 ms. Trace:
I20260709 13:14:52.964497 22766 rpcz_store.cc:276] 0709 13:14:51.527463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.845361 (+1317898us) service_pool.cc:224] Handling call
0709 13:14:52.962134 (+116773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:52.963389 (+  1255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.963714 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:52.968782 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.849897 30611 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.210s
I20260709 13:14:52.882989 30508 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.002211 22907 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
W20260709 13:14:52.932893 30557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.022s	sys 0.011s
W20260709 13:14:53.002288 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 285417us
I20260709 13:14:53.011824 22906 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:53.016886 22759 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:52.847087 30344 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.165s
I20260709 13:14:53.009258 22907 pending_rounds.cc:85] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 15
I20260709 13:14:53.048269 22907 pending_rounds.cc:107] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:14:53.048635 22907 raft_consensus.cc:2889] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:53.012624 30557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.651s	user 0.023s	sys 0.011s
W20260709 13:14:53.045917 22911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3831) took 2343 ms (client timeout 1550 ms). Trace:
I20260709 13:14:53.064129 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1148) took 1614 ms. Trace:
I20260709 13:14:52.950089 30608 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
W20260709 13:14:53.067809 22911 rpcz_store.cc:269] 0709 13:14:50.674845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.737871 (+ 63026us) service_pool.cc:224] Handling call
0709 13:14:52.995663 (+2257792us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54003ca0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:53.017615 (+ 21952us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046176512}
I20260709 13:14:53.007372 30321 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:53.079728 22915 rpcz_store.cc:276] 0709 13:14:51.450178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.481966 (+1031788us) service_pool.cc:224] Handling call
0709 13:14:52.514869 (+ 32903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.541157 (+ 26288us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 13:14:52.791732 (+250575us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.051724 (+259992us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.051916 (+   192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:53.051970 (+    54us) raft_consensus.cc:1645] Marking committed up to 21
0709 13:14:53.063706 (+ 11736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.063834 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.064036 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":335266,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":20957,"thread_start_us":426584,"threads_started":2}
I20260709 13:14:53.095250 22759 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:53.111433 22757 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
W20260709 13:14:53.123942 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 273516us
W20260709 13:14:53.128856 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 297789us
I20260709 13:14:53.133353 22759 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:14:53.133992 22759 pending_rounds.cc:85] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 10
I20260709 13:14:53.134272 22759 pending_rounds.cc:107] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:14:53.052050 30541 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:53.134341 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 184841us
I20260709 13:14:53.086300 30618 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:53.134577 22759 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:53.135174 22757 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:53.157321 30618 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:53.163743 30618 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:53.163662 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 184440us
I20260709 13:14:53.164793 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:14:53.165802 30620 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 13:14:53.167888 30622 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:14:53.070927 30508 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:53.174407 30613 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:14:53.176450 30508 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:53.154909 30310 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:53.177127 30616 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:14:53.162755 22766 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:14:53.183789 22766 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:53.187830 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.198655 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.219616 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.226636 22732 connection.cc:256] RPC call timeout handler was delayed by -2.0272750640001505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
I20260709 13:14:53.213852 22760 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:53.233486 22760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 3567) took 1070 ms. Trace:
I20260709 13:14:53.229908 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1182) took 2089 ms. Trace:
I20260709 13:14:53.235002 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:53.237771 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1346) took 1043 ms. Trace:
I20260709 13:14:53.237563 22762 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: [18.12-20.13]
I20260709 13:14:53.182862 30591 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:53.246202 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 124329us
I20260709 13:14:53.185413 30551 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:53.190640 30290 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.235122 22760 rpcz_store.cc:276] 0709 13:14:52.163418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.789423 (+626005us) service_pool.cc:224] Handling call
0709 13:14:53.233389 (+443966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:53.254191 22757 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:53.254658 22757 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:53.242013 30512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.001s	sys 0.012s
W20260709 13:14:53.250168 22732 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.506s (SENT) [suppressed 20 similar messages]
W20260709 13:14:53.255707 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 98712.8us
I20260709 13:14:53.225396 30621 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.180708 30311 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:14:53.238652 23464 rpcz_store.cc:276] 0709 13:14:52.194104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.194636 (+   532us) service_pool.cc:224] Handling call
0709 13:14:52.215829 (+ 21193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.315101 (+ 99272us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:52.596015 (+280914us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.867671 (+271656us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.066310 (+198639us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.088577 (+ 22267us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.229535 (+140958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:53.231058 (+  1523us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:53.237456 (+  6398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.237571 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.237705 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54380,"prepare.queue_time_us":35284,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":6758,"raft.queue_time_us":762561,"raft.run_cpu_time_us":618,"raft.run_wall_time_us":41007,"spinlock_wait_cycles":11955584,"thread_start_us":750830,"threads_started":4,"wal-append.queue_time_us":432525}
I20260709 13:14:53.236689 30520 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.238471 22763 rpcz_store.cc:276] 0709 13:14:51.140882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.807144 (+1666262us) service_pool.cc:224] Handling call
0709 13:14:52.810797 (+  3653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.817321 (+  6524us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:14:52.885269 (+ 67948us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.228865 (+343596us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:53.228994 (+   129us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:53.229143 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.229347 (+   204us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.229762 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65360,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":191334,"threads_started":1}
W20260709 13:14:53.264199 30512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.001s	sys 0.014s
I20260709 13:14:53.276456 22807 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:53.279726 22766 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 25.10, Last appended by leader: 10, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.264881 30344 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:53.313676 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:53.285416 23430 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:53.320261 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1146) took 1939 ms. Trace:
I20260709 13:14:53.321954 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2413) took 2120 ms. Trace:
I20260709 13:14:53.283857 30621 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.288198 30520 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.282166 30236 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:53.325938 30236 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.326552 30621 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:53.328282 30236 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.318699 30544 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:53.329665 30614 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.468s	user 0.002s	sys 0.000s
W20260709 13:14:53.330235 30626 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:14:53.331164 30520 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:53.331459 30642 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:53.337781 30619 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.406s	user 0.001s	sys 0.001s
W20260709 13:14:53.339653 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79204.9us
I20260709 13:14:53.340777 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:53.313547 30344 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:53.322374 22834 rpcz_store.cc:276] 0709 13:14:51.197318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.197768 (+   450us) service_pool.cc:224] Handling call
0709 13:14:51.198522 (+   754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.435852 (+237330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.312103 (+1876251us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540017ded4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:53.317552 (+  5449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267435136}
I20260709 13:14:53.267107 30551 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:14:53.351074 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 66911.8us
I20260709 13:14:53.353276 22766 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:53.320786 22910 rpcz_store.cc:276] 0709 13:14:51.381335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.314821 (+933486us) service_pool.cc:224] Handling call
0709 13:14:52.482100 (+167279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.585444 (+103344us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:52.827135 (+241691us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.318182 (+491047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:53.318242 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:53.318299 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.318387 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.320189 (+  1802us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78708,"prepare.queue_time_us":408599,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":355564,"threads_started":2}
W20260709 13:14:53.358840 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.350286 30344 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:53.376145 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 107802us
W20260709 13:14:53.378998 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.384581 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.385602 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.412086 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1352) took 1289 ms. Trace:
I20260709 13:14:53.403714 22913 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:53.414332 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:53.427201 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 177283us
I20260709 13:14:53.422693 22841 rpcz_store.cc:276] 0709 13:14:52.122519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.167622 (+ 45103us) service_pool.cc:224] Handling call
0709 13:14:52.168117 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.450995 (+282878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.411771 (+960776us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54003228d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:53.411982 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646484352}
I20260709 13:14:53.375188 30621 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:53.404013 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3547) took 4987 ms. Trace:
I20260709 13:14:53.434165 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2472) took 3293 ms. Trace:
I20260709 13:14:53.436154 22913 rpcz_store.cc:276] 0709 13:14:50.140284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.381388 (+3241104us) service_pool.cc:224] Handling call
0709 13:14:53.403503 (+ 22115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.433867 (+ 30364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.434104 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:53.376186 30304 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:53.389803 30627 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.257s	user 0.001s	sys 0.001s
I20260709 13:14:53.400655 30512 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:53.403291 30212 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:53.442054 22766 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 25 index: 10. Preceding OpId from leader: term: 26 index: 10. (term mismatch)
W20260709 13:14:53.414598 30515 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.853s	user 0.013s	sys 0.004s
W20260709 13:14:53.413838 30577 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 2.471s	user 0.005s	sys 0.013s
I20260709 13:14:53.404711 30520 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.405035 30640 raft_consensus.cc:2764] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:14:53.444751 22766 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 9
I20260709 13:14:53.411527 30624 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:53.404309 30461 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:53.455623 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 87114.1us
I20260709 13:14:53.441138 30512 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.436023 22758 rpcz_store.cc:276] 0709 13:14:48.328871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.262556 (+933685us) service_pool.cc:224] Handling call
0709 13:14:49.262962 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:52.369644 (+3106682us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18002a8e00. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f68a16994bc 00007f689e9e748b 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea01455 00007f68a1ae9060 00007f68a1ae9bfb 00007f68a1af464b
0709 13:14:52.403945 (+ 34301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.403982 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:52.451738 (+ 47756us) log.cc:844] Serialized 26 byte log entry
0709 13:14:52.682617 (+230879us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:52.682654 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.694657 (+ 12003us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.283654 (+588997us) raft_consensus.cc:1685] finished
0709 13:14:53.283688 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.316611 (+ 32923us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1175682,"raft.queue_time_us":1228966,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":13659,"spinlock_wait_cycles":7920896,"thread_start_us":365289,"threads_started":2,"wal-append.queue_time_us":364680}
W20260709 13:14:53.454505 30293 consensus_meta.cc:287] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.890s	user 0.013s	sys 0.004s
I20260709 13:14:53.472050 22766 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 25.10
I20260709 13:14:53.444689 30212 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.469861 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:53.473873 22766 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:53.449647 30570 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:53.471719 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:53.413954 30618 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:53.478133 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 313471us
I20260709 13:14:53.418557 30456 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:14:53.479823 22911 raft_consensus.cc:2410] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:14:53.477609 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2459) took 3386 ms. Trace:
I20260709 13:14:53.482069 22764 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:53.453399 30311 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.482260 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:53.480633 22908 rpcz_store.cc:276] 0709 13:14:50.086563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.866350 (+779787us) service_pool.cc:224] Handling call
0709 13:14:50.866949 (+   599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.471943 (+604994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.452655 (+1980712us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b5400041354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:53.472925 (+ 20270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096425856}
I20260709 13:14:53.485087 22761 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:14:53.482769 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68563.7us
I20260709 13:14:53.470350 30512 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:53.449427 30304 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.477787 30212 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:53.494957 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:53.453609 30515 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.498358 30648 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:14:53.499172 30407 raft_consensus.cc:3055] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:53.462380 30520 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.488420 30570 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:53.520944 30570 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.522464 30644 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:53.528437 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:53.471184 30461 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.524745 30234 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:53.534446 30234 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.534852 30234 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:53.535324 30567 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:53.537696 30648 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:53.537993 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1353) took 1415 ms. Trace:
I20260709 13:14:53.538192 30648 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:53.538569 22833 rpcz_store.cc:276] 0709 13:14:52.122802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.168417 (+ 45615us) service_pool.cc:224] Handling call
0709 13:14:52.168840 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.230626 (+ 61786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.230685 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.236694 (+  6009us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:52.236743 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.537697 (+1300954us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b540019f5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:53.537761 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.537922 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96745856}
I20260709 13:14:53.539561 30648 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.543489 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:53.544833 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3033) took 2061 ms. Trace:
I20260709 13:14:53.571835 22764 rpcz_store.cc:276] 0709 13:14:51.483762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.835290 (+1351528us) service_pool.cc:224] Handling call
0709 13:14:52.887183 (+ 51893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.482532 (+595349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.544593 (+ 62061us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b540002e9d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:53.544770 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367622912}
I20260709 13:14:53.516057 30311 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.547679 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:53.547968 30632 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
W20260709 13:14:53.548933 22732 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 2.140s (SENT)
I20260709 13:14:53.577440 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:53.550275 22909 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:53.480341 30293 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.554306 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1175) took 2417 ms. Trace:
I20260709 13:14:53.511574 30407 raft_consensus.cc:740] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:53.581208 22913 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:53.579039 22911 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:53.579864 22915 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:53.587977 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.591212 30618 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:53.591701 30213 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.842s	user 0.006s	sys 0.010s
I20260709 13:14:53.535446 30461 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:53.593327 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 172523us
I20260709 13:14:53.594642 23458 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:14:53.599097 22837 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:14:53.578063 30591 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.580875 22765 rpcz_store.cc:276] 0709 13:14:51.137909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.808287 (+1670378us) service_pool.cc:224] Handling call
0709 13:14:52.818591 (+ 10304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:52.825253 (+  6662us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:53.083863 (+258610us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.241115 (+157252us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:53.276949 (+ 35834us) log.cc:844] Serialized 26 byte log entry
0709 13:14:53.277159 (+   210us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:53.277194 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.277279 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.554064 (+276785us) raft_consensus.cc:1685] finished
0709 13:14:53.554088 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.554230 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50891,"prepare.queue_time_us":219724,"raft.queue_time_us":175428,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":44200,"spinlock_wait_cycles":34752128,"thread_start_us":218167,"threads_started":2,"wal-append.queue_time_us":374875}
W20260709 13:14:53.595031 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51716.8us
I20260709 13:14:53.604684 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:53.606563 22913 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:53.610561 30655 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:53.611498 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.611392 30655 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.612246 22840 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:53.613756 30660 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:53.614298 30660 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.614674 30660 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:53.618225 30618 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:53.618961 30658 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:53.619760 22909 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 23 index: 9. Preceding OpId from leader: term: 26 index: 10. (index mismatch)
W20260709 13:14:53.620656 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.622251 22762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3030) took 2139 ms. Trace:
I20260709 13:14:53.622666 22762 rpcz_store.cc:276] 0709 13:14:51.482853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.820452 (+1337599us) service_pool.cc:224] Handling call
0709 13:14:53.233163 (+412711us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:53.240819 (+  7656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.622040 (+381221us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540035dc50. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:53.622184 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867188992}
I20260709 13:14:53.621182 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:53.630230 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.632706 22909 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:14:53.629974 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.650260 23466 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 28.
I20260709 13:14:53.644515 22840 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:53.628671 30204 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 13:14:53.662711 30204 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:53.662850 22761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3009) took 5068 ms. Trace:
I20260709 13:14:53.663123 30204 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:53.663303 22761 rpcz_store.cc:276] 0709 13:14:48.594264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.712862 (+1118598us) service_pool.cc:224] Handling call
0709 13:14:53.356686 (+3643824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.488003 (+131317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.662555 (+174552us) spinlock_profiling.cc:243] Waited 3.58 s on lock 0x7b54003403d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:53.662776 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8224868096}
I20260709 13:14:53.664143 30204 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.666419 30311 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:53.631366 30304 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:53.667423 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 60826.2us
I20260709 13:14:53.668577 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:53.669445 30555 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:53.670050 30555 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:53.665848 30407 consensus_queue.cc:260] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.671078 30555 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.629163 30276 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.672550 23462 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:53.673034 30618 consensus_queue.cc:1059] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:53.673441 30234 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.674182 30618 raft_consensus.cc:3055] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:53.674571 30618 raft_consensus.cc:740] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:53.675503 30618 consensus_queue.cc:260] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:53.677214 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 128327us
W20260709 13:14:53.687350 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.625469 22757 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:53.690639 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47606.5us
I20260709 13:14:53.695648 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.695712 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.695503 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.633910 30515 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.680476 30648 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:53.703655 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.705672 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.708806 22757 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:53.728478 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:53.737659 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.730149 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:53.718150 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.718282 23460 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:14:53.738611 22840 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:14:53.714534 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:53.713780 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:53.742753 22730 connection.cc:256] RPC call timeout handler was delayed by -1.1506452490000356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:14:53.743379 22730 connection.cc:256] RPC call timeout handler was delayed by -1.06064524899989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:14:53.744771 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.752367 22913 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:53.745633 22832 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:53.729249 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.704435 30618 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:53.760769 22766 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:14:53.761534 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.688575 30591 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.764583 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.729758 30293 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.726329 30234 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:53.784050 22732 connection.cc:256] RPC call timeout handler was delayed by -1.8535177960000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 13:14:53.784592 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:53.717147 30520 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:53.753674 30567 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.776919 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:53.705605 30461 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 13:14:53.776716 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.787883 22757 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:53.784474 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:53.793800 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.778079 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:53.812793 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:53.813660 30643 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.814386 30643 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.816150 30591 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.816632 30643 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:53.816960 30567 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.811167 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:53.817605 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.819269 30567 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:53.819501 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.820302 22832 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 25: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:53.813607 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 44237.1us
I20260709 13:14:53.822988 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:53.810315 22765 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:53.824535 30515 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:53.825698 30655 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:53.828122 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:53.811596 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:53.724887 30344 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:53.824502 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.813724 22908 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:53.821244 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.829192 30515 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:53.807121 30608 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.006s	sys 0.036s
W20260709 13:14:53.850714 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.846861 22913 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:53.851987 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1145) took 3355 ms. Trace:
I20260709 13:14:53.857774 22906 rpcz_store.cc:276] 0709 13:14:50.496901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.924121 (+1427220us) service_pool.cc:224] Handling call
0709 13:14:53.006124 (+1082003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.036579 (+ 30455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.851731 (+815152us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003ca0d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:53.851896 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391799168}
I20260709 13:14:53.848547 22832 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:53.829281 30554 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:53.867729 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:53.830662 30643 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:53.879213 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 46787.8us
W20260709 13:14:53.861220 30608 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.006s	sys 0.036s
I20260709 13:14:53.882560 22911 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:53.890628 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:53.885042 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:53.897089 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.899215 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:53.899999 22908 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:53.905103 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:53.906193 22911 raft_consensus.cc:2410] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:14:53.907616 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.926743 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:53.929018 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 219988us
I20260709 13:14:53.851233 30640 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
W20260709 13:14:53.962999 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 146492us
W20260709 13:14:53.961010 23431 connection.cc:256] RPC call timeout handler was delayed by -2.3842227510001521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 13:14:53.963950 23431 connection.cc:256] RPC call timeout handler was delayed by -2.0706738330000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 13:14:53.964696 23431 connection.cc:256] RPC call timeout handler was delayed by -1.9099400509999214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 13:14:53.966177 23431 connection.cc:256] RPC call timeout handler was delayed by -1.4747317580001891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19830000000000003s for a response
W20260709 13:14:53.967294 23431 connection.cc:256] RPC call timeout handler was delayed by -1.4183119230001466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
W20260709 13:14:53.969741 23431 connection.cc:256] RPC call timeout handler was delayed by -1.3743340529999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:14:53.963894 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 135909us
I20260709 13:14:53.968066 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.975135 22841 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:53.879299 30660 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:53.878676 30432 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:14:53.926964 30642 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:53.978245 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:53.985688 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:53.985126 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:54.038400 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.050429 22913 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:54.042273 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:54.053576 22911 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 27.
I20260709 13:14:53.909926 30555 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 13:14:54.057070 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.058197 23463 raft_consensus.cc:2410] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 28 because replica has last-logged OpId of term: 27 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 13:14:54.059126 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.060376 23463 raft_consensus.cc:2410] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:14:54.065243 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.065871 22764 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:54.067344 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.068228 23465 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:54.061383 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.070421 23463 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:54.070241 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.071781 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.072528 23458 raft_consensus.cc:2393] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:54.073433 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.074901 23458 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:14:54.075724 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.078356 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:54.069332 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:54.093647 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:53.909466 30512 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:54.098973 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:53.929602 30212 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:54.107020 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.109845 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:54.111445 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:54.112170 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:54.113687 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:54.116963 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.117558 22764 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:54.117414 30663 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:14:54.118911 22915 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:14:54.119071 30660 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:54.119918 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 145877us
I20260709 13:14:54.158092 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:54.120314 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:54.120487 30324 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:54.123333 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 71691.4us
I20260709 13:14:54.126893 30311 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:54.128391 23460 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:54.131354 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:54.139437 22841 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:14:54.154135 23462 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:54.155228 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:54.155632 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.163924 22807 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:54.132032 30551 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:54.166168 22838 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:54.175196 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:54.175714 22908 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
W20260709 13:14:54.156112 30630 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.976s	user 0.012s	sys 0.003s
I20260709 13:14:54.177218 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:54.180677 22913 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 13:14:54.167021 22762 raft_consensus.cc:2376] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22. Re-sending same reply.
I20260709 13:14:54.155699 30512 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:54.193053 23457 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:54.194962 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:54.189229 22807 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:54.194133 30608 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:54.201860 30293 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:54.178974 30630 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.013s	sys 0.003s
W20260709 13:14:54.207664 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.212144 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.212462 30297 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:54.213785 22762 raft_consensus.cc:1240] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:54.214955 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.215592 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.215822 22761 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:54.216897 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.226861 22758 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:14:54.231627 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.232551 22760 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:54.233736 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.183343 30461 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:54.194310 30660 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:54.247474 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 58325.4us
W20260709 13:14:54.250669 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55850.4us
I20260709 13:14:54.225356 30296 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:54.225178 30297 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:54.284435 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.290879 22758 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:54.256474 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.249197 22915 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 16.
I20260709 13:14:54.226651 30293 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:54.224372 30567 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:54.225750 30304 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:54.293548 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:54.301403 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:54.249450 30344 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:54.292320 30544 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:54.303238 30653 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.303697 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.352672 22761 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:54.309839 30304 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:54.310381 30212 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:54.310658 30567 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:54.312008 22915 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:14:54.281734 30671 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:54.295107 30296 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:54.316376 30675 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:14:54.292682 22910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
W20260709 13:14:54.323351 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.388186 22910 raft_consensus.cc:606] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:54.388661 22910 raft_consensus.cc:614] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:54.323614 30455 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:54.325098 30297 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:54.331218 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 82613.2us
W20260709 13:14:54.389460 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 88159.7us
I20260709 13:14:54.332454 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1143) took 3855 ms. Trace:
W20260709 13:14:54.334079 30674 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:14:54.337749 22840 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:54.342444 30204 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:54.242874 30461 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:54.400106 22765 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:54.356743 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.400933 22763 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:54.358498 30684 log.cc:927] Time spent T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:14:54.369599 23459 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:14:54.369398 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:54.250057 30608 raft_consensus.cc:697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:54.390264 22907 rpcz_store.cc:276] 0709 13:14:50.476792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.671964 (+1195172us) service_pool.cc:224] Handling call
0709 13:14:51.738114 (+ 66150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.059390 (+1321276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.331443 (+1272053us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400043654. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:54.332341 (+   898us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2795643648}
I20260709 13:14:54.390763 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:54.392951 22760 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:54.303639 30293 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:54.394125 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:54.394125 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.399286 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.371549 30432 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.443341 22766 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:54.443915 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.441960 22764 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:54.368400 30304 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:54.403330 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 185541us
W20260709 13:14:54.441222 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:54.447245 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 77994.1us
I20260709 13:14:54.411284 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.304632 30660 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:54.444516 22760 raft_consensus.cc:2376] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20. Re-sending same reply.
W20260709 13:14:54.408540 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119208us
I20260709 13:14:54.442196 23460 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:54.377693 30653 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:54.617802 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:54.504572 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57181.2us
I20260709 13:14:54.447873 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:54.448112 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57394.8us
W20260709 13:14:54.466142 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.783651 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:54.449340 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.788455 22762 raft_consensus.cc:2393] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:54.790350 22763 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:14:54.508695 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:54.504853 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 57004.5us
W20260709 13:14:54.446023 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45073.4us
I20260709 13:14:54.597893 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.598402 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.794440 22764 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:14:54.794595 22766 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:54.715205 22758 raft_consensus.cc:2468] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:54.394472 30455 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:54.403060 30461 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:54.624714 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:54.401590 30530 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:54.798743 22761 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:54.791167 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.403152 30642 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:54.399241 30515 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:54.806121 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:54.809530 22805 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:54.507575 30675 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:14:54.833122 22765 raft_consensus.cc:2376] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22. Re-sending same reply.
I20260709 13:14:54.856086 22910 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:54.618856 30517 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:54.549631 30551 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:14:54.653353 30296 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:54.873903 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2430) took 1151 ms. Trace:
I20260709 13:14:54.555863 30685 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:54.886587 22841 rpcz_store.cc:276] 0709 13:14:53.722316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.166906 (+444590us) service_pool.cc:224] Handling call
0709 13:14:54.167344 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.196372 (+ 29028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.196404 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.209843 (+ 13439us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.432627 (+222784us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:54.432701 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.433925 (+  1224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:54.873591 (+439666us) raft_consensus.cc:1685] finished
0709 13:14:54.873625 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:54.873825 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10161,"raft.queue_time_us":29661,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"thread_start_us":190054,"threads_started":1,"wal-append.queue_time_us":659312}
I20260709 13:14:54.448899 30671 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:54.891664 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:54.892180 22912 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:14:54.865311 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55935.8us
I20260709 13:14:54.804229 30297 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:54.867832 22913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.010s	sys 0.034s
W20260709 13:14:54.894712 22913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.010s	sys 0.034s
I20260709 13:14:54.880204 22764 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:54.880429 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:54.844625 30276 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:54.875700 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:54.896135 22760 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:54.508806 30508 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.374s
I20260709 13:14:54.879091 30551 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.650297 30608 consensus_queue.cc:237] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:54.905323 30515 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:54.932658 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:54.944058 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.919381 30296 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:54.873482 30544 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:14:54.929811 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.012223 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.015597 22915 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:14:55.017416 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:54.989321 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 78437.5us
W20260709 13:14:54.989737 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 94254.9us
I20260709 13:14:54.446709 30432 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:54.999861 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2426) took 1277 ms (client timeout 1661 ms). Trace:
I20260709 13:14:55.003584 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:54.989114 22841 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:55.003082 30551 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:55.020922 30691 consensus_queue.cc:1059] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
W20260709 13:14:55.029491 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.030412 30515 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:55.032192 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3862) took 1046 ms. Trace:
I20260709 13:14:55.032521 22915 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:54.994324 30276 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:55.010848 30688 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.380s	user 0.003s	sys 0.002s
I20260709 13:14:55.031389 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2429) took 1309 ms. Trace:
I20260709 13:14:55.023136 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2434) took 1251 ms. Trace:
I20260709 13:14:55.044611 22841 rpcz_store.cc:276] 0709 13:14:53.771558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.894797 (+1123239us) service_pool.cc:224] Handling call
0709 13:14:54.895147 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.022792 (+127645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.023070 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:55.025035 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 159343us
I20260709 13:14:55.046765 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.048600 22915 raft_consensus.cc:1240] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:55.047575 22912 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:55.051018 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:55.026139 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1362) took 1333 ms. Trace:
I20260709 13:14:55.052053 22841 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:55.023442 30239 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:55.050993 30458 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:55.032761 22913 rpcz_store.cc:276] 0709 13:14:53.985599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.195875 (+210276us) service_pool.cc:224] Handling call
0709 13:14:54.243093 (+ 47218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.283766 (+ 40673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.283809 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.289875 (+  6066us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.895239 (+605364us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:54.895304 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.895416 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.031894 (+136478us) raft_consensus.cc:1685] finished
0709 13:14:55.031928 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.032112 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2590,"raft.queue_time_us":1161,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":10337,"thread_start_us":550115,"threads_started":1,"wal-append.queue_time_us":657698}
I20260709 13:14:55.051672 22759 raft_consensus.cc:2376] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21. Re-sending same reply.
I20260709 13:14:55.020679 30567 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:55.060075 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.052599 22834 rpcz_store.cc:276] 0709 13:14:53.693314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.140846 (+447532us) service_pool.cc:224] Handling call
0709 13:14:54.141305 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.163478 (+ 22173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.163513 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.191540 (+ 28027us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.750345 (+558805us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:54.750417 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.750525 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.025865 (+275340us) raft_consensus.cc:1685] finished
0709 13:14:55.025893 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.026070 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8647,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":463913,"threads_started":1,"wal-append.queue_time_us":776731}
W20260709 13:14:55.022285 22840 rpcz_store.cc:269] 0709 13:14:53.715232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.743014 (+ 27782us) service_pool.cc:224] Handling call
0709 13:14:54.992241 (+1249227us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54000393d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:54.992459 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159638784}
I20260709 13:14:55.056663 30344 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:55.056345 30239 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.062055 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.072854 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:55.078605 22907 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:14:55.078800 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.044036 22839 rpcz_store.cc:276] 0709 13:14:53.722088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.142289 (+420201us) service_pool.cc:224] Handling call
0709 13:14:54.161367 (+ 19078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.184463 (+ 23096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.184633 (+   170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.282687 (+ 98054us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.600626 (+317939us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:54.600686 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.600788 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.031014 (+430226us) raft_consensus.cc:1685] finished
0709 13:14:55.031047 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.031294 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49887,"raft.queue_time_us":5281,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":3730,"thread_start_us":268314,"threads_started":1,"wal-append.queue_time_us":669529}
I20260709 13:14:55.083617 22841 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:55.084794 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.085215 22841 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:55.086530 30618 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:55.087348 30618 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.090111 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.071612 30296 raft_consensus.cc:3055] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:55.092693 22880 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.070916 30458 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.099958 22841 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:55.070863 30657 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:55.023590 30658 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:14:55.025624 30671 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.105772 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.109009 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1364) took 1018 ms. Trace:
I20260709 13:14:55.109611 22840 rpcz_store.cc:276] 0709 13:14:54.090224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.083003 (+992779us) service_pool.cc:224] Handling call
0709 13:14:55.098360 (+ 15357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.102630 (+  4270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.102659 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:55.108656 (+  5997us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:55.108711 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.108779 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.108923 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:55.092130 30690 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 13:14:55.114709 30694 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:14:55.116269 22913 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:14:55.116868 22913 pending_rounds.cc:85] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 10
I20260709 13:14:55.117107 22913 pending_rounds.cc:107] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:14:55.117048 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2432) took 1370 ms. Trace:
I20260709 13:14:55.117487 22913 raft_consensus.cc:2889] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:55.098505 30567 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:55.097246 30296 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:55.031584 30515 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.104887 30458 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:55.105777 30618 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:55.133924 22732 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.615s (SENT)
W20260709 13:14:55.134481 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.111927 30660 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:55.142443 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.143232 22907 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:55.120808 22834 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:55.126091 30567 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.117542 22838 rpcz_store.cc:276] 0709 13:14:53.746297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.235551 (+489254us) service_pool.cc:224] Handling call
0709 13:14:54.243141 (+  7590us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:54.247566 (+  4425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.247599 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:54.313936 (+ 66337us) log.cc:844] Serialized 52 byte log entry
0709 13:14:54.793640 (+479704us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:54.873170 (+ 79530us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.083545 (+210375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.093873 (+ 10328us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.116791 (+ 22918us) raft_consensus.cc:1685] finished
0709 13:14:55.116815 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.116976 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79776,"prepare.run_cpu_time_us":6916,"prepare.run_wall_time_us":98492,"raft.queue_time_us":39701,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":11060,"thread_start_us":689199,"threads_started":2,"wal-append.queue_time_us":581140}
I20260709 13:14:55.110538 30691 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:55.106249 30344 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.117816 30530 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:55.155628 30432 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:55.158972 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61114.1us
I20260709 13:14:55.123955 30636 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:14:55.175767 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.109467 30657 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.174285 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.287050 22913 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
W20260709 13:14:55.287381 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 229453us
I20260709 13:14:55.290355 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:55.166674 30567 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:55.167229 30618 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.295298 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.301226 22912 raft_consensus.cc:2368] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19.
I20260709 13:14:55.103196 30276 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.290561 30296 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.306949 30296 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:55.170611 30701 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.307966 30701 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.308219 30530 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.309689 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:55.311002 30517 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:55.329133 22757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3600) took 2791 ms. Trace:
I20260709 13:14:55.311498 30517 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:55.313334 30304 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:55.332000 22836 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:55.340395 30304 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:55.341863 22836 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2433) took 1578 ms. Trace:
I20260709 13:14:55.341703 30304 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.342286 22836 rpcz_store.cc:276] 0709 13:14:53.763358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.289944 (+526586us) service_pool.cc:224] Handling call
0709 13:14:55.331603 (+1041659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.340670 (+  9067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.341646 (+   976us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000a5350. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:55.341795 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2335070592}
I20260709 13:14:55.343194 30304 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:55.341876 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.312284 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3861) took 1349 ms. Trace:
I20260709 13:14:55.321307 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.320055 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3866) took 1257 ms. Trace:
W20260709 13:14:55.325238 30667 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.251s	user 0.006s	sys 0.006s
W20260709 13:14:55.337939 30683 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.008s	user 0.012s	sys 0.004s
I20260709 13:14:55.327020 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2856) took 1837 ms. Trace:
I20260709 13:14:55.344436 22913 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:55.314814 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.322399 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.345266 22906 rpcz_store.cc:276] 0709 13:14:54.062363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.363936 (+301573us) service_pool.cc:224] Handling call
0709 13:14:54.364995 (+  1059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:54.442500 (+ 77505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.442527 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:54.442560 (+    33us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:54.442593 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.319736 (+877143us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b5400002154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:55.319794 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.319977 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177685888}
I20260709 13:14:55.346429 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1158) took 1799 ms. Trace:
I20260709 13:14:55.344365 22908 rpcz_store.cc:276] 0709 13:14:53.963154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.185211 (+222057us) service_pool.cc:224] Handling call
0709 13:14:54.210803 (+ 25592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.428588 (+217785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.428644 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.448791 (+ 20147us) log.cc:844] Serialized 26 byte log entry
0709 13:14:55.023231 (+574440us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:55.023297 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.023464 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.307217 (+283753us) raft_consensus.cc:1685] finished
0709 13:14:55.307240 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.312209 (+  4969us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3980,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"thread_start_us":573718,"threads_started":2,"wal-append.queue_time_us":619533}
I20260709 13:14:55.349728 22914 rpcz_store.cc:276] 0709 13:14:53.546793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.550387 (+  3594us) service_pool.cc:224] Handling call
0709 13:14:53.550734 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.550978 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:53.551003 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:53.551571 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:14:53.741678 (+190107us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:53.741716 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.741788 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.324494 (+1582706us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:55.346073 (+ 21579us) raft_consensus.cc:1685] finished
0709 13:14:55.346101 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.346362 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13568768,"thread_start_us":189699,"threads_started":1,"wal-append.queue_time_us":355958}
I20260709 13:14:55.347898 22912 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
I20260709 13:14:55.351366 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:55.345649 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 208342us
I20260709 13:14:55.348989 22757 rpcz_store.cc:276] 0709 13:14:52.538159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.155245 (+617086us) service_pool.cc:224] Handling call
0709 13:14:53.169953 (+ 14708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.625080 (+455127us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54000530a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a6f06f 00007f68a1ae7f60 00007f68a1ae97de 00007f68a1af3b49 00007f68a1af4416 00007f68a1af0d8f 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62
0709 13:14:53.811473 (+186393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.328886 (+1517413us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b54002dbed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:55.329064 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336645376}
I20260709 13:14:55.311655 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:55.360222 22907 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
W20260709 13:14:55.312661 30669 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.223s	user 0.009s	sys 0.007s
I20260709 13:14:55.347522 22837 rpcz_store.cc:276] 0709 13:14:53.489376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.489610 (+   234us) service_pool.cc:224] Handling call
0709 13:14:53.516633 (+ 27023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.601809 (+ 85176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.326753 (+1724944us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b54000a5354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:55.326958 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189015936}
W20260709 13:14:55.295181 30204 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.886s	user 0.011s	sys 0.008s
I20260709 13:14:55.376976 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.371884 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:55.374168 22910 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:14:55.380195 22910 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
W20260709 13:14:55.369056 30611 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.282s	user 0.041s	sys 0.055s
W20260709 13:14:55.318293 30697 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:14:55.294474 30702 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.299573 30648 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.584s	user 0.013s	sys 0.007s
W20260709 13:14:55.376541 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64961.4us
I20260709 13:14:55.406437 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:55.297392 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:14:55.328171 30293 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
I20260709 13:14:55.379029 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.379153 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:55.328279 30633 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:55.334170 30344 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.381137 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.311939 30432 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:55.331156 30455 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.362363 30618 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.381798 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.382403 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.383234 30567 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.384248 30710 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:55.378387 30660 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:55.387898 30461 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.002s
I20260709 13:14:55.389750 30512 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.392542 30517 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:55.369305 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 289011us
I20260709 13:14:55.441624 30517 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.396327 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.442657 22908 raft_consensus.cc:2376] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23. Re-sending same reply.
I20260709 13:14:55.442955 30567 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:55.443720 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.444958 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.446552 22911 raft_consensus.cc:2368] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
I20260709 13:14:55.451122 22912 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:55.451269 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3863) took 1415 ms. Trace:
I20260709 13:14:55.451920 22909 rpcz_store.cc:276] 0709 13:14:54.036111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.237077 (+200966us) service_pool.cc:224] Handling call
0709 13:14:54.247826 (+ 10749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:54.369913 (+122087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.369962 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:54.370028 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:54.370074 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.450796 (+1080722us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003381d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:55.450866 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.451181 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280152448}
I20260709 13:14:55.453778 30709 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:55.454435 30709 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:55.455510 30709 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.458061 30643 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:55.458613 30643 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:55.459937 30643 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.452198 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.465967 22908 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:14:55.466894 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.467909 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:55.402359 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.469851 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.470114 22914 raft_consensus.cc:2410] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:14:55.410091 22730 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.411963 22906 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21. Re-sending same reply.
I20260709 13:14:55.381141 30297 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:55.422004 30344 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:55.417325 30293 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.422173 30644 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.427743 30455 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:55.474571 22841 raft_consensus.cc:1240] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:14:55.404284 30611 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.322s	user 0.041s	sys 0.055s
W20260709 13:14:55.432385 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 258262us
I20260709 13:14:55.477249 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.475973 22915 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:55.478209 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.432418 30655 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.435668 30512 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.405810 30458 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.396497 22907 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:55.468683 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.472591 30710 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:55.461376 30643 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:55.477419 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67429.6us
I20260709 13:14:55.481945 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.484022 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.501039 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:55.484597 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 187317us
I20260709 13:14:55.502089 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.490236 22838 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:55.491359 22912 raft_consensus.cc:2376] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23. Re-sending same reply.
W20260709 13:14:55.434191 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.496104 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.461844 30705 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:55.496773 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.508630 22908 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:55.497285 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:55.509406 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43702 (request call id 1102) took 1112 ms. Trace:
I20260709 13:14:55.491946 22880 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.504554 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.509913 23457 rpcz_store.cc:276] 0709 13:14:54.397463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.397658 (+   195us) service_pool.cc:224] Handling call
0709 13:14:54.398141 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.409838 (+ 11697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.409886 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.507133 (+ 97247us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.895801 (+388668us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:54.895845 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.895985 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.509189 (+613204us) raft_consensus.cc:1685] finished
0709 13:14:55.509211 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.509343 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2958,"raft.queue_time_us":11299,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5469,"thread_start_us":388245,"threads_started":1,"wal-append.queue_time_us":605873}
I20260709 13:14:55.511569 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.511906 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.485706 30660 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:55.474479 30297 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:55.501921 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.507566 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.517590 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:55.505757 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.483614 30690 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:55.513973 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.515040 22880 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.519402 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.518512 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.519860 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.520463 22912 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 13:14:55.511723 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:55.466276 30461 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.432896 30702 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.522003 22908 raft_consensus.cc:2393] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:55.521540 22913 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:55.496047 30710 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:55.522806 22880 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.522367 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.498524 30713 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:55.521559 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.544831 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.556526 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.547792 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.485554 30344 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:14:55.585466 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.515525 30432 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.594532 22837 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:55.555503 30304 raft_consensus.cc:2804] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:55.547057 30708 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:55.595417 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.595908 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.596505 22836 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 23: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:55.551717 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:55.586927 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63703.4us
I20260709 13:14:55.590394 22907 raft_consensus.cc:2376] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21. Re-sending same reply.
I20260709 13:14:55.557648 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.598572 22913 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:55.553321 22840 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:55.589207 22914 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:55.550920 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:55.563649 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.600394 22912 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:55.519354 30690 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:55.491528 30655 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.550302 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:55.602941 22912 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:14:55.551535 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.595665 30708 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.597716 22763 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:55.606185 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.625023 22911 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:55.479959 30644 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:55.607474 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 102833us
I20260709 13:14:55.608264 22839 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:55.523250 30296 raft_consensus.cc:2804] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:55.625821 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.637681 22914 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:55.630497 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.519119 30297 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:55.633024 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.522713 30657 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.635733 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.635943 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.642768 22907 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:55.554888 30458 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:55.636780 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.646994 22908 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:55.639067 22837 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
W20260709 13:14:55.646467 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 129787us
I20260709 13:14:55.647837 22911 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:14:55.606917 30668 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 1.515s	user 0.003s	sys 0.027s
I20260709 13:14:55.557248 30710 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:55.620676 30690 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:55.646700 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.517023 30642 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 13:14:55.596858 30311 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
W20260709 13:14:55.721556 22879 connection.cc:256] RPC call timeout handler was delayed by -2.1353759640001044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:14:55.722078 22879 connection.cc:256] RPC call timeout handler was delayed by -2.1353759640001044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:14:55.722609 22879 connection.cc:256] RPC call timeout handler was delayed by -2.011175964000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 13:14:55.722990 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9742759640000713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 13:14:55.723344 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9742759640000713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 13:14:55.687477 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45149.6us
I20260709 13:14:55.702350 22832 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:55.547384 30567 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:14:55.648242 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:55.614768 30713 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:55.703488 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.626729 30708 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:55.723095 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:55.688627 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:55.725634 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9742759640000713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 13:14:55.736819 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9364759640000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:14:55.737178 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9328759640000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 13:14:55.737491 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9265759639999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 13:14:55.728463 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.730716 22911 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:14:55.737803 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9193759639999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:14:55.736910 22837 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:14:55.739548 22879 connection.cc:256] RPC call timeout handler was delayed by -1.9193759639999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:14:55.633001 30432 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.741070 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:55.740579 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:55.741212 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:55.740113 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:55.741292 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:55.742544 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8536759640001037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260709 13:14:55.744936 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:55.747881 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8536759640001037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 13:14:55.748234 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8410759640000833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:14:55.748531 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8374759640000775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 13:14:55.748798 22879 connection.cc:256] RPC call timeout handler was delayed by -1.8356759640000746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 13:14:55.749058 22879 connection.cc:256] RPC call timeout handler was delayed by -1.7852759639999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:14:55.749321 22879 connection.cc:256] RPC call timeout handler was delayed by -1.6043759640001554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:14:55.745491 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53691.5us
W20260709 13:14:55.749604 22879 connection.cc:256] RPC call timeout handler was delayed by -1.5503759640000681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:14:55.749965 22879 connection.cc:256] RPC call timeout handler was delayed by -1.4999759639999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
I20260709 13:14:55.749794 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:55.750449 22879 connection.cc:256] RPC call timeout handler was delayed by -1.4792759640001805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2283s for a response
W20260709 13:14:55.750844 22879 connection.cc:256] RPC call timeout handler was delayed by -1.3163759640001444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2464s for a response
I20260709 13:14:55.750608 23459 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:55.702967 30691 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:55.754048 22908 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:55.755600 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.757148 23466 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 23: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:55.759281 23457 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:14:55.763427 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:55.729794 30458 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:55.764662 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:55.733119 30701 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.737025 30713 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:55.787806 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 139652us
I20260709 13:14:55.729863 30671 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:14:55.796056 22806 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:55.754942 30703 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.745873 30567 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer acfda9b350734def9b6c621f66a5f247 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:55.736521 30611 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:14:55.801270 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.802559 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.815197 22878 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.778753 30293 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:55.798883 30691 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.807480 30567 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:14:55.810354 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55831.5us
I20260709 13:14:55.811151 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:55.812649 30716 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:14:55.815806 22880 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.943s (SENT)
I20260709 13:14:55.777784 30458 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.846060 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.847009 22907 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:55.797278 30618 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.821753 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:55.808598 30296 raft_consensus.cc:697] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:55.857406 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 106373us
I20260709 13:14:55.815053 30344 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:55.866781 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:55.868613 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:55.898478 22766 raft_consensus.cc:2376] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23. Re-sending same reply.
I20260709 13:14:55.850369 30461 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:55.869364 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2858) took 2300 ms. Trace:
I20260709 13:14:55.907440 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:55.870105 30618 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:14:55.850755 30458 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:55.872156 30551 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.846s	user 0.009s	sys 0.005s
I20260709 13:14:55.920868 22834 raft_consensus.cc:2410] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:14:55.821120 30567 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:14:55.881458 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.926954 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:55.927716 22766 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 15.
I20260709 13:14:55.889823 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46956 (request call id 3053) took 1291 ms. Trace:
I20260709 13:14:55.854521 30530 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:55.896325 22906 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:55.904059 22833 rpcz_store.cc:276] 0709 13:14:53.569464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.569681 (+   217us) service_pool.cc:224] Handling call
0709 13:14:53.580267 (+ 10586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.583554 (+  3287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:53.583590 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:53.584220 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:14:53.743566 (+159346us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:53.915992 (+172426us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.926497 (+ 10505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.926588 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.555733 (+1629145us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:55.869007 (+313274us) raft_consensus.cc:1685] finished
0709 13:14:55.869118 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.869277 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28621,"prepare.queue_time_us":82737,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":14224,"raft.run_cpu_time_us":485,"raft.run_wall_time_us":39883,"spinlock_wait_cycles":12424960,"thread_start_us":287570,"threads_started":2,"wal-append.queue_time_us":459752}
I20260709 13:14:55.804399 30724 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:14:55.943676 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.816741 30725 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:14:55.867815 22908 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 16: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:55.872624 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:55.944416 22906 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:55.944744 22761 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:55.877977 30702 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:55.882637 30432 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.901448 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:55.910027 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:55.910257 22762 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:55.910949 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:55.951584 22760 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
W20260709 13:14:55.942293 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 77573.5us
I20260709 13:14:55.899821 30655 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:55.925330 22909 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
W20260709 13:14:55.884220 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 116915us
I20260709 13:14:55.928380 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:55.851073 30234 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:14:55.883677 30310 raft_consensus.cc:3055] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:55.945356 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:55.953926 22766 raft_consensus.cc:2376] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24. Re-sending same reply.
W20260709 13:14:55.949059 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 127409us
I20260709 13:14:55.945489 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:55.956235 22907 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:55.951365 22915 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:14:55.952900 22912 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:14:55.960126 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.961321 22909 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:55.961259 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:55.963297 22909 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 26. Current term is 27. Ops: [26.10-26.10]
I20260709 13:14:55.955451 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:55.924826 30633 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:55.969065 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:55.969849 22766 raft_consensus.cc:2376] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23. Re-sending same reply.
I20260709 13:14:55.947031 22759 rpcz_store.cc:276] 0709 13:14:54.598272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.598891 (+   619us) service_pool.cc:224] Handling call
0709 13:14:55.889505 (+1290614us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400369554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:55.889729 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434898560}
I20260709 13:14:55.969820 30304 raft_consensus.cc:697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:55.977380 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:55.948292 30658 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.945591 30657 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:55.928124 30718 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:55.858248 30703 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.985956 23465 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:55.856434 30723 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.111s
I20260709 13:14:55.999559 22879 leader_election.cc:400] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:56.005244 22879 leader_election.cc:403] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:55.959240 30310 raft_consensus.cc:740] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:55.974879 30655 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:55.971547 30611 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:55.944027 30551 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:55.977263 30530 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:56.034241 22907 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
W20260709 13:14:56.034714 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49298.6us
I20260709 13:14:55.984227 30660 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:55.984660 30544 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:55.984907 30234 consensus_queue.cc:1243] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:55.988556 30718 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:56.043030 22915 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:14:55.974959 30432 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
W20260709 13:14:56.053442 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.006240 30642 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 23.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:56.037777 30655 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:56.030510 30618 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:56.006047 30276 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:14:56.041913 30544 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:56.006932 30657 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.077247 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:56.108095 30655 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:56.030133 30297 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.065438 30729 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:56.074070 30660 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.122690 30729 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:56.006778 30567 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:55.990319 30658 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.031970 30296 consensus_queue.cc:237] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:56.134673 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 266145us
I20260709 13:14:56.065016 30344 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:56.130661 30729 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:56.107216 30461 consensus_queue.cc:1059] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:14:56.114395 30544 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:56.066370 30739 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:56.108647 30618 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:56.070293 30732 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:56.144176 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 144912us
W20260709 13:14:56.147634 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 219351us
I20260709 13:14:56.049044 30611 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.066103 30728 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.200s
W20260709 13:14:56.144922 30719 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:14:56.139760 30717 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:56.142554 30455 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:56.063941 30304 consensus_queue.cc:237] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.128015 30293 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:14:56.070178 30530 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.179478 30657 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:56.180037 22761 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 16.
I20260709 13:14:56.156154 30732 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:56.166924 30455 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:56.189849 30658 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.180037 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.181577 30618 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:56.215229 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 138071us
I20260709 13:14:56.222157 22761 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:56.075258 30718 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.225071 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.226677 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.233588 30748 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:56.222436 30732 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:56.224327 30455 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:56.232059 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 52133.7us
W20260709 13:14:56.242396 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69097.3us
I20260709 13:14:56.237682 30344 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:56.244200 30710 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.000s
W20260709 13:14:56.237476 30643 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.729s	user 0.010s	sys 0.000s
I20260709 13:14:56.242152 30718 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:56.239023 30297 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.238076 30703 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.237977 30723 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:56.265973 22764 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:14:56.239318 30551 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:56.268837 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 71504us
W20260709 13:14:56.281430 22656 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 247) took 79154 ms (client timeout 15000 ms). Trace:
I20260709 13:14:56.260906 30618 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:56.270816 30653 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.272071 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:56.284427 22656 rpcz_store.cc:269] 0709 13:13:37.131506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.142830 (+ 11324us) service_pool.cc:224] Handling call
0709 13:14:56.281108 (+79138278us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:43.472728 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:43.474504 (+  1776us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:43.493605 (+ 19101us) write_op.cc:435] Acquired schema lock
0709 13:14:43.496841 (+  3236us) tablet.cc:662] Decoding operations
0709 13:14:43.497994 (+  1153us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:43.498109 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:43.498150 (+    41us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:14:43.498893 (+   743us) tablet.cc:701] Row locks acquired
0709 13:14:43.498937 (+    44us) write_op.cc:260] PREPARE: finished
0709 13:14:43.499163 (+   226us) write_op.cc:270] Start()
0709 13:14:43.499326 (+   163us) write_op.cc:276] Timestamp: P: 1783602883499081 usec, L: 0
0709 13:14:43.499365 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:14:43.578992 (+ 79627us) log.cc:844] Serialized 2828 byte log entry
0709 13:14:44.123888 (+544896us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b
0709 13:14:44.887648 (+763760us) op_driver.cc:464] REPLICATION: finished
0709 13:14:45.202295 (+314647us) write_op.cc:301] APPLY: starting
0709 13:14:45.202497 (+   202us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:45.349584 (+147087us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:45.349643 (+    59us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:45.382957 (+ 33314us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:45.447833 (+ 64876us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:14:45.447910 (+    77us) write_op.cc:312] APPLY: finished
0709 13:14:45.448334 (+   424us) log.cc:844] Serialized 73 byte log entry
0709 13:14:45.448573 (+   239us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:45.448719 (+   146us) write_op.cc:454] Released schema lock
0709 13:14:45.448783 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:45.449266 (+   483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:45.449294 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":554376,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":248567,"apply.run_cpu_time_us":16797,"apply.run_wall_time_us":269949,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33401,"num_ops":7,"prepare.queue_time_us":673165,"prepare.run_cpu_time_us":44566,"prepare.run_wall_time_us":1087841,"raft.queue_time_us":406786,"raft.run_cpu_time_us":27220,"raft.run_wall_time_us":363414,"replication_time_us":1356110,"spinlock_wait_cycles":26516864,"thread_start_us":1379599,"threads_started":4,"wal-append.queue_time_us":614850,"wal-append.run_cpu_time_us":27573,"wal-append.run_wall_time_us":2167628}]]}
I20260709 13:14:56.273743 30739 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20.
I20260709 13:14:56.238392 30743 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:56.240468 30611 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.255995 30748 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:14:56.265834 30710 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.279127 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.307874 22761 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:56.281925 22655 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 11 to 18, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 18 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.283803 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:56.269323 30690 consensus_meta.cc:287] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.000s
I20260709 13:14:56.220989 30458 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.295663 30508 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:56.294329 30657 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.302098 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1367) took 1283 ms. Trace:
I20260709 13:14:56.313686 22841 rpcz_store.cc:276] 0709 13:14:55.018895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.112749 (+ 93854us) service_pool.cc:224] Handling call
0709 13:14:55.113359 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.474971 (+361612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.298294 (+823323us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b5400411ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:56.301915 (+  3621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830185856}
W20260709 13:14:56.317957 30708 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.001s
I20260709 13:14:56.318447 30708 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.320679 30708 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.321273 30739 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:56.310853 30643 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:56.315825 30690 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.324681 30690 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.325423 22911 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:56.290172 30530 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:56.333809 22655 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.339758 30455 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.340996 30544 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:56.342495 22732 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.540s (SENT)
I20260709 13:14:56.350116 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.327409 30657 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.348387 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.357724 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.358381 22761 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
W20260709 13:14:56.361472 22732 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.620s (SENT)
I20260709 13:14:56.357942 30728 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:56.355726 30544 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.365623 22655 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 16, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 16 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:56.369944 30455 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.363113 30710 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:56.370395 22656 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 139) took 24617 ms (client timeout 15000 ms). Trace:
I20260709 13:14:56.365463 30690 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:14:56.382635 22656 rpcz_store.cc:269] 0709 13:14:31.752638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.370098 (+24617460us) service_pool.cc:211] Skipping call since client already timed out
0709 13:14:56.370274 (+   176us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:14:56.371519 30717 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:56.390264 30458 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:56.376629 30512 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.360075 30657 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:56.392302 22732 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.650s (SENT)
I20260709 13:14:56.395092 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.395795 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.372726 30729 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.396649 22761 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:56.397722 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.398438 30729 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.398614 22757 raft_consensus.cc:2393] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:56.400578 30634 consensus_queue.cc:1059] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:56.402163 22655 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.410127 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.411223 22761 raft_consensus.cc:2393] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:56.411546 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.349578 30530 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.412964 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.415570 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.414923 22655 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 9 to 15, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.417688 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.419034 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.419919 22760 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:56.421100 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.422135 22757 raft_consensus.cc:2393] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:56.426347 30644 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.424106 22655 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 11 to 15, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 15 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.432869 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.435025 22765 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 16: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:56.440476 22655 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 17, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.443807 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.379968 30655 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.436940 30458 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:56.446086 30726 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:56.448624 30710 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:56.449038 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.444689 30717 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:56.447090 30690 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:14:56.450382 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.446468 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:56.452405 22757 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:56.454569 22655 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:14:56.476841 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 1093) took 2744 ms (client timeout 1909 ms). Trace:
W20260709 13:14:56.477557 23462 rpcz_store.cc:269] 0709 13:14:53.713179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.713371 (+   192us) service_pool.cc:224] Handling call
0709 13:14:56.457164 (+2743793us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b540068ef54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:56.457406 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917497216}
I20260709 13:14:56.482250 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.482077 22655 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.484440 22766 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:56.486851 22655 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.488082 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.488549 22655 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 16 to 20, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 20 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:14:56.489989 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.490206 22655 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.493888 22655 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 10 to 16, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 16 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.484211 30739 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:56.500429 30739 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.500861 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.505304 30530 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:56.496819 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:56.497709 30759 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:14:56.499400 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.500877 30739 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:56.510236 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:56.510496 30732 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.511183 22910 raft_consensus.cc:2376] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20. Re-sending same reply.
I20260709 13:14:56.511897 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2518) took 1120 ms. Trace:
I20260709 13:14:56.512292 22910 rpcz_store.cc:276] 0709 13:14:55.391552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.509850 (+1118298us) service_pool.cc:224] Handling call
0709 13:14:56.511815 (+  1965us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.513026 30732 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.499707 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:56.509136 30642 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:14:56.514762 23462 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:56.514783 30642 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:56.518321 30642 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 26.10, Last appended by leader: 10, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.520191 30642 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:56.523002 22760 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:14:56.523588 22760 pending_rounds.cc:85] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Aborting all ops after (but not including) 10
I20260709 13:14:56.523932 22760 pending_rounds.cc:107] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:14:56.524267 22760 raft_consensus.cc:2889] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:56.532572 30655 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.547881 30567 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:56.564674 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.565083 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:56.565929 23466 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:14:56.542255 22655 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 16, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 16 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:56.551478 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.553975 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1370) took 1525 ms. Trace:
I20260709 13:14:56.567122 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.567598 22839 rpcz_store.cc:276] 0709 13:14:55.028779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.169838 (+141059us) service_pool.cc:224] Handling call
0709 13:14:55.290980 (+121142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.634731 (+343751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.553708 (+918977us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400321ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:56.553903 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729021184}
W20260709 13:14:56.568459 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 67230us
I20260709 13:14:56.569912 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:56.555087 22732 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.507s (SENT)
I20260709 13:14:56.571548 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:56.572290 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.573352 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.575712 30618 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.576139 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.576534 30618 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.577595 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:56.577840 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.578537 30544 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:56.578783 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.578979 30544 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.579380 30544 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:56.579581 30618 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:56.585923 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.574987 22655 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.586704 23459 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:56.587488 22655 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 18, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 18 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.566778 30655 consensus_queue.cc:1059] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:14:56.590003 30707 raft_consensus.cc:3055] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:56.589447 22655 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 15, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 15 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.590432 30707 raft_consensus.cc:740] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:56.592013 30707 consensus_queue.cc:260] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.572504 30530 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:56.593466 30707 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:56.589708 30732 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.592089 22655 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.595491 30732 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.596257 22655 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 12 to 16, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 16 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.598104 22655 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 8 to 13, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 13 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.599543 22655 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 18, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 18 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.579382 22760 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:14:56.604343 30714 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.867s	user 0.001s	sys 0.001s
I20260709 13:14:56.607265 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.610222 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.611413 22655 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 11 to 18, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 18 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.624090 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:56.615682 30711 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:56.617484 30748 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:56.618120 30455 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:56.618548 30732 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:56.624934 22838 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:56.638727 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 83685.7us
I20260709 13:14:56.640554 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:56.652217 22732 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.526s (SENT)
I20260709 13:14:56.653117 22732 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:56.640699 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 70809.9us
I20260709 13:14:56.643079 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.655032 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:56.656180 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:56.657150 22840 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 27 because replica has last-logged OpId of term: 26 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 9.
I20260709 13:14:56.658699 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.659550 22839 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:56.654762 22655 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.662560 22760 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:14:56.664815 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:56.673452 22732 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.609s (SENT)
I20260709 13:14:56.674270 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.664289 22655 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 12 to 16, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 16 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.638399 30711 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:56.649801 30642 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:56.661077 30530 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:56.688516 22732 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.693s (SENT)
I20260709 13:14:56.690016 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.690384 22655 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 18, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 18 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.663502 30618 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:56.685647 30713 consensus_meta.cc:287] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.881s	user 0.014s	sys 0.004s
I20260709 13:14:56.696889 22655 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 15 to 19, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 19 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.702132 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.685492 30311 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:56.703042 22760 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:56.707460 22732 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.695s (SENT)
W20260709 13:14:56.708635 22732 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.705s (SENT)
I20260709 13:14:56.691783 30725 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:56.711758 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.712736 22758 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:56.717430 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.721472 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.731964 22655 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 7 to 15, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.733606 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.734062 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.734753 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.735304 22834 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
W20260709 13:14:56.735973 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.724189 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:56.737172 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:14:56.725018 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:56.738070 23461 raft_consensus.cc:2410] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:14:56.725822 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69137.3us
I20260709 13:14:56.726177 23465 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:56.726032 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
W20260709 13:14:56.726869 30771 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:14:56.727226 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.726907 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:56.730147 22732 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.749s (SENT)
I20260709 13:14:56.718421 22837 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:14:56.740243 22911 raft_consensus.cc:606] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.741607 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:56.746215 22911 raft_consensus.cc:614] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:56.746512 23462 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:56.747854 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.748594 22758 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:56.749918 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.750661 22758 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:56.751462 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.754001 23460 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:14:56.702831 30691 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:56.758810 22760 raft_consensus.cc:2468] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:14:56.754216 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.756925 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.760788 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.761360 22841 raft_consensus.cc:2393] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:56.762835 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46723.3us
I20260709 13:14:56.764175 22655 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 19, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.766826 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:56.768025 22732 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.840s (SENT)
I20260709 13:14:56.768102 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.768612 22836 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:14:56.768740 22732 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.775740 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.776513 22833 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:56.776787 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.777374 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:56.778008 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:56.778576 22837 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:56.778748 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 27 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.779651 22833 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 27 because replica has last-logged OpId of term: 26 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 9.
I20260709 13:14:56.780768 22833 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:56.786168 22762 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:56.755865 30642 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:14:56.793556 30725 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:56.794912 30544 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.797863 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:56.763706 30710 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:56.795492 30541 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:56.810603 22838 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:56.829103 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61124.3us
W20260709 13:14:56.832557 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 78435us
W20260709 13:14:56.832999 22732 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Timed out: RequestConsensusVote RPC to 127.21.33.3:35233 timed out after 1.889s (SENT)
I20260709 13:14:56.806811 23459 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 29.
I20260709 13:14:56.835029 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:56.834012 22655 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 12 to 16, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 16 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.741170 30304 raft_consensus.cc:2764] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:14:56.753073 30703 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.841272 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:56.841272 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.798867 30544 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:56.755019 30711 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.701330 30713 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.842070 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:56.849953 22655 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 4 to 17, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:14:56.857586 22837 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:56.867100 22760 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:56.862775 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:56.868611 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 104762us
I20260709 13:14:56.804566 30729 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 13:14:56.842053 22838 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:14:56.869850 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 130173us
I20260709 13:14:56.871665 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:56.872506 22833 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:56.837314 30653 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:56.869467 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.874881 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:56.851255 30730 log.cc:927] Time spent T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.786s	user 0.003s	sys 0.000s
I20260709 13:14:56.805179 30541 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:56.878983 22655 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 8 to 18, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 18 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.883219 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:56.883873 30731 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:56.805218 30618 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:14:56.860677 30567 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:56.799770 30739 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.769913 30691 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:56.906872 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.914918 22655 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:56.928692 22655 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 9 to 13, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 13 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:56.930012 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 60648us
I20260709 13:14:56.939918 30362 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:56.943480 30310 consensus_queue.cc:260] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.946924 30432 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:56.948138 30407 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:56.862088 30703 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:56.953415 22911 raft_consensus.cc:1275] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:56.954281 22910 raft_consensus.cc:1240] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 22. Current term is 23. Ops: []
W20260709 13:14:56.955565 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.956915 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2512) took 1622 ms. Trace:
I20260709 13:14:56.980481 22913 rpcz_store.cc:276] 0709 13:14:55.334644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.599121 (+264477us) service_pool.cc:224] Handling call
0709 13:14:56.956647 (+1357526us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540010e5d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:56.956843 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2322886528}
I20260709 13:14:56.973688 22909 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:14:56.984508 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.969103 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58100 (request call id 1197) took 1103 ms. Trace:
I20260709 13:14:56.986251 22764 rpcz_store.cc:276] 0709 13:14:55.865708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.865844 (+   136us) service_pool.cc:224] Handling call
0709 13:14:56.968788 (+1102944us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54001004d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:56.969012 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822561024}
I20260709 13:14:56.989627 22764 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:56.992516 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:14:56.994098 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.998875 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.000113 22907 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 18.
I20260709 13:14:56.997155 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:56.881783 30718 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:56.992970 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:56.893905 30711 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:57.007405 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:56.894529 30653 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:56.900115 30731 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:56.906566 30567 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.992194 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:14:57.020565 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.020143 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:56.898172 30634 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:57.036258 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:57.028399 22760 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:57.033844 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:57.043001 22907 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:57.033905 22757 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 16.
I20260709 13:14:57.031904 30458 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:56.974931 30691 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:57.043759 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:57.055539 22913 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:56.962443 30703 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:57.044927 22906 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:14:57.057096 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:57.057818 22906 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:57.045914 22765 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:57.043056 22805 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:56.993870 30234 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:56.991076 30611 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:57.018620 30310 raft_consensus.cc:3060] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:57.033140 30772 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.306s	user 0.000s	sys 0.003s
W20260709 13:14:57.052035 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59944.4us
I20260709 13:14:57.052157 22655 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 9 to 17, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:14:57.052922 30732 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:57.019976 30557 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:56.957559 30636 consensus_queue.cc:1059] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 13:14:57.004280 30718 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.061057 22914 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:57.050945 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 144196us
I20260709 13:14:57.062340 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:57.073071 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:57.039882 30731 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:57.041493 30567 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:57.007558 30723 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:14:57.118996 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.121860 22906 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:57.133940 22907 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:57.140170 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.141109 22907 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:57.090791 22880 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:57.147804 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 279236us
I20260709 13:14:57.150257 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:57.153587 22906 raft_consensus.cc:2376] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21. Re-sending same reply.
I20260709 13:14:57.154856 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.157372 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.159909 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:57.164659 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.076108 30461 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:14:57.169281 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 78629.9us
I20260709 13:14:57.190589 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.192373 22880 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.169965 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:57.170094 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.181175 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.177018 22913 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 29.
I20260709 13:14:57.074962 30671 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:57.225284 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:57.079804 30718 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:57.080294 30725 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:57.199481 30541 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.200964 30239 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:57.060029 30634 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:57.130295 30705 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:57.188082 30691 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:57.214512 22913 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:57.215147 22906 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:14:57.065236 30618 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.160s
I20260709 13:14:57.128530 30729 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:57.263165 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:57.075574 30690 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:57.074045 30458 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:57.168614 30732 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:57.168149 30724 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:57.278360 30691 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:57.290751 30691 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 23.9, Last appended by leader: 9, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.256891 30682 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:57.293385 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:57.261775 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101945us
I20260709 13:14:57.088142 30455 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.160s
I20260709 13:14:57.273670 22757 raft_consensus.cc:3060] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:57.243088 30772 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:14:57.186501 30311 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:57.276275 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.277040 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.277645 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:57.316298 22906 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:57.079303 30611 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:57.294931 22761 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:57.256016 30432 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:57.257416 30705 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.243506 30530 consensus_queue.cc:1059] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:14:57.305682 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6"
dest_uuid: "6545453e74344bb2957dd3729c5b3870"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:14:57.307444 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:57.310967 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:57.274919 30239 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:57.289184 30541 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:57.315783 22833 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:14:57.293426 30713 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:57.337236 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 310752us
W20260709 13:14:57.337392 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 435561us
I20260709 13:14:57.287127 30608 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 13:14:57.354591 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.361228 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.353319 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:57.364091 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:57.355674 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 266102us
I20260709 13:14:57.367197 22805 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:57.369460 22805 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:57.383448 22833 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:57.332069 30705 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:57.387347 22835 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:14:57.271539 30653 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:57.393932 22835 pending_rounds.cc:85] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 9
I20260709 13:14:57.397302 22835 pending_rounds.cc:107] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:14:57.397666 22835 raft_consensus.cc:2889] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.345192 30239 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:57.362854 30707 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.769s	user 0.006s	sys 0.007s
I20260709 13:14:57.405606 22757 raft_consensus.cc:2468] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
W20260709 13:14:57.411346 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 91559.7us
I20260709 13:14:57.412961 22732 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.385457 30725 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:57.438896 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2522) took 2046 ms. Trace:
W20260709 13:14:57.440451 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 402867us
I20260709 13:14:57.440845 22912 rpcz_store.cc:276] 0709 13:14:55.392636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.622876 (+230240us) service_pool.cc:224] Handling call
0709 13:14:55.952568 (+329692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.957134 (+  4566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.438545 (+1481411us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54003c8cd0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:57.438790 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757204736}
I20260709 13:14:57.343580 30311 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:57.388957 30706 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:57.451489 22836 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.016s	sys 0.011s
I20260709 13:14:57.398943 30703 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:57.460014 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.408052 30239 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:57.459638 22836 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.017s	sys 0.011s
I20260709 13:14:57.460105 22832 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:57.388846 30432 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:14:57.389088 30557 consensus_queue.cc:1059] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:14:57.450944 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:57.399258 30530 raft_consensus.cc:3055] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:14:57.501824 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 88985.6us
W20260709 13:14:57.495378 22759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.019s
I20260709 13:14:57.502526 30734 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:57.509208 22732 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.514675 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.450546 30455 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader election won for term 29
W20260709 13:14:57.509223 22759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.002s	sys 0.022s
I20260709 13:14:57.478899 22906 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:57.519014 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.526165 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.528725 22906 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
W20260709 13:14:57.530146 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 343012us
I20260709 13:14:57.457692 30706 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:57.536619 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:57.506933 30725 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:14:57.542462 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.516462 30682 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:57.458667 30707 consensus_queue.cc:1059] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
W20260709 13:14:57.540122 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89280.8us
I20260709 13:14:57.531445 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:57.511955 30530 raft_consensus.cc:740] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:57.555140 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.558254 30729 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:14:57.543802 30748 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.882s	user 0.007s	sys 0.007s
I20260709 13:14:57.460093 30461 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:57.571167 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.574982 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.569406 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:57.575994 22837 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 13:14:57.537078 30455 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 29 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:57.545084 30706 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:57.579674 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:57.571224 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.575307 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.586889 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 394615us
W20260709 13:14:57.664896 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74090.8us
I20260709 13:14:57.676433 22880 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.677815 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:57.666328 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 129781us
I20260709 13:14:57.667025 22837 raft_consensus.cc:1240] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 21. Ops: []
I20260709 13:14:57.678277 22914 raft_consensus.cc:2468] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:14:57.591384 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.569747 30734 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:57.578615 30710 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:57.720180 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126423us
I20260709 13:14:57.579047 30729 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:57.692317 30513 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:57.698717 30634 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:57.668251 30557 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:57.743834 30557 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:57.744196 30781 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:14:57.728437 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 48438.5us
I20260709 13:14:57.745299 22732 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.747380 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:57.747633 22759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58100 (request call id 1207) took 1692 ms. Trace:
W20260709 13:14:57.748759 30567 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:14:57.748822 30682 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:57.719300 30461 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.707374 30432 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:57.744311 30557 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:57.591336 30748 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:57.732198 30793 raft_consensus.cc:2804] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:57.588678 30788 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:57.779721 22913 raft_consensus.cc:606] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:57.861265 22913 raft_consensus.cc:614] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:57.749099 30784 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:57.801112 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:57.757321 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:57.768687 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.594771 30721 raft_consensus.cc:3055] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:57.783504 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.804252 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:57.748183 22759 rpcz_store.cc:276] 0709 13:14:56.055457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.055588 (+   131us) service_pool.cc:224] Handling call
0709 13:14:56.074364 (+ 18776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.074778 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.074813 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.974910 (+900097us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b2c000582e8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:56.975404 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.517951 (+542547us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:57.518039 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.518161 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.747249 (+229088us) raft_consensus.cc:1685] finished
0709 13:14:57.747281 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.747551 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47754,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"spinlock_wait_cycles":14621056,"thread_start_us":541930,"threads_started":1,"wal-append.queue_time_us":747991}
I20260709 13:14:57.721592 30710 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.956331 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:57.888252 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.685308 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 94031.1us
I20260709 13:14:57.975952 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:57.930105 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:57.970638 22832 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
W20260709 13:14:58.048774 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.745302 30691 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:57.969376 30784 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.049610 30784 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:58.049816 30721 raft_consensus.cc:740] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:58.050830 30721 consensus_queue.cc:260] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.051944 30721 raft_consensus.cc:3060] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:57.759313 30682 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:57.860368 30690 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:57.771122 30787 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.055030 30787 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.733675 30611 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:57.841979 30707 consensus_queue.cc:1059] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.216s
I20260709 13:14:58.061076 30784 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.741120 30530 consensus_queue.cc:260] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.065222 30530 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:58.066645 30784 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:57.749426 30567 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:58.062007 30407 consensus_meta.cc:287] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.113s	user 0.016s	sys 0.000s
W20260709 13:14:57.863878 30310 consensus_meta.cc:287] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.785s	user 0.008s	sys 0.006s
I20260709 13:14:57.763252 30455 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:57.930963 22833 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:58.091355 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.595554 30458 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:57.777741 30513 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:57.740567 30713 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.091787 30711 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 1.055s	user 0.001s	sys 0.012s
I20260709 13:14:58.103405 23464 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
W20260709 13:14:58.103816 22912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.026s	sys 0.083s
I20260709 13:14:58.113649 22841 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:57.888051 23462 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:14:58.105798 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 348544us
I20260709 13:14:57.755698 30321 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.338s
W20260709 13:14:57.929965 30776 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.472s	user 0.000s	sys 0.002s
W20260709 13:14:58.113623 22912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.026s	sys 0.083s
I20260709 13:14:58.115495 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:58.117295 22838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.038s	sys 0.102s
I20260709 13:14:57.770364 30311 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:58.136451 22838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.038s	sys 0.102s
I20260709 13:14:58.069428 30787 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:58.128134 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 440170us
I20260709 13:14:57.956727 30793 raft_consensus.cc:697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:58.136828 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.102943 30513 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:57.778780 30790 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.359s
I20260709 13:14:57.938444 30634 consensus_queue.cc:1059] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.178s
I20260709 13:14:58.116319 30682 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:58.112658 30788 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.147599 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.150146 22732 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.120443 30648 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.138963 30732 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.843s	user 0.010s	sys 0.003s
I20260709 13:14:58.148584 30784 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:58.154721 30701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.427s	user 0.010s	sys 0.025s
I20260709 13:14:58.116667 30713 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:57.759605 30725 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.158869 22758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3063) took 1585 ms. Trace:
I20260709 13:14:58.172564 22758 rpcz_store.cc:276] 0709 13:14:56.573294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.806727 (+233433us) service_pool.cc:224] Handling call
0709 13:14:56.807017 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.833868 (+ 26851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.833905 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.852423 (+ 18518us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.092031 (+239608us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:57.092100 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.092216 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.158558 (+1066342us) raft_consensus.cc:1685] finished
0709 13:14:58.158590 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.158788 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8004,"raft.queue_time_us":20428,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":20200,"thread_start_us":239096,"threads_started":1,"wal-append.queue_time_us":512843}
W20260709 13:14:58.161103 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 483354us
I20260709 13:14:58.155571 30682 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:58.166337 30701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.439s	user 0.011s	sys 0.025s
I20260709 13:14:58.154935 30784 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.158388 30732 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:58.204536 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.183321 30682 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:58.166415 30648 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.185272 22832 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c"
dest_uuid: "776baf137f8745dd866379ca15ab8cee"
mode: GRACEFUL
new_leader_uuid: "acfda9b350734def9b6c621f66a5f247"
 from {username='slave'} at 127.0.0.1:55944
I20260709 13:14:58.151226 30703 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:14:58.186648 30784 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:58.175220 30458 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.187541 30239 consensus_meta.cc:287] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.001s
W20260709 13:14:58.189352 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45598.6us
I20260709 13:14:58.189721 30691 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.190299 30796 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:14:58.192444 30769 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.465s	user 0.005s	sys 0.000s
W20260709 13:14:58.194093 30797 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:14:58.194372 30793 consensus_queue.cc:237] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:58.195377 30789 log.cc:927] Time spent T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 13:14:58.159067 30788 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.199777 30713 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.148768 30702 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 13:14:58.172214 30541 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.004s
I20260709 13:14:58.186260 30701 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:58.208123 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.244704 22912 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 13:14:58.208352 22832 raft_consensus.cc:606] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Received request to transfer leadership to TS acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.172854 30344 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:58.246114 30321 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.211187 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.246699 30321 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.248891 30321 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.183296 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:14:58.212610 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 316859us
I20260709 13:14:58.217234 22914 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:14:58.210629 30705 consensus_meta.cc:287] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.800s	user 0.015s	sys 0.000s
I20260709 13:14:58.226468 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:58.229137 23431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.026s
W20260709 13:14:58.228554 22881 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.012s	sys 0.025s
I20260709 13:14:58.241595 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:58.252604 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 465137us
W20260709 13:14:58.253960 23431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.009s	sys 0.026s
I20260709 13:14:58.254262 22913 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 29: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:58.254388 22881 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.013s	sys 0.025s
I20260709 13:14:58.255841 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.256105 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.256662 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.258075 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.258919 22912 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:58.261452 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.262041 30567 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.262552 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:58.265026 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81876.1us
I20260709 13:14:58.267179 23459 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:58.272053 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.216075 30239 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.218775 30691 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.271328 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.277918 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.275838 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.269459 22879 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:58.278875 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.281656 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.284693 22913 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:58.244009 30701 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:58.288051 22914 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.284637 23458 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:58.220355 30458 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.291131 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.291961 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:58.294819 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:58.262940 30321 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:58.278275 30718 consensus_meta.cc:287] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.011s	user 0.015s	sys 0.000s
I20260709 13:14:58.278573 30567 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:58.315645 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:58.293309 30636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.125s	user 0.021s	sys 0.042s
I20260709 13:14:58.311051 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:58.273634 30713 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.320117 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.325645 23459 raft_consensus.cc:2368] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:14:58.328006 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.292665 30787 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.346866 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.265058 30705 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.295423 30344 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:14:58.324504 30636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.022s	sys 0.043s
I20260709 13:14:58.311565 30432 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:58.352485 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.353688 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.324690 30567 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.324555 30710 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.358568 23459 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 25: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:58.306524 30734 consensus_queue.cc:1059] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:14:58.346026 30653 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.374634 22834 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 26: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:58.368579 22912 raft_consensus.cc:2468] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:14:58.350791 30702 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:58.346794 30711 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:14:58.381031 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 86307.6us
I20260709 13:14:58.353094 30807 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:58.357124 30634 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.352422 30732 raft_consensus.cc:2804] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:58.359866 30432 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:58.376961 30567 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:58.334977 30713 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.327061 30718 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.392273 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.392970 22912 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:58.352970 30787 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:58.392407 30799 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:14:58.387979 30702 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:58.387418 30807 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.391141 30634 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.409600 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:58.407601 22732 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.382335 30710 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:14:58.368350 30804 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:14:58.395591 30432 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:58.420574 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2889) took 1561 ms. Trace:
W20260709 13:14:58.423259 23430 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.2:41075 timed out after 1.625s (SENT) [suppressed 66 similar messages]
I20260709 13:14:58.387234 30711 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.429966 22840 rpcz_store.cc:276] 0709 13:14:56.858938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.859102 (+   164us) service_pool.cc:224] Handling call
0709 13:14:56.871547 (+ 12445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:56.872001 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:57.346433 (+474432us) log.cc:844] Serialized 14 byte log entry
0709 13:14:58.129096 (+782663us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:58.142986 (+ 13890us) log.cc:844] Serialized 52 byte log entry
0709 13:14:58.143194 (+   208us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:58.143531 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:14:58.144168 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.144491 (+   323us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.420239 (+275748us) raft_consensus.cc:1685] finished
0709 13:14:58.420263 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.420488 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54052,"prepare.queue_time_us":568722,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":66041,"raft.run_cpu_time_us":471,"raft.run_wall_time_us":9394,"spinlock_wait_cycles":23107968,"thread_start_us":1148468,"threads_started":3,"wal-append.queue_time_us":583253}
I20260709 13:14:58.406710 30321 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:58.378394 30701 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.437889 30239 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.446594 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.438046 30513 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.452450 30513 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.441232 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:58.404155 30732 raft_consensus.cc:697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:58.385990 30653 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:58.437273 30731 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 1.269s	user 0.011s	sys 0.000s
I20260709 13:14:58.449194 30458 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.439851 30634 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.424820 30807 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:58.429363 30734 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:58.455410 30701 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:58.482074 30787 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.465160 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.471678 30788 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.472268 30732 consensus_queue.cc:237] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.474227 30718 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.474416 30711 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.474920 30728 raft_consensus.cc:3055] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:58.510473 30728 raft_consensus.cc:740] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:58.510950 30321 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:58.475452 30691 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.479012 30653 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.513134 30734 raft_consensus.cc:2804] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:58.511550 30321 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.513628 30734 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.514014 30734 raft_consensus.cc:3060] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:58.515223 30321 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.475236 30705 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.479571 30702 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.468796 30731 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.518719 30690 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.519328 30690 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:58.520406 30731 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:58.480635 30811 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:14:58.480087 30817 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:14:58.500712 30713 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:58.470185 30802 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.161s	user 0.002s	sys 0.000s
W20260709 13:14:58.503010 30729 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.003s
W20260709 13:14:58.531702 30770 log.cc:927] Time spent T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.804s	user 0.001s	sys 0.007s
I20260709 13:14:58.462975 30344 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:14:58.533865 30648 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.466954 30805 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.536011 30805 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:58.537803 30810 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:14:58.538424 30805 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.517985 23463 raft_consensus.cc:2410] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:14:58.457155 30824 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:14:58.523365 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.526010 30731 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.511557 30728 consensus_queue.cc:260] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.531373 30729 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.506021 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3882) took 1551 ms. Trace:
I20260709 13:14:58.540673 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.479596 30682 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.547446 22758 raft_consensus.cc:1240] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:58.564819 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.546015 30728 raft_consensus.cc:3060] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:58.566576 30739 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.569597 22758 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:58.548611 22760 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:58.548813 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.549153 22766 raft_consensus.cc:1240] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:58.550406 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.550060 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.577943 22757 raft_consensus.cc:3060] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:58.550751 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.579406 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.579619 22763 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:58.551522 22761 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:58.582626 22880 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:58.554355 22762 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:14:58.586926 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.587478 22760 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:58.578184 22759 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:58.547818 22908 rpcz_store.cc:276] 0709 13:14:56.954605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.003927 (+ 49322us) service_pool.cc:224] Handling call
0709 13:14:57.040345 (+ 36418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:57.386349 (+346004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.386384 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:57.436907 (+ 50523us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.834175 (+397268us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:57.834233 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.834352 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.505768 (+671416us) raft_consensus.cc:1685] finished
0709 13:14:58.505795 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.505954 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32521,"raft.queue_time_us":32403,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":14472,"thread_start_us":652928,"threads_started":2,"wal-append.queue_time_us":682123}
I20260709 13:14:58.590452 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
W20260709 13:14:58.592980 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 238381us
I20260709 13:14:58.549611 30234 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:58.550604 30826 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:14:58.600894 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
W20260709 13:14:58.557412 30706 consensus_meta.cc:287] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.835s	user 0.011s	sys 0.002s
I20260709 13:14:58.549566 30824 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:58.606580 22760 raft_consensus.cc:2468] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:58.607596 22757 raft_consensus.cc:2468] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:58.549377 30304 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:58.547924 30718 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.552735 30682 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.547662 30729 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.630563 30832 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.631145 30832 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:58.632438 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.551669 30805 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.633775 30805 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.635866 30805 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.638374 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3874) took 2818 ms. Trace:
I20260709 13:14:58.638739 22909 rpcz_store.cc:276] 0709 13:14:55.819468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.972104 (+152636us) service_pool.cc:224] Handling call
0709 13:14:56.969989 (+997885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.985166 (+ 15177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.638147 (+1652981us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b540010e5d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:58.638310 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294160128}
I20260709 13:14:58.639410 30807 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.641522 30718 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.643582 30718 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:58.644917 30557 consensus_meta.cc:287] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.000s
I20260709 13:14:58.645689 30832 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.607868 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:58.647372 22911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43274 (request call id 3883) took 1656 ms (client timeout 1995 ms). Trace:
W20260709 13:14:58.647917 22911 rpcz_store.cc:269] 0709 13:14:56.990781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.992672 (+  1891us) service_pool.cc:224] Handling call
0709 13:14:58.647100 (+1654428us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003ca0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:58.647289 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2801958144}
I20260709 13:14:58.608132 30234 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.584961 30713 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:58.612100 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:58.649878 23458 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:58.649976 30713 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:58.544144 30784 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.651418 30234 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.652997 30784 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.612882 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.653874 22763 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:14:58.654156 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3067) took 1665 ms. Trace:
I20260709 13:14:58.654968 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 26 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.653378 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1379) took 1195 ms. Trace:
I20260709 13:14:58.655673 22763 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 27.
I20260709 13:14:58.655712 22835 rpcz_store.cc:276] 0709 13:14:57.458441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.464635 (+  6194us) service_pool.cc:224] Handling call
0709 13:14:58.653136 (+1188501us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b540028a0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:58.653276 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189171456}
I20260709 13:14:58.625694 30706 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.654637 22765 rpcz_store.cc:276] 0709 13:14:56.988770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.009788 (+ 21018us) service_pool.cc:224] Handling call
0709 13:14:57.045642 (+ 35854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.062163 (+ 16521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.653894 (+1591731us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b54003408d0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:58.654076 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2446720}
I20260709 13:14:58.658499 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.659081 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1375) took 1262 ms. Trace:
I20260709 13:14:58.659529 30706 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.659940 23466 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:58.659507 22833 rpcz_store.cc:276] 0709 13:14:57.396364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.410589 (+ 14225us) service_pool.cc:224] Handling call
0709 13:14:57.674858 (+264269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.102023 (+427165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.658826 (+556803us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:58.659022 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588376320}
I20260709 13:14:58.659473 22765 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 18: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.663482 22765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.589847 30739 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:58.671207 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 492567us
I20260709 13:14:58.671406 30739 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.589985 30793 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:58.672232 30793 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.672605 30793 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:58.672787 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.673540 22763 raft_consensus.cc:2376] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20. Re-sending same reply.
I20260709 13:14:58.549656 30731 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.679172 30703 raft_consensus.cc:3055] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:58.679751 30703 raft_consensus.cc:740] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:58.680423 30800 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.502s	user 0.001s	sys 0.001s
I20260709 13:14:58.680580 30731 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.682855 30748 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.683363 30748 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.684511 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2892) took 1443 ms. Trace:
I20260709 13:14:58.620767 22834 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1382) took 1066 ms. Trace:
I20260709 13:14:58.618906 22764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3066) took 1630 ms. Trace:
I20260709 13:14:58.591744 30567 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.685457 30748 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:58.691808 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.685699 22834 rpcz_store.cc:276] 0709 13:14:57.554370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.554752 (+   382us) service_pool.cc:224] Handling call
0709 13:14:58.620547 (+1065795us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54000c7bd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:58.620703 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852002688}
I20260709 13:14:58.685853 22764 rpcz_store.cc:276] 0709 13:14:56.988434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.988902 (+   468us) service_pool.cc:224] Handling call
0709 13:14:56.989400 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.003989 (+ 14589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.618549 (+1614560us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b5400051ca0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1a68dc0 00007f68a1af4a02 00007f68a1af0dd6 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:14:58.618829 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13322624}
I20260709 13:14:58.693732 30788 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.694238 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.694380 30788 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.695688 22764 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 22. Ops: []
I20260709 13:14:58.696321 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:58.696923 22764 raft_consensus.cc:1240] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:58.699908 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.701301 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:14:58.702035 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.702423 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:14:58.686462 23464 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1354) took 5057 ms (client timeout 2546 ms). Trace:
I20260709 13:14:58.703132 30544 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:58.703552 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.703927 30544 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.704939 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.705590 22758 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:58.707530 22758 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:58.708642 22758 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:14:58.645295 30557 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.611848 23462 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:58.664093 22766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.665606 30239 raft_consensus.cc:3055] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:58.723663 22911 raft_consensus.cc:1240] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:58.668536 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.724490 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.725170 22764 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:58.726821 30362 raft_consensus.cc:3055] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:58.727358 30362 raft_consensus.cc:740] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:58.727620 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59149.9us
I20260709 13:14:58.727843 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:58.728426 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.728364 30362 consensus_queue.cc:260] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.668903 30407 raft_consensus.cc:3055] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:58.729560 30785 raft_consensus.cc:3055] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:58.729655 30407 raft_consensus.cc:740] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:58.730062 30648 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:58.730615 30648 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.730667 30407 consensus_queue.cc:260] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.731855 30407 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:58.620009 30824 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.732643 30707 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:14:58.733729 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.670706 22761 raft_consensus.cc:1275] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:58.671885 30739 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:58.702628 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.736665 22762 raft_consensus.cc:2468] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:58.678303 22757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.686089 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2550) took 1504 ms. Trace:
I20260709 13:14:58.737998 22757 raft_consensus.cc:2393] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.737771 22907 rpcz_store.cc:276] 0709 13:14:57.182110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.367824 (+185714us) service_pool.cc:224] Handling call
0709 13:14:57.403315 (+ 35491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:57.413954 (+ 10639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.413989 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:57.414560 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.962464 (+547904us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:57.962534 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.962651 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.679950 (+717299us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b24002054a8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e88a9c4 00007f689e888acb 00007f689ea43dc9 00007f689ea43a55 00007f689ea4309d 00007f68a167510e 00007f68a168c88c 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221
0709 13:14:58.685689 (+  5739us) raft_consensus.cc:1685] finished
0709 13:14:58.685724 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.686011 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32158,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":11980,"spinlock_wait_cycles":37038848,"thread_start_us":496307,"threads_started":1,"wal-append.queue_time_us":690890}
I20260709 13:14:58.739492 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.689446 22913 raft_consensus.cc:2368] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17.
W20260709 13:14:58.714239 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.702059 23459 raft_consensus.cc:2393] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:58.684883 22841 rpcz_store.cc:276] 0709 13:14:57.240915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.241368 (+   453us) service_pool.cc:224] Handling call
0709 13:14:57.241882 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.114167 (+872285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.684254 (+570087us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b540003a2d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:58.684444 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1971858816}
I20260709 13:14:58.742722 30824 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:58.743120 30823 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.257s	user 0.000s	sys 0.002s
I20260709 13:14:58.743543 30311 consensus_queue.cc:1048] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:58.744709 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.745438 22833 raft_consensus.cc:3060] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:58.723788 30234 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.747313 30234 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.747201 30608 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:14:58.749658 22766 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:58.750365 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.750986 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.751613 22840 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:14:58.751005 22759 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
W20260709 13:14:58.716238 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.717751 23457 raft_consensus.cc:1275] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 13:14:58.688511 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:14:58.754379 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65983.9us
I20260709 13:14:58.753436 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.753886 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.719949 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.755666 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.756104 22909 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:58.668076 30633 raft_consensus.cc:3055] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:58.649394 30732 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:58.726903 30544 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.626658 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.670596 30706 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:58.730062 30785 raft_consensus.cc:740] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:58.778699 30732 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:14:58.779201 30732 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:58.779608 30785 consensus_queue.cc:260] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.781127 30785 raft_consensus.cc:3060] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:58.782240 22835 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:58.782795 30734 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.783578 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.784345 22835 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:58.785913 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.786556 22759 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:58.787387 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.788358 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.789012 22759 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.789628 30734 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.789814 30808 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.790351 30808 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.792580 30808 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.792421 30690 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:58.793742 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.795125 30831 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.175s	user 0.000s	sys 0.002s
W20260709 13:14:58.797623 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67976.4us
I20260709 13:14:58.798599 23432 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.799889 30815 raft_consensus.cc:2804] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:58.800566 30815 raft_consensus.cc:697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:58.697484 30788 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:58.801716 30815 consensus_queue.cc:237] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.802830 30432 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:58.805660 30432 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.806005 30362 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:58.813033 30807 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.688019 30567 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.785336 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.740777 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:58.754518 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.756467 23457 pending_rounds.cc:85] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting all ops after (but not including) 17
I20260709 13:14:58.756935 22834 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 26 index: 10. Preceding OpId from leader: term: 27 index: 10. (term mismatch)
I20260709 13:14:58.718268 22914 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:58.703374 23464 rpcz_store.cc:269] 0709 13:14:53.629098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.629994 (+   896us) service_pool.cc:224] Handling call
0709 13:14:58.686259 (+5056265us) spinlock_profiling.cc:243] Waited 4.45 s on lock 0x7b5400603454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:58.686366 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10243230592}
I20260709 13:14:58.757372 22758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.731216 30648 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:58.763986 22832 raft_consensus.cc:1240] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:58.764631 22837 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:58.764931 22760 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.767819 23458 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:58.758876 30633 raft_consensus.cc:740] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:58.735235 30731 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:58.769200 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.724027 30239 raft_consensus.cc:740] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:58.770002 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.770114 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.833102 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.836302 22759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.861267 22760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247" is_pre_election: true
I20260709 13:14:58.770627 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.861951 23462 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:14:58.861588 22838 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:14:58.859289 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.847251 22841 raft_consensus.cc:2393] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 26: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:58.847306 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 26 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.863535 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:14:58.771538 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.864425 22838 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:58.865034 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.847962 22834 pending_rounds.cc:85] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 9
I20260709 13:14:58.866017 22834 pending_rounds.cc:107] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 26.10
I20260709 13:14:58.866357 22834 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.771625 30707 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:58.771553 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.877913 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.764214 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.774909 30557 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.680821 30703 consensus_queue.cc:260] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.776976 30234 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:58.777643 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:14:58.778219 30721 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:58.777966 30706 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:58.767805 22766 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:58.820855 22765 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:14:58.823550 22835 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:58.832211 23466 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:58.833906 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:58.926605 23459 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:58.834978 23461 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:58.757907 30642 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.012s
W20260709 13:14:58.926649 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 149087us
I20260709 13:14:58.840824 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.932744 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.934320 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.935688 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.936378 22907 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:58.867522 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.846961 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.940047 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.940920 22915 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
I20260709 13:14:58.941987 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.942780 23462 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:58.944038 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:58.944747 30827 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:14:58.854916 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.945379 30827 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:58.945890 22837 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:58.947119 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.948411 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.860886 30731 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.981403 22757 raft_consensus.cc:1240] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:14:58.853752 22758 raft_consensus.cc:2393] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.830847 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.984537 22766 raft_consensus.cc:1240] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:58.985203 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.986385 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.986806 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.987004 22911 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:58.987800 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.847507 23457 pending_rounds.cc:107] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:14:58.836081 23465 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:58.854003 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 27 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.826046 22833 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:14:58.861974 22760 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:14:58.851336 22762 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:58.861706 22759 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:14:58.862164 22836 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:14:58.875176 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.947818 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.882431 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.998246 22840 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 24: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
W20260709 13:14:58.883968 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 119850us
I20260709 13:14:58.858357 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:58.999657 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:58.887686 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.895294 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:58.896618 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.898234 22838 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:58.853756 30832 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:58.830456 30710 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:59.007500 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 110962us
I20260709 13:14:58.855010 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:59.008903 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:59.009253 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.001180 22834 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:59.009924 22914 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:14:58.988296 22766 raft_consensus.cc:1240] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:58.988435 23457 raft_consensus.cc:2889] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.878666 22764 raft_consensus.cc:2468] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
W20260709 13:14:58.994246 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 62758.5us
I20260709 13:14:58.996191 22760 raft_consensus.cc:1240] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:58.997053 22759 raft_consensus.cc:1240] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:58.902900 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.830812 30690 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:58.982725 30541 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.926316 30794 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:14:59.034075 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 131318us
I20260709 13:14:59.034679 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:59.035246 22880 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:59.027238 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.036190 22880 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
W20260709 13:14:59.005066 30837 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:14:58.836014 30793 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.050532 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:58.993765 30731 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:59.018011 22758 raft_consensus.cc:1240] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:14:59.024747 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:59.025025 22761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acfda9b350734def9b6c621f66a5f247"
I20260709 13:14:59.025573 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:59.026687 22840 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:58.894421 30721 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:59.031172 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:58.897398 30567 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:58.897246 30808 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:58.817449 30739 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.042464 30805 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:59.077265 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 68947.7us
I20260709 13:14:58.947707 30827 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:59.078619 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:58.980904 30809 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:59.048694 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:58.949162 30608 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.861673 22839 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:59.054279 30691 raft_consensus.cc:2804] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:14:59.062120 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.054860 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:59.089788 22914 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:59.067539 22761 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:59.069537 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:59.069780 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.068853 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:14:59.071247 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44125.2us
I20260709 13:14:59.030910 30304 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:59.095494 22841 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:59.097224 22912 raft_consensus.cc:1240] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:59.099689 22762 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:14:59.086880 22911 raft_consensus.cc:2368] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21.
I20260709 13:14:59.102103 22907 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:14:59.079511 30706 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:14:59.108428 23430 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:58.949735 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.084491 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:59.113783 22806 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:14:59.053092 23459 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 26.
I20260709 13:14:59.054404 30653 raft_consensus.cc:2804] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:59.092696 22836 raft_consensus.cc:2393] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:14:58.949333 30671 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:14:59.086197 22879 leader_election.cc:400] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:14:59.115183 22879 leader_election.cc:403] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:59.098871 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:59.085032 30809 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.122756 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:14:59.125276 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 175657us
W20260709 13:14:59.127890 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.072256 30310 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:14:59.130743 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:59.043557 30710 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.136792 22913 raft_consensus.cc:2393] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:59.131325 22906 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:59.083232 30805 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:59.074947 30705 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:14:59.072750 30794 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.089993 30461 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:59.164582 30461 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.165058 30461 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:59.028092 30832 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.091671 30713 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:59.152220 23459 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:14:59.153117 30827 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:14:59.105284 30455 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.110927 30691 raft_consensus.cc:697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:59.133366 30814 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:59.089071 30608 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.158208 30310 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:59.136142 30544 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:59.160414 30710 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:59.140059 30805 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:59.140244 30721 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.140479 30653 raft_consensus.cc:697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:59.162806 30793 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:59.166445 22914 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:14:59.054746 30541 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:14:59.167248 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 122995us
I20260709 13:14:59.168128 22915 raft_consensus.cc:2393] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:14:59.053993 30530 consensus_queue.cc:1059] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:14:59.131129 30809 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:59.176448 22907 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 13:14:59.248629 22907 pending_rounds.cc:85] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 13
I20260709 13:14:59.248953 22907 pending_rounds.cc:107] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 13:14:59.249238 22907 raft_consensus.cc:2889] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:59.135246 30671 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.135694 30808 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:14:59.134260 30432 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:59.196534 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.195021 22834 raft_consensus.cc:2410] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:14:59.206611 22906 raft_consensus.cc:1240] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [19.16-21.17]
I20260709 13:14:59.199621 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.191936 30785 consensus_queue.cc:1059] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:14:59.216471 22761 raft_consensus.cc:2468] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:14:59.194927 30455 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.222853 30739 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:59.090077 30239 consensus_queue.cc:260] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.241911 22880 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:59.241902 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:59.195566 30310 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:59.248251 30544 raft_consensus.cc:697] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:14:59.296487 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 101682us
W20260709 13:14:59.299252 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57475.1us
I20260709 13:14:59.302862 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:59.284240 30808 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.306128 22832 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:59.306568 30808 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:59.311254 22907 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:14:59.314057 22907 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:59.319080 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77293.5us
I20260709 13:14:59.286468 30794 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:59.278503 30732 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.304041 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:59.465536 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 161538us
W20260709 13:14:59.466126 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46349us
I20260709 13:14:59.466362 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.300486 30734 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:59.405083 22906 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:59.416476 22835 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:14:59.416735 22832 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:14:59.478154 22880 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:59.456696 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 325252us
I20260709 13:14:59.395627 30705 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:59.318953 30691 consensus_queue.cc:237] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:59.417835 30843 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.138s	user 0.004s	sys 0.000s
W20260709 13:14:59.417572 30841 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.062s	user 0.001s	sys 0.002s
I20260709 13:14:59.404037 30728 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:59.404090 30793 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:59.484704 22840 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
W20260709 13:14:59.464509 30838 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:14:59.463454 30844 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.201s	user 0.004s	sys 0.000s
I20260709 13:14:59.355832 30653 consensus_queue.cc:237] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.484661 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:59.417484 30846 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.165s	user 0.000s	sys 0.004s
I20260709 13:14:59.467272 30725 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.478505 30702 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:59.548815 30728 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:14:59.493458 30739 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:59.488467 30713 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:14:59.482466 30710 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
I20260709 13:14:59.497014 30734 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:14:59.488291 30705 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:59.494871 30840 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:59.600421 30827 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:59.588333 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 452755us
W20260709 13:14:59.591427 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107516us
I20260709 13:14:59.491160 22807 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:14:59.595860 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 292901us
W20260709 13:14:59.598018 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 511983us
I20260709 13:14:59.610126 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:59.496021 30310 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.542606 30731 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.593595 30608 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:59.596236 30702 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.615866 22763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 3645) took 1062 ms. Trace:
I20260709 13:14:59.596748 30832 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:14:59.586916 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.601125 30541 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:59.623132 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:14:59.557627 30544 consensus_queue.cc:237] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.604856 30739 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:59.634110 23431 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.621063 30705 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:59.635730 22763 rpcz_store.cc:276] 0709 13:14:58.553687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.674273 (+120586us) service_pool.cc:224] Handling call
0709 13:14:58.674639 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.674991 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:58.675021 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:58.675524 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.087402 (+411878us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:59.087447 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.087521 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.615512 (+527991us) raft_consensus.cc:1685] finished
0709 13:14:59.615549 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.615778 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1159,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":411432,"threads_started":1,"wal-append.queue_time_us":520308}
I20260709 13:14:59.619930 30671 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:59.622539 30455 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:59.598256 30725 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.622211 30827 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:59.642164 30702 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:59.636025 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 169756us
I20260709 13:14:59.645390 30739 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:59.680583 23431 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:59.688135 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 197058us
W20260709 13:14:59.691298 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66679.1us
I20260709 13:14:59.663009 30310 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:59.627091 30633 consensus_queue.cc:260] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.692698 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:59.668692 30827 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:59.696306 22763 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:59.695369 22806 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:59.668601 30671 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:59.665119 30854 raft_consensus.cc:2804] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:14:59.706678 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.665567 30659 consensus_queue.cc:1059] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:14:59.679286 30728 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.680259 30455 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:14:59.669454 30461 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:59.716095 30852 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:14:59.719751 30871 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:59.699949 30732 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:59.723791 30808 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.748051 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:59.752995 23457 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:59.725451 30648 consensus_meta.cc:287] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.876s	user 0.010s	sys 0.002s
W20260709 13:14:59.742305 30842 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:14:59.758059 30861 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.061s	user 0.003s	sys 0.000s
I20260709 13:14:59.742748 30857 raft_consensus.cc:2804] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:59.726382 30864 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 13:14:59.735460 30461 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:59.793574 30857 raft_consensus.cc:697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:59.795193 30857 consensus_queue.cc:237] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.774119 30648 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.801019 30808 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:59.802296 23461 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:14:59.787508 30827 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:14:59.810686 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 115409us
I20260709 13:14:59.811707 30872 raft_consensus.cc:2804] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:59.811641 22806 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:59.812505 30872 raft_consensus.cc:697] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:59.813365 22806 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:14:59.813815 30871 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:59.817706 30541 raft_consensus.cc:2804] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:59.741750 30854 raft_consensus.cc:697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:14:59.818667 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 138200us
W20260709 13:14:59.819031 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 69507.2us
I20260709 13:14:59.828611 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.832803 22909 raft_consensus.cc:1240] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:59.836042 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.837193 22907 raft_consensus.cc:1240] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:59.838438 22915 raft_consensus.cc:2393] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 17: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:14:59.751890 30805 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.818296 30541 raft_consensus.cc:697] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:59.842213 30877 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.844317 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.781807 30655 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:59.730842 30855 consensus_queue.cc:1059] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:14:59.809305 30827 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:59.801376 30648 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:14:59.862342 22834 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:59.859268 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:59.840845 30642 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.867449 23457 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:14:59.872609 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2466) took 1086 ms. Trace:
I20260709 13:14:59.873216 22841 rpcz_store.cc:276] 0709 13:14:58.786300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.096656 (+310356us) service_pool.cc:224] Handling call
0709 13:14:59.136869 (+ 40213us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:59.158817 (+ 21948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:59.158866 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:59.159618 (+   752us) log.cc:844] Serialized 52 byte log entry
0709 13:14:59.542567 (+382949us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:59.542621 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.542756 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.872342 (+329586us) raft_consensus.cc:1685] finished
0709 13:14:59.872364 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.872546 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21446,"raft.queue_time_us":25861,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":348788,"threads_started":1,"wal-append.queue_time_us":517901}
I20260709 13:14:59.877857 30642 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.880470 30642 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:14:59.882984 30847 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:14:59.882441 30854 consensus_queue.cc:237] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.885313 30790 raft_consensus.cc:3055] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:59.885807 30790 raft_consensus.cc:740] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:59.886693 30790 consensus_queue.cc:260] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.887797 30790 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:59.890174 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.890604 30845 log.cc:927] Time spent T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.596s	user 0.001s	sys 0.000s
I20260709 13:14:59.891969 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3910) took 1051 ms. Trace:
W20260709 13:14:59.894658 30850 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 13:14:59.893980 30541 consensus_queue.cc:237] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 3
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:14:59.896016 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
I20260709 13:14:59.854393 30655 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:14:59.839105 30873 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.060s
W20260709 13:14:59.899360 30809 consensus_meta.cc:287] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.001s
W20260709 13:14:59.899890 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 276868us
I20260709 13:14:59.899848 30809 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.901356 23458 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:14:59.901950 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3908) took 1064 ms. Trace:
I20260709 13:14:59.902091 30848 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:59.902572 30848 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:14:59.902853 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:14:59.867789 30671 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.903561 30858 consensus_queue.cc:1059] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:59.904103 30671 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.904855 30648 raft_consensus.cc:3060] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:59.904963 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 250449us
I20260709 13:14:59.905779 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.906387 30671 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:14:59.868341 30432 consensus_meta.cc:287] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.587s	user 0.004s	sys 0.009s
I20260709 13:14:59.907478 30432 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:59.908313 30862 consensus_queue.cc:1059] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:14:59.909693 30304 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:59.910241 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:59.910444 30713 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:59.910862 22836 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:59.911232 30304 consensus_queue.cc:1048] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:14:59.913090 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2567) took 1057 ms. Trace:
I20260709 13:14:59.913271 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2561) took 1061 ms. Trace:
I20260709 13:14:59.914861 30860 raft_consensus.cc:3055] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:59.915268 30860 raft_consensus.cc:740] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:14:59.913815 22914 rpcz_store.cc:276] 0709 13:14:58.852249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.243932 (+391683us) service_pool.cc:224] Handling call
0709 13:14:59.244451 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:59.244938 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:59.403712 (+158774us) log.cc:844] Serialized 14 byte log entry
0709 13:14:59.897746 (+494034us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b180029eba0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e97ec0a 00007f68a80be219 00007f689ea06a40 00007f689ea014e4 00007f689ea01455 00007f68a1ae9060 00007f68a1afb716 00007f68a1b0c287 00007f68a1b0c208
0709 13:14:59.897953 (+   207us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:59.898614 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.912750 (+ 14136us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:59.912804 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.912892 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.912948 (+    56us) raft_consensus.cc:1685] finished
0709 13:14:59.913064 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.913207 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54842,"prepare.queue_time_us":345413,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":294754,"raft.run_cpu_time_us":7710,"raft.run_wall_time_us":35929,"spinlock_wait_cycles":9706368,"thread_start_us":247097,"threads_started":2,"wal-append.queue_time_us":261152}
I20260709 13:14:59.916128 30860 consensus_queue.cc:260] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.916630 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:59.878319 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:59.910243 22837 raft_consensus.cc:1240] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:14:59.899215 22765 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46956 (request call id 3095) took 1174 ms. Trace:
I20260709 13:14:59.918843 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.919533 22915 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:14:59.862555 30877 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.903193 30848 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:59.922935 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:59.904515 30858 raft_consensus.cc:3055] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:59.923794 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:59.924055 30858 raft_consensus.cc:740] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:14:59.854671 30793 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:59.923811 23465 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 25: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:14:59.930174 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:14:59.904099 23432 leader_election.cc:400] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17.
I20260709 13:14:59.932705 23432 leader_election.cc:403] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:14:59.933848 30874 raft_consensus.cc:3060] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:59.935366 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:14:59.935861 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.936347 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.937667 30868 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.938138 30868 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.939251 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2465) took 1153 ms. Trace:
I20260709 13:14:59.939508 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:59.940264 30868 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:14:59.941051 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.941440 30864 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:59.942006 22915 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 13:14:59.942899 30729 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:59.944815 30705 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.919131 22765 rpcz_store.cc:276] 0709 13:14:58.725159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.997922 (+272763us) service_pool.cc:224] Handling call
0709 13:14:59.034046 (+ 36124us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:59.044490 (+ 10444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:59.044542 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:59.045201 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.250287 (+205086us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:59.250333 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.250408 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.894003 (+643595us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:59.898816 (+  4813us) raft_consensus.cc:1685] finished
0709 13:14:59.898846 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.899121 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9873,"raft.queue_time_us":10946,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":17436,"spinlock_wait_cycles":7057664,"thread_start_us":164287,"threads_started":1,"wal-append.queue_time_us":418190}
I20260709 13:14:59.902343 22908 rpcz_store.cc:276] 0709 13:14:58.837098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.837334 (+   236us) service_pool.cc:224] Handling call
0709 13:14:58.837717 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.838123 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:58.838149 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:58.838575 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.106016 (+267441us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:59.106067 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.106141 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.889943 (+783802us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:59.901586 (+ 11643us) raft_consensus.cc:1685] finished
0709 13:14:59.901611 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.901868 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1577,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":47945984,"thread_start_us":267023,"threads_started":1,"wal-append.queue_time_us":388096}
I20260709 13:14:59.947801 30432 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:59.913578 22913 rpcz_store.cc:276] 0709 13:14:58.855645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.293155 (+437510us) service_pool.cc:224] Handling call
0709 13:14:59.418272 (+125117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.418582 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:59.617604 (+199022us) log.cc:844] Serialized 14 byte log entry
0709 13:14:59.912716 (+295112us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:59.912779 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:59.912829 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.912904 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.913029 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218524,"prepare.queue_time_us":200614,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":104,"raft.queue_time_us":219904,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":220185,"threads_started":2,"wal-append.queue_time_us":112118}
I20260709 13:14:59.948848 30739 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:14:59.917263 30860 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:59.939654 22839 rpcz_store.cc:276] 0709 13:14:58.786063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.092615 (+306552us) service_pool.cc:224] Handling call
0709 13:14:59.122973 (+ 30358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:59.559368 (+436395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:59.559423 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:59.560136 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.669804 (+109668us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:59.669883 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.938804 (+268921us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540013abd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:14:59.938878 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.938931 (+    53us) raft_consensus.cc:1685] finished
0709 13:14:59.938947 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.939189 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36743,"raft.queue_time_us":103661,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6751,"spinlock_wait_cycles":274574080,"thread_start_us":334664,"threads_started":2,"wal-append.queue_time_us":117563}
I20260709 13:14:59.892373 22910 rpcz_store.cc:276] 0709 13:14:58.840397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.843576 (+  3179us) service_pool.cc:224] Handling call
0709 13:14:58.843906 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.844255 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:58.844290 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:58.844810 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.090704 (+245894us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:59.092019 (+  1315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.092112 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:59.882359 (+790247us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:14:59.891626 (+  9267us) raft_consensus.cc:1685] finished
0709 13:14:59.891651 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.891903 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1422,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":3966848,"thread_start_us":245445,"threads_started":1,"wal-append.queue_time_us":631869}
I20260709 13:14:59.950191 22840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2477) took 1161 ms. Trace:
I20260709 13:14:59.819406 30872 consensus_queue.cc:237] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:14:59.979418 22840 rpcz_store.cc:276] 0709 13:14:58.789006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.169708 (+380702us) service_pool.cc:224] Handling call
0709 13:14:59.479927 (+310219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.498534 (+ 18607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.949957 (+451423us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54000a5350. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:14:59.950119 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712479104}
W20260709 13:14:59.949977 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46014.9us
I20260709 13:14:59.953670 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:14:59.954682 30785 raft_consensus.cc:3055] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:59.933182 30721 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:59.887135 30805 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:14:59.925038 30858 consensus_queue.cc:260] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.039646 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:59.961880 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:59.966547 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:00.004276 30432 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.973276 22834 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:15:00.046778 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:00.047748 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:00.048553 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:00.049034 22834 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:59.966118 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:59.951632 30814 raft_consensus.cc:3055] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:14:59.980289 30293 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:14:59.991670 22908 raft_consensus.cc:1240] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:59.992959 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:14:59.994890 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:14:59.995389 22910 raft_consensus.cc:1275] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:59.975117 30310 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.001402 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:14:59.941164 30868 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:59.975970 30642 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.005774 22906 raft_consensus.cc:1240] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
W20260709 13:15:00.005354 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.992955 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:00.055652 22913 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:14:59.982617 22805 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:00.022822 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:00.021999 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:00.021271 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:00.020712 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:00.030294 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:00.033095 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:15:00.036677 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 91825.3us
I20260709 13:14:59.959717 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:14:59.970623 22836 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 17.
I20260709 13:15:00.061687 23457 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:14:59.978086 30808 consensus_queue.cc:1059] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:15:00.051281 23459 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 13:15:00.060281 23460 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:15:00.064671 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:14:59.998569 30461 consensus_queue.cc:1059] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:15:00.029515 30785 raft_consensus.cc:740] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:00.072929 22834 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:15:00.084029 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 101892us
I20260709 13:15:00.085599 22879 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.089257 22910 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 13:15:00.092242 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.041364 30304 consensus_queue.cc:1059] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:15:00.094622 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:00.038311 30827 consensus_queue.cc:1048] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:15:00.092236 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:15:00.129791 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 48280.9us
I20260709 13:15:00.051229 30814 raft_consensus.cc:740] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:00.043231 30702 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.088372 30362 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:00.074328 30642 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.074631 30659 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:00.031122 30815 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:15:00.094563 30805 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:15:00.096555 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.142174 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:00.148097 30793 raft_consensus.cc:2804] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:00.099063 22908 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:15:00.147264 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 53545.5us
I20260709 13:15:00.095893 30653 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:15:00.169826 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 73424.8us
I20260709 13:15:00.074559 30648 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:00.177198 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.056232 30432 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.168905 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:00.074687 30310 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.135138 30530 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:00.181784 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:00.186782 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 157267us
I20260709 13:15:00.135516 30832 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:00.189474 22839 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:00.195339 23466 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:00.173002 30304 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:00.182626 30713 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:00.202543 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.178761 30871 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:00.181264 30653 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:00.204806 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47098.9us
I20260709 13:15:00.203442 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:00.159524 30808 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.043s
I20260709 13:15:00.188958 30805 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:00.180218 30793 raft_consensus.cc:697] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:00.195215 30461 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:00.222167 22915 raft_consensus.cc:1240] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:00.199230 30832 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.226207 30885 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.004s
W20260709 13:15:00.227209 30879 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:15:00.231031 30432 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.212100 30653 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:00.214854 30530 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:00.208662 30868 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:00.232995 30874 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 16. Current term is 17.
W20260709 13:15:00.260316 30839 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.974s	user 0.006s	sys 0.004s
I20260709 13:15:00.282114 30848 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:00.266314 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.283546 23463 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 17.
I20260709 13:15:00.238917 30713 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:00.292701 30785 consensus_queue.cc:260] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.270511 30887 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:00.294543 30785 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:00.294328 30887 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.296954 30887 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.253137 30827 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:00.297703 30891 consensus_queue.cc:1048] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:15:00.297855 30827 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.288318 30705 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.300395 30862 raft_consensus.cc:3055] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:00.301631 30690 consensus_queue.cc:1059] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:00.273983 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3918) took 1220 ms. Trace:
I20260709 13:15:00.308537 22906 rpcz_store.cc:276] 0709 13:14:59.053461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.472830 (+419369us) service_pool.cc:224] Handling call
0709 13:14:59.493156 (+ 20326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.057008 (+563852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.273699 (+216691us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54000eae54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:00.273915 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177819392}
I20260709 13:15:00.273952 30530 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.211678 30814 consensus_queue.cc:260] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.217386 30648 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:00.238448 30461 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.247478 30304 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:00.269786 30889 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.298380 30642 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:00.300539 30827 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:00.271677 30880 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:15:00.309823 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 107383us
I20260709 13:15:00.249809 30832 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:00.312922 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:00.300985 30862 raft_consensus.cc:740] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:00.326699 30868 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.239163 30659 consensus_queue.cc:1059] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:15:00.335364 22907 raft_consensus.cc:1240] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 23. Current term is 24. Ops: []
W20260709 13:15:00.389657 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.392257 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.384349 30793 consensus_queue.cc:237] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.334272 30739 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:00.337520 30461 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:00.310904 30805 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.436832 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44708.8us
I20260709 13:15:00.421103 30648 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.444065 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:00.418593 30827 consensus_queue.cc:1059] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 13:15:00.460131 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.470237 22880 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:00.445580 30648 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.483497 30827 raft_consensus.cc:3055] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:00.485324 30827 raft_consensus.cc:740] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:15:00.485560 30895 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 13:15:00.486447 30827 consensus_queue.cc:260] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.487732 30893 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:00.489192 30888 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:15:00.489568 22880 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:00.489895 30888 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:00.490504 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.387961 30889 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:00.500994 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57057.5us
I20260709 13:15:00.504982 23457 raft_consensus.cc:1240] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 13:15:00.506866 30739 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:15:00.507539 30739 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:00.508656 30739 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.521610 30713 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.522089 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:00.522300 30713 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:00.523078 30860 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.001s
I20260709 13:15:00.524688 30713 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:00.474426 30702 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.528187 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 30 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:00.528890 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.533803 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.448506 30805 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.609766 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:00.530908 30874 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.530686 30857 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:00.698107 30900 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.699427 30901 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.605254 30897 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.701314 30874 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:00.703431 30849 log.cc:927] Time spent T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 1.146s	user 0.003s	sys 0.012s
I20260709 13:15:00.639000 30721 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.724907 22841 raft_consensus.cc:2468] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:00.497725 30862 consensus_queue.cc:260] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.643813 30891 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.754602 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.742012 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:00.647171 30648 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:00.757567 22880 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:00.705423 30655 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.753611 30898 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.779920 30898 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.781472 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:00.782246 22910 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:00.782754 30898 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.755364 30721 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.732604 30901 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.764906 30858 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:00.765848 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:00.765240 22908 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:00.770823 30889 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.771960 30888 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:00.786142 30891 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:00.764369 22880 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:00.795225 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:00.795816 30876 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.003s	user 0.006s	sys 0.018s
I20260709 13:15:00.796254 30702 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:00.800012 22913 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:00.756528 30848 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:00.801944 22908 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:15:00.803978 22910 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:00.801744 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:00.788118 30728 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:00.844532 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:00.789034 30648 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.734755 30857 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.800777 30862 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:00.854480 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 268254us
I20260709 13:15:00.736577 30897 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:00.811705 30653 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.000s
I20260709 13:15:00.786149 30721 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:00.810722 30234 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:00.835278 30888 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.861287 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.862075 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.860980 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 96648.8us
W20260709 13:15:00.918493 22880 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:00.919242 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.919639 22836 raft_consensus.cc:3055] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:00.920234 22836 raft_consensus.cc:740] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:00.734242 30900 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.863013 30848 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:00.855705 30899 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.844491 30702 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.910686 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3906) took 2077 ms. Trace:
I20260709 13:15:00.854605 30648 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.853010 30728 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.847353 30885 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:15:00.919771 22880 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:00.845158 30889 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.956547 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.922680 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:00.921257 22836 consensus_queue.cc:260] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:00.926874 30748 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.865836 30653 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:00.943343 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:00.944459 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.944456 22832 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2471) took 2156 ms (client timeout 1622 ms). Trace:
I20260709 13:15:00.945931 22912 rpcz_store.cc:276] 0709 13:14:58.832837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.836090 (+  3253us) service_pool.cc:224] Handling call
0709 13:14:58.836347 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.100272 (+263925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.901280 (+1801008us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540020a8d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:00.910575 (+  9295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599511296}
I20260709 13:15:00.965862 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:00.966548 22912 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:00.937489 30310 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:00.960177 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.960129 22836 raft_consensus.cc:3060] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:00.929270 30234 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:00.964534 22832 rpcz_store.cc:269] 0709 13:14:58.787548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.854802 (+ 67254us) service_pool.cc:224] Handling call
0709 13:15:00.944227 (+2089425us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b540019ebd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:00.944369 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683500800}
I20260709 13:15:00.964147 22837 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:15:00.947270 30805 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:00.971481 22880 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:01.033478 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1382) took 1013 ms. Trace:
W20260709 13:15:01.045328 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73888.5us
W20260709 13:15:01.047171 22880 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:01.045149 23463 rpcz_store.cc:276] 0709 13:15:00.020447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.022224 (+  1777us) service_pool.cc:224] Handling call
0709 13:15:01.033112 (+1010888us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400740c54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:01.033373 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557740928}
I20260709 13:15:01.048308 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:01.053932 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 209492us
I20260709 13:15:01.055482 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:01.054733 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.951506 30899 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:01.048096 30906 log.cc:927] Time spent T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:15:01.045861 30848 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.035598 30908 log.cc:927] Time spent T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:15:00.933777 30888 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:01.063308 22880 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:01.057377 30805 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:00.963254 30889 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:01.000118 22805 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.970628 30748 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.089721 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:00.961033 30885 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.080616 30891 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.083915 30920 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.963109 30868 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:01.132079 22912 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:01.133600 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.138026 22912 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:01.144425 22836 raft_consensus.cc:1275] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:15:01.144949 22836 pending_rounds.cc:85] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 13
W20260709 13:15:01.129879 30909 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:15:01.145249 22836 pending_rounds.cc:107] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:15:01.145964 22836 raft_consensus.cc:2889] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.147002 30653 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.150291 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.150983 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
W20260709 13:15:01.101332 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 45874us
I20260709 13:15:01.106653 30710 raft_consensus.cc:3055] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:15:01.107295 30888 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:01.111709 30893 consensus_meta.cc:287] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.005s
I20260709 13:15:01.113915 30889 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.116734 22879 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.086869 30848 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.116532 30897 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.179955 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.137403 30636 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:15:01.177686 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 61054.5us
I20260709 13:15:01.178637 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.185464 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.190153 30636 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.190949 30636 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:15:01.191835 30915 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:15:01.190259 30234 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.157051 30710 raft_consensus.cc:740] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:01.195647 30710 consensus_queue.cc:260] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.197448 30900 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.178454 30921 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:01.179495 30893 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:01.151381 30890 consensus_queue.cc:1048] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:15:01.204149 22732 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:01.197386 30710 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:15:01.202435 30910 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:15:01.157619 30857 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.202744 30748 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.208022 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:01.206287 30916 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:15:01.191223 30917 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:15:01.211438 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:01.202744 30461 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.000s
I20260709 13:15:01.216091 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.223364 22912 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:15:01.182116 30888 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:01.132100 30920 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.218263 23457 raft_consensus.cc:2468] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
I20260709 13:15:01.206836 30900 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:01.224366 22732 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:01.224438 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.209882 30728 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.213585 30858 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:01.240870 30858 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.241371 30858 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:01.242362 30725 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:01.243921 22841 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2917) took 1278 ms (client timeout 1553 ms). Trace:
I20260709 13:15:01.217391 30893 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:01.229547 30648 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.219448 30702 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.225523 30925 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:01.247370 30920 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.249359 22839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1399) took 1163 ms. Trace:
I20260709 13:15:01.249825 22839 rpcz_store.cc:276] 0709 13:15:00.086043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.086185 (+   142us) service_pool.cc:224] Handling call
0709 13:15:00.147904 (+ 61719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.200450 (+ 52546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.249069 (+1048619us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b540017d4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:01.249290 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88690048}
I20260709 13:15:01.216837 30234 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:01.229924 30888 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.236708 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.259621 30785 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:01.235730 30814 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:01.263547 30901 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.265458 30748 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:01.266036 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.241183 30877 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:01.244380 22841 rpcz_store.cc:269] 0709 13:14:59.965829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.966621 (+   792us) service_pool.cc:224] Handling call
0709 13:15:01.243735 (+1277114us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b540027f1d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:01.243856 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083159424}
I20260709 13:15:01.227895 30461 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:01.268924 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:01.271118 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:01.271610 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.254328 30893 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.269491 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.274307 22837 raft_consensus.cc:1240] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 27. Current term is 28. Ops: [27.10-27.10]
I20260709 13:15:01.274955 22879 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:01.277112 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 52830.4us
I20260709 13:15:01.277110 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.279341 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.280014 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:01.281391 22832 raft_consensus.cc:3055] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:01.281687 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 30 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.282147 22832 raft_consensus.cc:740] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:01.282246 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.286058 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:01.288501 23432 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.284602 30901 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:01.284646 30748 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.292126 30868 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:01.293962 30748 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:01.294967 30832 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.004s
I20260709 13:15:01.295456 30832 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.295899 22841 raft_consensus.cc:1275] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:01.297951 30832 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.298589 30885 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.293843 22879 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:15:01.300452 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.301059 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.316457 22732 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:01.317868 22879 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.302170 22832 consensus_queue.cc:260] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.321169 22912 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:15:01.321362 22732 leader_election.cc:400] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21.
I20260709 13:15:01.323810 22732 leader_election.cc:403] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:15:01.292227 30901 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:01.327502 22832 raft_consensus.cc:3060] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:01.298615 30832 raft_consensus.cc:2804] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:01.312933 30885 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:01.322207 30748 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.324584 30920 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:01.309047 30234 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:15:01.335520 30925 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:01.347208 22832 raft_consensus.cc:1275] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:01.332692 30931 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:01.324461 30868 consensus_queue.cc:1059] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:15:01.345247 30927 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:01.325093 30729 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:15:01.356034 30848 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.346043 30901 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:01.344679 30885 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:15:01.378094 23430 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.378834 30885 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.337879 30636 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.382766 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1380) took 2450 ms. Trace:
I20260709 13:15:01.383294 23461 rpcz_store.cc:276] 0709 13:14:58.931973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.945448 (+ 13475us) service_pool.cc:224] Handling call
0709 13:14:58.945845 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:59.802935 (+857090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.382419 (+1579484us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54002c9554. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:01.382672 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968786304}
I20260709 13:15:01.384410 30636 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.385205 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40710 (request call id 964) took 1419 ms. Trace:
I20260709 13:15:01.385635 23459 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:01.385599 23466 rpcz_store.cc:276] 0709 13:14:59.965628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.965832 (+   204us) service_pool.cc:224] Handling call
0709 13:15:01.385001 (+1419169us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400603454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:01.385134 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334439808}
I20260709 13:15:01.386080 30541 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:15:01.386845 30927 raft_consensus.cc:697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:01.387028 30541 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:01.387468 30541 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:01.387912 30927 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.388319 30541 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.365845 23462 raft_consensus.cc:1240] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:01.369347 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.370255 30728 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:15:01.399030 30671 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
W20260709 13:15:01.370667 23458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43702 (request call id 1117) took 2512 ms (client timeout 1623 ms). Trace:
I20260709 13:15:01.352852 30888 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.375197 23463 raft_consensus.cc:1240] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:15:01.389050 30691 raft_consensus.cc:2804] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:01.389183 30931 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.401558 30310 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:15:01.402294 30931 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.347296 30234 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:15:01.391183 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.396684 23464 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 23: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:01.399808 23458 rpcz_store.cc:269] 0709 13:14:58.857938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.858094 (+   156us) service_pool.cc:224] Handling call
0709 13:15:01.370433 (+2512339us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400375354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:01.370585 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072076032}
W20260709 13:15:01.405128 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.405345 22832 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:01.346406 30748 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:01.413219 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 95480.4us
W20260709 13:15:01.417953 21636 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 22778)
I20260709 13:15:01.406497 30311 consensus_queue.cc:1059] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:15:01.420089 30311 consensus_queue.cc:1243] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:15:01.413662 30691 raft_consensus.cc:697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:01.420758 30931 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.421833 30899 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.421581 30691 consensus_queue.cc:237] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.422427 30893 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.428750 22832 raft_consensus.cc:2468] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:15:01.429370 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:01.429828 23430 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:01.407168 30832 raft_consensus.cc:697] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:01.414541 30541 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:01.435638 30893 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:01.425191 30901 consensus_queue.cc:1059] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 23.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:15:01.401013 30858 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.451503 30748 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.433816 30877 raft_consensus.cc:2804] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:01.455015 30877 raft_consensus.cc:697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:01.456228 30877 consensus_queue.cc:237] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:01.462209 22806 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.462399 30925 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21.
I20260709 13:15:01.465564 30655 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:01.473886 22906 raft_consensus.cc:2410] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 13:15:01.445747 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:01.474900 30890 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.498904 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2476) took 2710 ms. Trace:
W20260709 13:15:01.502166 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65610us
I20260709 13:15:01.490602 30893 raft_consensus.cc:3055] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:15:01.504861 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.499419 22838 rpcz_store.cc:276] 0709 13:14:58.788769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.097331 (+308562us) service_pool.cc:224] Handling call
0709 13:14:59.741613 (+644282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.741909 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:14:59.742250 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:14:59.854562 (+112312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:59.854649 (+    87us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:59.854718 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.854846 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.886687 (+ 31841us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:15:01.498683 (+1611996us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400038ed0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:01.498836 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16523,"prepare.queue_time_us":785,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":15801,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9762,"spinlock_wait_cycles":1584650496,"thread_start_us":96852,"threads_started":1,"wal-append.queue_time_us":97540,"wal-append.run_cpu_time_us":2584,"wal-append.run_wall_time_us":1063847}
I20260709 13:15:01.503868 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:01.505568 22832 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:01.500332 30858 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.516938 22832 raft_consensus.cc:2468] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 25.
I20260709 13:15:01.505106 30809 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.525740 30691 consensus_queue.cc:1048] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:01.531975 30544 raft_consensus.cc:3055] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:01.532568 30544 raft_consensus.cc:740] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:01.533245 22911 raft_consensus.cc:1240] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:01.533535 30544 consensus_queue.cc:260] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.535106 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:01.535424 30544 raft_consensus.cc:3060] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:01.536695 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.538118 30739 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:01.538040 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:01.538793 30739 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.539378 22909 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 24: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:15:01.519814 30901 consensus_queue.cc:1048] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:15:01.540489 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.541049 30739 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.517998 30939 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:15:01.498704 30890 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.505040 30893 raft_consensus.cc:740] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:01.547355 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.547456 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.559607 22836 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:01.548790 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.548594 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.560593 22840 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:01.548877 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.541922 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.561722 23459 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:01.561797 22914 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:15:01.557833 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.563740 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:01.558238 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.541460 22909 raft_consensus.cc:2393] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:01.564515 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.546423 23462 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 13:15:01.560858 23461 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:15:01.574093 23457 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:15:01.571117 22837 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 24: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:01.574213 22841 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:01.564064 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:01.576556 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:01.509034 30938 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:01.533324 30633 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:01.582376 30858 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:01.592001 30858 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.592460 30858 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:01.585047 30739 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.011s
I20260709 13:15:01.602072 30461 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.603020 30731 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.585537 30939 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:01.608398 30939 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.608790 30939 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:01.620204 30721 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.620863 30721 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.626881 30703 raft_consensus.cc:3060] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:01.627228 30893 consensus_queue.cc:260] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 27.10, Last appended by leader: 10, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.628901 30893 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:15:01.513671 30832 consensus_queue.cc:237] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.648036 30706 raft_consensus.cc:3055] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:01.720471 22835 raft_consensus.cc:2410] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:15:01.657603 30858 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.732615 22838 raft_consensus.cc:1275] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:15:01.728353 30706 raft_consensus.cc:740] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:01.648648 30827 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:01.735694 30706 consensus_queue.cc:260] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.545809 22832 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 25. Current term is 26. Ops: []
I20260709 13:15:01.653196 30721 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.739790 30937 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:01.741732 30706 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:01.747025 30897 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.748338 30897 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.751921 30897 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.751820 30748 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.752858 30788 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.753192 30748 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.753880 30788 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.756446 30748 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.721691 30713 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.757591 30788 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.758314 30713 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.658627 30939 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:01.715086 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.741091 30937 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.722333 23463 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:15:01.783898 30893 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.791041 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3932) took 1819 ms. Trace:
I20260709 13:15:01.825106 22907 rpcz_store.cc:276] 0709 13:14:59.971307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.035482 (+ 64175us) service_pool.cc:224] Handling call
0709 13:15:00.035813 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.438649 (+402836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.790326 (+1351677us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54000447d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:01.790736 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688373120}
I20260709 13:15:01.793668 22837 raft_consensus.cc:1240] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:01.795100 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.796646 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.799099 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3933) took 1823 ms. Trace:
I20260709 13:15:01.802613 30858 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.806695 30713 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.812866 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:01.832968 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.834728 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:01.820194 30939 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.836776 22840 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:15:01.822496 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.823225 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:01.795492 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:01.840948 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.800732 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:01.794507 30893 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.800945 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.818521 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.848747 30890 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.856508 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:01.861763 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62006.8us
I20260709 13:15:01.863953 22915 rpcz_store.cc:276] 0709 13:14:59.975517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.051059 (+ 75542us) service_pool.cc:224] Handling call
0709 13:15:00.086808 (+ 35749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.247076 (+160268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.798431 (+1551355us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540010dbd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:01.798851 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310649216}
I20260709 13:15:01.871308 22914 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:15:01.840950 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.864435 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.862769 30732 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.882200 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.884687 23462 raft_consensus.cc:2393] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 25: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:01.884286 22838 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:15:01.896544 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:01.900939 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.905736 30748 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:01.906852 30748 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.907279 30713 consensus_queue.cc:1048] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:01.908532 22833 raft_consensus.cc:1275] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:15:01.909029 22833 pending_rounds.cc:85] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 11
I20260709 13:15:01.909340 22833 pending_rounds.cc:107] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:15:01.909751 22833 raft_consensus.cc:2889] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.906899 30703 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.913692 30788 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:01.914608 30788 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:01.916451 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.917147 22837 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 25.
W20260709 13:15:01.917721 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 83439.5us
I20260709 13:15:01.917869 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:01.918220 30432 raft_consensus.cc:3055] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:01.918689 30432 raft_consensus.cc:740] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:01.919656 30721 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:01.920498 22908 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:01.920221 30432 consensus_queue.cc:260] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.920917 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.927098 22906 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:01.931223 22909 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:01.931834 30703 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.931878 22911 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 13:15:01.933292 22911 pending_rounds.cc:85] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 16
I20260709 13:15:01.933647 22911 pending_rounds.cc:107] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 13:15:01.934031 22911 raft_consensus.cc:2889] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.934410 22841 raft_consensus.cc:2468] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:01.932476 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.937259 22910 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:01.937666 22914 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:01.932520 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.943748 22832 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:01.920228 30721 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.951472 30862 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.930330 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.953459 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.952407 30703 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.917433 30788 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:01.955451 30721 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:01.967890 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.967013 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.966315 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:01.972051 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.973287 23465 raft_consensus.cc:2368] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:01.974226 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.974496 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:01.975404 23457 raft_consensus.cc:2410] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:15:01.976471 22910 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:01.975397 22837 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:01.896950 30732 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:01.983909 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 137955us
I20260709 13:15:01.979350 23432 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:01.979502 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:01.983201 30862 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.992635 30896 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:01.994163 30896 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:01.995987 30732 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.996688 30896 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:01.992523 30940 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:15:01.997332 22913 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:01.997900 30940 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:15:01.998277 30940 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:01.989459 22915 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:15:02.001034 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:02.004192 23431 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.005671 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68262.5us
W20260709 13:15:02.009807 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64281.4us
I20260709 13:15:02.010448 30729 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.011363 30707 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:01.985641 22807 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.015475 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:02.004230 30432 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:02.017927 23461 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:15:02.018466 30732 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.019344 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:02.020500 30874 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:02.022027 30937 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.023823 30748 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:02.026489 30893 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:01.999733 30940 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.022507 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:01.986027 30809 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.046698 22909 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
W20260709 13:15:02.046941 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.038878 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:02.047652 22832 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
I20260709 13:15:02.049474 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2919) took 2076 ms. Trace:
I20260709 13:15:02.057245 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2483) took 2198 ms. Trace:
I20260709 13:15:02.054502 22833 rpcz_store.cc:276] 0709 13:14:59.967271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.967403 (+   132us) service_pool.cc:224] Handling call
0709 13:14:59.990770 (+ 23367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:01.910478 (+1919708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.030680 (+120202us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b54000c62d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:02.043344 (+ 12664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4408419712}
I20260709 13:15:02.057751 22835 rpcz_store.cc:276] 0709 13:14:59.858745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.858911 (+   166us) service_pool.cc:224] Handling call
0709 13:15:02.052561 (+2193650us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540019f0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:02.057132 (+  4571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4263804800}
I20260709 13:15:02.060617 30809 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.064841 23466 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:15:02.066047 30940 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:02.066238 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:02.066923 22841 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:15:02.034847 30707 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:02.136062 23459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:02.098237 23465 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:02.056860 23460 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 26.
I20260709 13:15:02.071811 23457 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:15:02.068964 22806 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:02.140064 30707 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:02.150514 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:02.158406 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:02.129741 30809 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:02.155406 30239 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:02.140367 30897 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:02.161388 22806 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:02.162142 23430 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.181313 23459 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:15:02.183784 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.159089 30874 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:02.183846 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:02.146908 30732 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:02.202982 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.236549 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 279023us
I20260709 13:15:02.253659 22835 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 13:15:02.254016 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 215797us
I20260709 13:15:02.158867 30729 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.285493 22837 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:15:02.289584 22909 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:15:02.289795 30748 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:02.290429 30748 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.290812 30748 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:02.252787 30893 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:02.305979 30893 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.309019 30893 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:02.311350 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 242460us
I20260709 13:15:02.274462 30311 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:15:02.314715 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 293349us
I20260709 13:15:02.317138 22805 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.318456 30671 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:15:02.319013 22879 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.320402 30671 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:02.320873 30671 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:02.320957 30653 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:02.321468 30653 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:02.321945 30653 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:02.322418 30362 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.323074 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.289902 22915 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [22.13-24.14]
I20260709 13:15:02.321278 30671 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:02.331899 23466 raft_consensus.cc:1275] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:02.271214 30896 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.276351 30874 raft_consensus.cc:2804] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:15:02.258797 30897 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.323072 30362 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.337347 30311 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:02.347816 30874 raft_consensus.cc:697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:02.368222 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:02.368486 30815 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:02.369220 30815 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.369704 22879 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.370977 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.371646 30937 raft_consensus.cc:2804] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:02.372157 30939 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:02.372568 30939 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.372684 30788 consensus_queue.cc:1048] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:02.372699 30937 raft_consensus.cc:697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:02.374311 30939 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:02.369259 30874 consensus_queue.cc:237] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.362857 30311 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:02.375728 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101922us
W20260709 13:15:02.377068 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.387192 30827 consensus_meta.cc:287] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.652s	user 0.018s	sys 0.001s
I20260709 13:15:02.398128 30721 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.400185 30671 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.401129 30362 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:02.403308 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.381387 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:02.413038 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53431.1us
I20260709 13:15:02.402308 30311 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:02.425601 23462 raft_consensus.cc:2368] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
W20260709 13:15:02.426381 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 261651us
W20260709 13:15:02.440524 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.442747 30793 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.443320 22837 raft_consensus.cc:1275] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:15:02.443400 30793 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.444908 30362 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:02.374523 30937 consensus_queue.cc:237] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:02.470705 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.481760 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 54568.4us
I20260709 13:15:02.412016 30896 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.447757 30362 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:02.486886 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.477381 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.417150 30707 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.505724 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:02.507022 22835 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
I20260709 13:15:02.508391 30858 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:02.509325 30858 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.511231 30893 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:02.511715 30893 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.512104 30893 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:02.513177 30858 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.513664 30793 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:02.516233 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.478670 30815 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:02.525214 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46281us
I20260709 13:15:02.497196 30362 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:02.534065 30793 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:02.559813 23463 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:02.542001 30793 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:02.578408 22880 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:02.579658 30793 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:02.596494 22880 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:02.606401 22879 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:02.572640 30862 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:02.596666 30713 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.591040 30729 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.595044 30732 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:02.636873 30955 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.576s	user 0.005s	sys 0.000s
I20260709 13:15:02.644542 30897 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:02.646211 30713 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.652422 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:02.653371 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:02.655709 30707 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:02.611615 30671 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.656384 30815 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:02.669127 30721 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:02.667385 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.626650 30653 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:02.683670 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61187.9us
I20260709 13:15:02.683871 30732 consensus_queue.cc:1048] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:02.684717 22732 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:02.685093 30956 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.625s	user 0.005s	sys 0.000s
I20260709 13:15:02.696966 30706 consensus_queue.cc:1048] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
W20260709 13:15:02.714972 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 132013us
I20260709 13:15:02.716850 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:02.717622 30959 log.cc:927] Time spent T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.240s	user 0.000s	sys 0.002s
I20260709 13:15:02.720866 30963 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:02.726227 22835 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 17.
W20260709 13:15:02.701439 30940 consensus_meta.cc:287] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.635s	user 0.014s	sys 0.003s
I20260709 13:15:02.720854 30748 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.727238 30707 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.739616 30748 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:02.753199 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.758637 30713 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:02.761591 30913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.056s
W20260709 13:15:02.762048 30913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.010s	sys 0.056s
I20260709 13:15:02.727504 30815 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.752717 30963 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:02.753953 30707 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.778295 30707 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:15:02.746587 30729 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:15:02.772938 30940 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:02.781674 23459 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.009s
I20260709 13:15:02.782076 23459 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:15:02.786527 30793 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.791460 30732 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:15:02.792302 30362 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.793943 23462 raft_consensus.cc:2368] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 26. Current term is 27.
I20260709 13:15:02.781119 30940 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.796648 30940 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.798202 30788 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.799099 30788 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:02.802670 30239 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.635s	user 0.014s	sys 0.000s
I20260709 13:15:02.803575 22912 raft_consensus.cc:2468] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:15:02.804097 30896 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.804240 30853 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.805094 30853 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.803567 30815 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:02.808090 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60658.6us
I20260709 13:15:02.809453 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:02.810875 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:02.812114 22732 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:02.813375 30961 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:15:02.813596 30793 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:02.814105 30961 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:02.815291 30961 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.816460 30362 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:02.775085 30963 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.774821 30713 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:02.819854 30963 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:02.822312 30896 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:02.816387 30311 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.826495 30729 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:02.833029 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:02.830003 30713 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:15:02.865495 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 46140us
I20260709 13:15:02.840276 30896 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:02.843762 30729 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.873667 30896 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:02.876019 30729 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:02.918722 22732 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:02.919411 22732 leader_election.cc:403] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:15:02.931824 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.940465 30432 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.869s	user 0.014s	sys 0.000s
I20260709 13:15:02.941810 30311 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:02.942771 30311 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.943503 30311 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:02.945699 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.958196 30793 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.959048 30793 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.961889 30793 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:02.961642 30893 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.964322 30893 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:02.949406 30939 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.004s
I20260709 13:15:02.966770 30939 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:02.969856 30939 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:02.977656 30713 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:02.977999 30653 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:02.976662 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 101083us
I20260709 13:15:02.981899 30432 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:02.983484 30432 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:15:02.984215 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.984009 30432 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:02.934671 30788 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:02.988059 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.989812 30788 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:02.990552 30788 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.044180 30311 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.046078 30788 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:03.047004 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 471609us
I20260709 13:15:03.042829 30853 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.053674 30966 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.054409 30966 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:03.037294 30809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.013s	sys 0.036s
W20260709 13:15:03.082744 30809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.013s	sys 0.036s
I20260709 13:15:03.092761 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2594) took 1240 ms. Trace:
W20260709 13:15:03.109488 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.081719 30966 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.082464 30827 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:03.115422 30963 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:03.116845 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 500716us
W20260709 13:15:03.136548 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 570837us
W20260709 13:15:03.142078 30874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.007s	sys 0.018s
W20260709 13:15:03.156481 23430 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.018s	sys 0.022s
W20260709 13:15:03.155787 22837 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.016s	sys 0.024s
W20260709 13:15:03.172248 23430 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.021s	sys 0.022s
W20260709 13:15:03.172783 22837 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.016s	sys 0.027s
W20260709 13:15:03.173763 22879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.024s	sys 0.036s
W20260709 13:15:03.174137 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 552109us
I20260709 13:15:03.145715 30966 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:15:03.111294 22914 rpcz_store.cc:276] 0709 13:15:01.852180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.950195 (+ 98015us) service_pool.cc:224] Handling call
0709 13:15:01.976764 (+ 26569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:01.977369 (+   605us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:15:02.143260 (+165891us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.081029 (+937769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:03.081122 (+    93us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:03.081198 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:03.092446 (+ 11248us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.092677 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13096,"prepare.queue_time_us":154350,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":165028,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":10648,"thread_start_us":844436,"threads_started":2,"wal-append.queue_time_us":585560}
W20260709 13:15:03.159284 30874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.008s	sys 0.019s
I20260709 13:15:03.179853 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:03.188805 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 204224us
I20260709 13:15:03.148993 30827 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:03.174168 22879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.024s	sys 0.036s
W20260709 13:15:03.159215 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.148602 30432 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.200552 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:03.202443 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 596116us
W20260709 13:15:03.206992 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 157317us
I20260709 13:15:03.204975 23466 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:03.180596 30809 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.189831 30815 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:03.185175 30897 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.192236 30362 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.222288 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:03.222198 22878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.021s
W20260709 13:15:03.230986 22878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.011s	sys 0.021s
I20260709 13:15:03.230368 23462 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:15:03.231405 30809 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.241734 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:03.242449 23466 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:03.215626 30815 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:03.243462 22730 leader_election.cc:400] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:03.246168 22730 leader_election.cc:403] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:03.255650 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.256107 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.256554 22834 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:15:03.257180 22835 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 18.
W20260709 13:15:03.257568 30970 log.cc:927] Time spent T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:15:03.256140 30974 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.258978 30974 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:03.256254 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 82015.8us
I20260709 13:15:03.264032 30974 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.266631 23458 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:03.268875 30978 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.268757 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:03.273607 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.183488 30971 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:15:03.275564 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:03.280180 23464 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 25: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:03.276312 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:03.299625 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.250052 30815 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:03.307466 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:03.310500 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:03.313107 22914 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:03.313582 30974 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:03.314358 30974 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.316856 30974 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:03.318367 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74591.2us
I20260709 13:15:03.310268 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.320638 22834 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:03.279642 30963 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.318802 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:03.334278 30982 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:15:03.335120 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:03.336410 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.339759 30853 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.340412 30853 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.321767 30988 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.341676 30653 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.342093 30988 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.342387 30653 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.344743 30988 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.344856 30653 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.346606 30827 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.349025 30977 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:03.349478 30977 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.349845 30977 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:03.350432 30980 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:03.351006 30980 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.352788 30729 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:03.326205 30973 log.cc:927] Time spent T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:15:03.326731 22839 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:03.326862 30937 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.327211 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.363456 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.330490 30972 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.330416 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.269579 30978 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.388082 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.354287 30432 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.285980 30713 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:03.355609 30975 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:03.251449 30897 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:03.356237 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.323611 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:03.357187 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.361786 23466 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:15:03.329732 30979 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.371634 22910 raft_consensus.cc:2410] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:15:03.392905 22837 raft_consensus.cc:2393] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:03.365711 30827 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.380290 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.408602 22912 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:15:03.245373 30362 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.317397 30974 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:03.402420 22909 raft_consensus.cc:2393] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 24: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:03.395795 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.404414 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:03.405686 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 96732.1us
I20260709 13:15:03.408998 22879 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:03.415162 23465 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:03.363219 30980 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.411281 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.424963 22840 raft_consensus.cc:3060] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:03.428732 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.432829 22832 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:03.434073 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:03.423470 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.435258 22833 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:03.438637 22910 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:03.438721 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.455001 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.372455 30311 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.052s
I20260709 13:15:03.457331 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:03.459373 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.415645 30971 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.468318 23459 raft_consensus.cc:1275] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:03.465230 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:03.423673 30432 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.439239 30969 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:03.478704 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:03.415496 30963 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.499573 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.500259 22835 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:15:03.500871 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61107.3us
I20260709 13:15:03.437393 30827 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.526223 30963 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.526266 22910 raft_consensus.cc:2393] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:03.528602 30963 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.494760 30858 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:03.526614 30961 raft_consensus.cc:3060] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:15:03.439486 30937 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.540206 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.513934 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.537492 30989 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.456782 30972 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.438627 30713 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.538375 30977 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.551563 30989 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.552582 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:03.553974 30896 consensus_meta.cc:287] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
I20260709 13:15:03.554349 30897 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.554483 30966 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:03.495393 30939 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:15:03.554267 30971 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.499325 30974 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.613081 30974 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.556440 30998 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:15:03.559182 30937 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.614970 30977 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.559908 22914 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:15:03.561750 30978 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.433990 30975 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.617545 30975 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.563884 30362 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.564031 30809 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.533700 30858 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:03.566421 30988 consensus_queue.cc:1059] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:03.566735 30862 consensus_queue.cc:1059] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:03.566473 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:03.570283 30853 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.505781 30432 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.571430 30671 consensus_queue.cc:1059] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:03.499846 30969 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.640869 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:03.587015 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.588564 22912 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:03.532124 30963 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.597957 22910 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 29.
I20260709 13:15:03.609519 30989 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.597958 30896 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.505671 30979 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.624823 22836 raft_consensus.cc:1275] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:15:03.619131 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:03.657945 22906 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 13:15:03.632385 30815 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.633738 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:03.646167 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.629979 30961 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:03.652472 22834 raft_consensus.cc:2468] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:03.633437 30858 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:03.664825 22805 leader_election.cc:400] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 26. Current term is 27.
I20260709 13:15:03.660403 22832 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:15:03.642472 30969 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:03.655735 30988 consensus_queue.cc:1243] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:03.651077 30729 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.665205 22805 leader_election.cc:403] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
W20260709 13:15:03.667001 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:03.711402 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 46754.5us
I20260709 13:15:03.682303 30971 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.683722 30432 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.685719 30979 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.688922 30827 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.687981 30815 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:03.687765 30896 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.700723 30713 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.703234 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.718567 22837 raft_consensus.cc:2410] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
W20260709 13:15:03.665863 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99517.8us
I20260709 13:15:03.720554 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:03.722556 30999 raft_consensus.cc:2764] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:15:03.682288 30963 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.725384 30963 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.682648 31000 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:15:03.683037 30862 raft_consensus.cc:3055] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:15:03.728322 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 202903us
I20260709 13:15:03.731680 22840 raft_consensus.cc:2468] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:15:03.736416 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:03.738201 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.747589 22833 raft_consensus.cc:2468] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
W20260709 13:15:03.661760 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.740690 22834 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:15:03.767372 30961 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.767972 30963 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.770358 30963 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.770265 30961 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.767238 31008 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:03.773123 30963 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.767746 30432 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.776244 30432 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.779608 30432 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.774263 31008 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.777251 30961 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.746850 22835 raft_consensus.cc:2468] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 27.
I20260709 13:15:03.789124 31008 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:03.805953 30996 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:03.806632 30996 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.807000 30996 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:03.809689 22664 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:03.761206 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:15:03.812283 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51167.7us
W20260709 13:15:03.806000 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68523.3us
I20260709 13:15:03.702932 30972 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:03.797542 23431 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.054s (SENT) [suppressed 88 similar messages]
I20260709 13:15:03.822896 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:03.824862 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:03.826025 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:03.821645 22805 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:03.730384 30862 raft_consensus.cc:740] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:03.740494 30788 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:03.831431 22834 raft_consensus.cc:1240] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:15:03.833417 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:03.752617 30999 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.082s
I20260709 13:15:03.835331 30788 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.835963 30999 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:03.678144 30858 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.836567 30999 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:03.838562 30858 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:03.839787 30862 consensus_queue.cc:260] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.840811 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.841336 22841 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:03.841688 22837 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 24: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:03.811267 30988 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for earlier term 26. Current term is 27.
I20260709 13:15:03.823442 30978 raft_consensus.cc:3055] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:03.738535 31000 consensus_queue.cc:1059] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 13:15:03.843827 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.844029 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.845743 22911 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:15:03.846072 22834 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:15:03.846650 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.848803 31011 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:03.827207 30815 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:15:03.817584 31012 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.841161 30862 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:03.854333 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:03.933864 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 79627.2us
I20260709 13:15:03.935273 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:03.881709 30362 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.854789 30972 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:15:03.949848 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 152364us
I20260709 13:15:03.929423 31012 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.929700 31011 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.856912 30978 raft_consensus.cc:740] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:15:03.877841 22730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.058s	sys 0.082s
W20260709 13:15:03.957795 22730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.058s	sys 0.082s
W20260709 13:15:03.924446 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.931319 22838 raft_consensus.cc:2368] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
W20260709 13:15:03.901957 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:03.961488 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.952757 30858 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:03.855197 30975 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:03.910545 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:03.889415 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:03.996093 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.947275 22879 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:03.953518 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.945797 30827 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:03.942237 30972 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:03.951333 22664 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 12 to 24, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 24 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:03.977334 31012 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:03.978392 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 755806us
I20260709 13:15:03.959259 31011 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:03.986057 30969 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.948333 30362 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:03.992406 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.994448 22840 raft_consensus.cc:1240] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:03.994268 30978 consensus_queue.cc:260] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:03.995311 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.011055 22837 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:15:03.999688 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 52515.1us
W20260709 13:15:04.003230 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 118811us
W20260709 13:15:04.015793 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 79016.3us
I20260709 13:15:03.888993 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.008320 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.008406 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:03.854941 30999 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.012539 22879 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:04.033700 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.035801 22841 raft_consensus.cc:2410] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 13:15:04.037627 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:04.003260 30972 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:04.003858 30974 raft_consensus.cc:2804] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:04.039433 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.001185 30827 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.929308 30815 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:04.041718 22879 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:04.044103 22909 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:04.008069 30980 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:04.055450 22840 raft_consensus.cc:2393] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:04.068041 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.070550 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.072089 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.079339 22838 raft_consensus.cc:1275] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:04.046391 30975 raft_consensus.cc:2804] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:04.080346 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:04.082576 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 45100.4us
I20260709 13:15:04.082927 22834 raft_consensus.cc:1275] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:15:04.083964 22837 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:04.085953 30969 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:04.054867 30827 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:04.087463 30788 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:04.073242 31007 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:04.096199 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 27 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.097112 22833 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 28.
I20260709 13:15:04.098351 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.100536 30978 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:04.101227 30362 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:04.054088 30815 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:15:04.103546 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.081022 23464 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:15:04.114003 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:04.150568 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.150835 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.070123 30853 raft_consensus.cc:2804] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:04.152038 23462 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:15:04.153647 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:04.155314 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 142114us
I20260709 13:15:04.155825 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:15:04.157382 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43544.1us
I20260709 13:15:04.166368 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.167065 22911 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:04.168784 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.179538 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.185050 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.186702 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.187497 22840 raft_consensus.cc:2393] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 18: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:04.190109 22838 raft_consensus.cc:2393] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:04.192013 30311 raft_consensus.cc:3055] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:04.192531 30311 raft_consensus.cc:740] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:04.192996 30961 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:04.193475 30961 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:04.193933 30961 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:04.193645 30311 consensus_queue.cc:260] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.195194 30311 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:04.198071 22732 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:04.199328 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.200019 22906 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:04.204697 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.165057 30653 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.210078 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.212600 22915 raft_consensus.cc:2410] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 28 because replica has last-logged OpId of term: 27 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 9.
I20260709 13:15:04.164381 30969 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:04.203936 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.217487 22914 raft_consensus.cc:2393] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
W20260709 13:15:04.214083 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99171us
I20260709 13:15:04.172101 30980 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:04.224171 30980 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.226281 30980 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:04.226909 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.227535 22907 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:04.230579 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.232069 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.232617 22914 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 19.
I20260709 13:15:04.224327 30963 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:15:04.170787 30975 raft_consensus.cc:697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:04.181062 30939 consensus_queue.cc:1048] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:15:04.189646 30853 raft_consensus.cc:697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:04.186278 31008 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 13:15:04.233989 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.235670 22732 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:04.242199 22664 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:04.242635 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.242997 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.233472 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.264765 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:04.267729 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 123160us
I20260709 13:15:04.246358 31002 raft_consensus.cc:2804] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:04.267905 22839 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 22: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:04.236115 30653 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.275935 22839 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:04.277168 22839 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:04.248224 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.235889 30432 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:15:04.267680 31026 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.281163 31026 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.224582 30971 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:15:04.284113 31007 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:04.285591 30969 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:04.287137 23431 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.715s (SENT)
W20260709 13:15:04.288233 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.751s (SENT)
I20260709 13:15:04.249673 22833 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:15:04.288641 23431 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:04.289816 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.240964 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.292977 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.292518 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:04.300370 30939 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:04.245604 30853 consensus_queue.cc:237] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.300835 30939 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.301554 30939 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:04.302026 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:04.302482 22837 raft_consensus.cc:1275] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:15:04.302893 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 67331.9us
I20260709 13:15:04.303337 22730 leader_election.cc:400] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
W20260709 13:15:04.263098 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:04.305210 22880 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.238305 30981 raft_consensus.cc:3055] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:04.305231 22911 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:04.309926 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.274535 31002 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.314802 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:04.313680 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 230004us
I20260709 13:15:04.291488 30858 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:04.316077 23465 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:15:04.291905 30362 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:04.320993 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.315902 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.324476 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:04.328670 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.329267 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.332396 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:04.308252 30729 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:04.306239 30963 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:15:04.337188 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:04.339805 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.342579 23462 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:04.336112 22880 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.318914 31002 raft_consensus.cc:3060] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:04.343425 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.313405 30974 raft_consensus.cc:697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:04.309120 30981 raft_consensus.cc:740] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:04.345122 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.343605 22664 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 12 to 16, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 16 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.242378 30975 consensus_queue.cc:237] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.348732 22915 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
I20260709 13:15:04.344075 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.368863 22835 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:15:04.350049 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.336985 30729 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:04.308475 31008 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:15:04.322788 31012 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:15:04.373540 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:04.376796 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53508.1us
I20260709 13:15:04.350647 30998 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:04.377815 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.385392 30974 consensus_queue.cc:237] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.398607 31008 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:15:04.390476 30981 consensus_queue.cc:260] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.392311 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:04.402379 30981 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:04.402216 30827 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.376122 30729 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:04.404647 30827 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:04.405490 22912 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:15:04.394248 30937 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:04.395468 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:04.406337 30996 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.599s	user 0.016s	sys 0.001s
I20260709 13:15:04.406831 22911 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:04.406792 30996 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.408234 22832 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:15:04.408949 22907 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:04.409125 22909 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:15:04.408383 22911 raft_consensus.cc:1275] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:15:04.378178 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.431129 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.415294 22907 raft_consensus.cc:1275] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:04.356447 30858 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:15:04.418988 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.809s (SENT)
I20260709 13:15:04.409461 30996 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:04.439146 22907 raft_consensus.cc:1240] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:04.408459 22664 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:15:04.434463 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 270562us
W20260709 13:15:04.413121 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:04.428957 23429 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:04.432839 22655 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 269) took 41500 ms (client timeout 15000 ms). Trace:
I20260709 13:15:04.429394 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.437186 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.414299 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:04.443425 22880 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:04.440977 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:04.447470 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69347.3us
W20260709 13:15:04.448335 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.448684 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.449101 22906 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:04.438679 30858 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.389004 30961 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:04.465281 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51073.7us
I20260709 13:15:04.475416 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:04.473889 22880 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.416649 30815 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.414175 30937 consensus_queue.cc:1048] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:15:04.478091 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 59150.8us
W20260709 13:15:04.489032 23431 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.885s (SENT)
W20260709 13:15:04.488121 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46444.8us
I20260709 13:15:04.420105 30729 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:04.430123 30972 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.476744 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.430615 31011 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:04.480463 22806 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.497099 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.447773 30981 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:15:04.485488 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:04.502177 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.503288 23459 raft_consensus.cc:1240] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 29. Current term is 30. Ops: []
I20260709 13:15:04.486691 22906 raft_consensus.cc:1240] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:04.463650 30858 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:04.492699 22664 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 6 to 21, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:04.477752 31028 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 13:15:04.500746 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.942s (SENT)
I20260709 13:15:04.511461 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.509415 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.518460 22906 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 23. Current term is 24. Ops: [22.11-23.12]
I20260709 13:15:04.517071 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.519042 23462 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 27: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:04.520448 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:04.521225 22838 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:04.522357 22664 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:04.479549 30827 consensus_queue.cc:1048] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 13:15:04.565399 22806 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.568603 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:04.587620 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 282920us
I20260709 13:15:04.489902 31027 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:15:04.511257 31028 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.498567 31031 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:04.609921 30827 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:04.563074 30986 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.013s	sys 0.035s
I20260709 13:15:04.563032 30998 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:15:04.621774 23461 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:04.618474 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.619380 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:04.623654 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 129408us
W20260709 13:15:04.624576 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 149247us
I20260709 13:15:04.628782 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.594784 30979 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:04.626281 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:04.634255 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:04.642604 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 122730us
W20260709 13:15:04.627767 30986 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.013s	sys 0.036s
W20260709 13:15:04.661264 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 187400us
I20260709 13:15:04.613896 30432 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.617368 31028 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:15:04.663390 22732 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:04.668378 22880 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:04.669345 22806 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:04.670280 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.676973 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.676865 23460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:04.623806 30971 consensus_queue.cc:1059] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:15:04.681813 22908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51142 (request call id 1202) took 1344 ms. Trace:
I20260709 13:15:04.668987 30713 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:04.687918 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61967.9us
W20260709 13:15:04.689323 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 188606us
W20260709 13:15:04.698053 23431 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.020s (SENT)
I20260709 13:15:04.699433 23430 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:04.704039 23430 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:04.687573 22908 rpcz_store.cc:276] 0709 13:15:03.337392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.337573 (+   181us) service_pool.cc:224] Handling call
0709 13:15:03.337951 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.618248 (+280297us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:15:03.768423 (+150175us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.068730 (+300307us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.069147 (+   417us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.184095 (+114948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:04.184194 (+    99us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:15:04.184247 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.235353 (+ 51106us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b1800493120. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:04.681445 (+446092us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54000fc154. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:04.681554 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:04.681729 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66851,"prepare.queue_time_us":163662,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":91,"raft.queue_time_us":23120,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":30048,"spinlock_wait_cycles":664462464,"thread_start_us":544549,"threads_started":3,"wal-append.queue_time_us":498959}
I20260709 13:15:04.663091 31031 consensus_queue.cc:1048] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:15:04.688514 31046 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:04.723246 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58841us
W20260709 13:15:04.731525 22880 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.719336 22664 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 16 to 17, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 17 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.687057 30937 consensus_queue.cc:1048] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 13:15:04.739903 30980 consensus_meta.cc:287] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.002s
I20260709 13:15:04.682832 30979 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:15:04.740370 30980 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.711686 30961 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:04.742322 31035 log.cc:927] Time spent T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:15:04.744282 30939 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.679916 30432 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:04.745859 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75616.1us
W20260709 13:15:04.746658 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.747213 30432 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:04.441727 22655 rpcz_store.cc:269] 0709 13:14:22.876664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.529787 (+7653123us) service_pool.cc:224] Handling call
0709 13:15:04.370075 (+33840288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:58.352801 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:58.438243 (+ 85442us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:58.438318 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:14:58.438348 (+    30us) tablet.cc:662] Decoding operations
0709 13:14:58.528407 (+ 90059us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:58.528557 (+   150us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:58.544734 (+ 16177us) tablet.cc:685] Acquiring locks for 34 operations
0709 13:14:58.561107 (+ 16373us) tablet.cc:701] Row locks acquired
0709 13:14:58.561159 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:14:58.561418 (+   259us) write_op.cc:270] Start()
0709 13:14:58.594556 (+ 33138us) write_op.cc:276] Timestamp: P: 1783602898561332 usec, L: 0
0709 13:14:58.594600 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:14:58.595541 (+   941us) log.cc:844] Serialized 13261 byte log entry
0709 13:14:58.708960 (+113419us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b
0709 13:14:59.880678 (+1171718us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:15:00.705639 (+824961us) op_driver.cc:464] REPLICATION: finished
0709 13:15:01.246399 (+540760us) write_op.cc:301] APPLY: starting
0709 13:15:01.246549 (+   150us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:01.630954 (+384405us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:01.631002 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:02.190915 (+559913us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:02.191565 (+   650us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=68,key_file_lookups=68,delta_file_lookups=68,mrs_lookups=0
0709 13:15:02.191633 (+    68us) write_op.cc:312] APPLY: finished
0709 13:15:02.192610 (+   977us) log.cc:844] Serialized 292 byte log entry
0709 13:15:02.398050 (+205440us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b
0709 13:15:02.762576 (+364526us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:02.763271 (+   695us) write_op.cc:454] Released schema lock
0709 13:15:02.763372 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:02.763996 (+   624us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:02.764029 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":948491,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":502741,"apply.run_cpu_time_us":125115,"apply.run_wall_time_us":1540958,"cfile_cache_hit":138,"cfile_cache_hit_bytes":146315,"num_ops":34,"prepare.queue_time_us":590215,"prepare.run_cpu_time_us":99373,"prepare.run_wall_time_us":817804,"raft.queue_time_us":247879,"raft.run_cpu_time_us":15648,"raft.run_wall_time_us":372665,"replication_time_us":2043441,"spinlock_wait_cycles":152640512,"thread_start_us":1909019,"threads_started":5,"wal-append.queue_time_us":1483083,"wal-append.run_cpu_time_us":87386,"wal-append.run_wall_time_us":3231918}]]}
W20260709 13:15:04.748929 22806 leader_election.cc:336] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:04.749658 31033 log.cc:927] Time spent T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:15:04.737885 31046 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:04.701224 31038 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:04.756031 31038 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.756450 31038 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:04.759907 30858 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.739107 31031 raft_consensus.cc:2804] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:15:04.761672 22880 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.738413 31023 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:04.761749 30729 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:04.761758 31011 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:04.745898 31032 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:15:04.762444 30972 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:04.763042 30939 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:04.733678 30815 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:04.767608 31050 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:15:04.787601 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.793865 23466 raft_consensus.cc:2368] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:15:04.749402 22806 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.801543 22907 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:04.805233 30963 raft_consensus.cc:3055] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:04.820866 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:04.842501 23462 raft_consensus.cc:3060] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:04.822335 22807 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.822821 30896 raft_consensus.cc:3055] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:04.824280 30862 consensus_queue.cc:1059] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 13:15:04.824869 30653 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:04.829558 31026 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:04.832852 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 169556us
I20260709 13:15:04.832971 22838 raft_consensus.cc:2468] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
I20260709 13:15:04.830418 22664 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:04.846416 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.853600 22835 raft_consensus.cc:2468] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 17.
W20260709 13:15:04.848443 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 99531us
I20260709 13:15:04.846376 30896 raft_consensus.cc:740] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:04.849608 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.856590 22834 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:04.837071 31031 raft_consensus.cc:697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:04.852051 22879 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:04.858439 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 145835us
I20260709 13:15:04.855544 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.874728 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:04.875093 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 113457us
W20260709 13:15:04.876447 22880 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.877415 22836 raft_consensus.cc:2393] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 27: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:04.842168 30963 raft_consensus.cc:740] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:15:04.859234 22730 connection.cc:256] RPC call timeout handler was delayed by -2.2763549619999139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 13:15:04.882651 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.883028 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.883136 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:04.883926 22833 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 28: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:04.850481 31050 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:15:04.885883 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.852806 30998 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:04.877526 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:15:04.886530 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.889815 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:04.890702 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.855192 30671 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:04.889858 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.884524 31026 raft_consensus.cc:2804] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:04.905301 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.908701 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.908815 22834 raft_consensus.cc:2468] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:15:04.908883 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:04.901991 23431 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.906278 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:04.907958 22806 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:04.894634 31002 consensus_meta.cc:287] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
W20260709 13:15:04.913739 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 209793us
I20260709 13:15:04.915812 23430 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:04.916649 22806 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.922201 22807 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:04.929313 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.933864 22837 raft_consensus.cc:1240] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 23. Ops: []
I20260709 13:15:04.940599 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:04.942812 22833 raft_consensus.cc:2393] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:04.907517 30939 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:04.947989 22732 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:04.927438 30978 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.826s	user 0.017s	sys 0.000s
I20260709 13:15:04.945569 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.945808 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:04.956403 22839 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 21.
I20260709 13:15:04.951049 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.951985 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:04.960479 22909 raft_consensus.cc:2368] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19.
I20260709 13:15:04.919240 30969 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:04.918336 30862 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:04.935832 30998 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:04.930042 30977 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:04.962539 22911 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:04.972517 22664 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 7 to 20, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.946547 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:04.932976 31023 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:04.928894 31026 raft_consensus.cc:697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:04.967507 22907 raft_consensus.cc:1240] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
W20260709 13:15:04.968004 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66065.3us
W20260709 13:15:05.002965 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.951442 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:05.005489 30978 consensus_queue.cc:1048] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:04.974329 22841 raft_consensus.cc:2410] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
W20260709 13:15:04.974280 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 97869.3us
W20260709 13:15:04.976856 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68921.9us
I20260709 13:15:04.917273 31002 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:04.989895 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 89586.4us
W20260709 13:15:05.007776 22730 connection.cc:256] RPC call timeout handler was delayed by -2.1518201459998636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:15:05.008972 22880 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.009457 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:05.009019 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.011914 22806 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:05.013973 22804 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:05.014212 23431 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.015659 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.016400 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.017017 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.014014 22806 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.018123 22880 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.019209 22880 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.019639 22880 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.020596 30939 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:15:05.020920 22880 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.021514 30939 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:15:05.023386 22730 connection.cc:256] RPC call timeout handler was delayed by -2.0232734159999382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
W20260709 13:15:05.023584 22806 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.023831 22730 connection.cc:256] RPC call timeout handler was delayed by -1.9448201459997563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 13:15:05.024536 22806 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.028316 22730 connection.cc:256] RPC call timeout handler was delayed by -1.5465637069999048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
I20260709 13:15:05.025256 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:05.027266 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.050040 22880 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.051029 22806 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.042173 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 86738.4us
W20260709 13:15:05.048969 22730 connection.cc:256] RPC call timeout handler was delayed by -1.4833014899998034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.36010000000000003s for a response
W20260709 13:15:05.054795 22730 connection.cc:256] RPC call timeout handler was delayed by -1.3316668379998191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 13:15:05.055924 22730 connection.cc:256] RPC call timeout handler was delayed by -1.3203169639998578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2641s for a response
W20260709 13:15:05.059111 22806 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.958671 31051 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:04.968878 30971 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:05.064262 23431 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:04.921437 30896 consensus_queue.cc:260] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:05.070331 31049 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 13:15:05.071892 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.064852 22664 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:05.134434 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:04.970623 31028 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.157485 31028 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:05.088212 30729 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.158671 30729 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:04.960984 30963 consensus_queue.cc:260] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.076884 31051 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:05.161494 30729 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.162912 31051 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.164382 30961 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.940754 31031 consensus_queue.cc:237] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:05.172724 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 108491us
W20260709 13:15:05.169881 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 109081us
I20260709 13:15:05.172791 22664 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:05.182780 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:05.176669 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.183624 22914 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 25.
I20260709 13:15:05.184132 30939 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:05.177553 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:05.184916 30939 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:05.186097 22806 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.186959 22806 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.187518 30939 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:05.190006 22730 connection.cc:256] RPC call timeout handler was delayed by -1.1687668379997831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
I20260709 13:15:05.192312 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:15:05.194041 22730 connection.cc:256] RPC call timeout handler was delayed by -1.1538169639998159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2826s for a response
W20260709 13:15:05.189821 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 136399us
W20260709 13:15:05.190197 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 223969us
I20260709 13:15:05.105798 30996 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:05.108456 30998 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.095849 30858 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.167050 30961 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.221535 22879 leader_election.cc:400] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:15:05.237007 22879 leader_election.cc:403] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:15:05.218751 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:05.230939 31022 log.cc:927] Time spent T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:15:05.232254 31056 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:15:05.181252 31028 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.242154 31074 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:05.241855 31028 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.151051 30980 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095)
I20260709 13:15:05.086516 30971 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.257690 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:05.243844 22664 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:05.261933 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39296 (request call id 2976) took 1187 ms. Trace:
I20260709 13:15:05.271531 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:05.262447 22841 rpcz_store.cc:276] 0709 13:15:04.066616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.073083 (+  6467us) service_pool.cc:224] Handling call
0709 13:15:05.253853 (+1180770us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54000c6cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:05.254024 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891563776}
I20260709 13:15:05.239730 30827 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:05.290778 22880 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.298425 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:05.315573 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 96925.2us
W20260709 13:15:05.303500 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.331409 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.272588 30980 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:05.332434 22664 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:15:05.319734 22880 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.100778 31050 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.362108 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:05.373234 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53707.8us
I20260709 13:15:05.247704 31038 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:05.382524 22880 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.346351 30937 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:05.365159 30980 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:05.387615 31049 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:15:05.384047 31012 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:05.254277 31026 consensus_queue.cc:237] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:05.409658 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 214463us
I20260709 13:15:05.422636 30996 consensus_queue.cc:1059] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:05.424705 30996 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:05.429586 31074 consensus_queue.cc:1059] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:05.430996 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:05.410348 31084 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:15:05.432796 31028 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:05.434296 31038 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:05.411103 31060 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
W20260709 13:15:05.411763 31061 log.cc:927] Time spent T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:15:05.411405 22664 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 15 to 20, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:05.414500 30971 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.415591 30961 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:05.400714 31007 raft_consensus.cc:3055] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:05.400132 31051 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.451869 31051 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.453051 31023 raft_consensus.cc:3055] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:05.454380 31051 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.425462 30996 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:05.455322 30996 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.455796 30996 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.402313 31041 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.457813 31007 raft_consensus.cc:740] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:05.458192 31048 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 13:15:05.399760 31075 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.458840 31007 consensus_queue.cc:260] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.460186 31007 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:05.442684 31077 consensus_queue.cc:1059] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 13:15:05.465775 23462 consensus_meta.cc:287] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.003s
I20260709 13:15:05.466733 23462 raft_consensus.cc:2468] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:15:05.406443 30980 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:05.369314 30998 consensus_queue.cc:1048] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:15:05.478734 22664 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 8 to 19. New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:05.479429 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:05.489359 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 107299us
I20260709 13:15:05.497933 31075 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.496956 31041 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:05.535023 31086 log.cc:927] Time spent T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:15:05.536870 31080 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:15:05.466367 31023 raft_consensus.cc:740] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:15:05.537453 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.469996 31037 consensus_queue.cc:1059] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:15:05.432837 31084 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 13:15:05.556674 31085 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.605085 31084 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:05.605952 31084 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:05.606657 31023 consensus_queue.cc:260] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.607020 31084 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.608510 31023 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:05.564218 31070 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:05.567404 31079 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.487231 22907 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
W20260709 13:15:05.574929 22880 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.584285 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3960) took 1308 ms. Trace:
I20260709 13:15:05.594409 22910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43286 (request call id 2613) took 1426 ms. Trace:
W20260709 13:15:05.639358 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64516.8us
W20260709 13:15:05.640228 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:05.640906 22880 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.597085 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:05.639529 22910 rpcz_store.cc:276] 0709 13:15:04.168291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.168470 (+   179us) service_pool.cc:224] Handling call
0709 13:15:05.594053 (+1425583us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540010efd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:05.594285 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854648064}
I20260709 13:15:05.602545 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40710 (request call id 998) took 1265 ms. Trace:
I20260709 13:15:05.602713 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:05.605067 31085 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.608705 31084 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:05.556641 30996 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.609249 31068 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:05.609591 30827 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) with higher term. Message: Invalid argument: T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for earlier term 21. Current term is 22.
I20260709 13:15:05.575453 30980 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.646914 31068 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.648856 30980 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:05.613768 31078 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:05.649756 31078 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:05.614296 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 27 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:05.654894 22838 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:15:05.657806 31079 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.658573 22914 raft_consensus.cc:2468] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 22.
I20260709 13:15:05.661291 31070 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:05.661832 31070 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:05.662439 22907 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:15:05.662744 31070 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.663170 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:05.663704 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:05.664059 22914 raft_consensus.cc:2393] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 22: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:05.664572 31053 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.433s	user 0.003s	sys 0.000s
I20260709 13:15:05.559484 30998 raft_consensus.cc:3055] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:05.664839 31078 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.665536 30998 raft_consensus.cc:740] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:05.665684 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:05.666597 22914 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:05.666893 30998 consensus_queue.cc:260] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.667619 31068 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.668383 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:05.633206 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 447140us
I20260709 13:15:05.669114 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:05.578790 30981 raft_consensus.cc:3055] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 29 LEADER]: Stepping down as leader of term 29
I20260709 13:15:05.669667 30981 raft_consensus.cc:740] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:05.670181 31079 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:05.664393 22907 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:05.670562 30981 consensus_queue.cc:260] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.671209 31041 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:05.671948 30981 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:15:05.672559 22912 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:05.620138 22805 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:05.635835 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.641541 22807 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:05.669292 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.643033 22833 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:05.675541 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.677304 31078 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:05.645337 31085 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:05.679273 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.664445 31070 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:05.668519 30998 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:05.615854 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:05.644062 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:05.642761 23465 rpcz_store.cc:276] 0709 13:15:04.332811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.332957 (+   146us) service_pool.cc:224] Handling call
0709 13:15:04.350220 (+ 17263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:04.355549 (+  5329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:04.355587 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:04.673146 (+317559us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b2c00040ad8. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:15:04.683545 (+ 10399us) log.cc:844] Serialized 26 byte log entry
0709 13:15:04.851953 (+168408us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:15:04.851991 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.852065 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.583891 (+731826us) raft_consensus.cc:1685] finished
0709 13:15:05.583916 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.598152 (+ 14236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38535,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":32565632,"thread_start_us":130730,"threads_started":1,"wal-append.queue_time_us":599798}
W20260709 13:15:05.679952 22832 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.026s
W20260709 13:15:05.684422 22832 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.008s	sys 0.026s
W20260709 13:15:05.689788 22880 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.690515 22880 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.683753 23457 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
W20260709 13:15:05.693288 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 51889.3us
W20260709 13:15:05.693336 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 73336.9us
I20260709 13:15:05.695827 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:05.696197 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:05.639051 22913 rpcz_store.cc:276] 0709 13:15:04.275358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.459804 (+184446us) service_pool.cc:224] Handling call
0709 13:15:04.479899 (+ 20095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:04.496303 (+ 16404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:04.496339 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:04.514712 (+ 18373us) log.cc:844] Serialized 26 byte log entry
0709 13:15:04.724873 (+210161us) raft_consensus.cc:1645] Marking committed up to 22
0709 13:15:04.735377 (+ 10504us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.765929 (+ 30552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.766017 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.583980 (+817963us) raft_consensus.cc:1685] finished
0709 13:15:05.584012 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.584192 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41584,"prepare.queue_time_us":10569,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":46875,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":14277,"thread_start_us":96695,"threads_started":1,"wal-append.queue_time_us":204855}
I20260709 13:15:05.692847 30980 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:05.707051 22880 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.714910 22840 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
W20260709 13:15:05.715837 22880 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.716276 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.716173 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:05.717202 22840 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:05.717366 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:05.707602 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:05.714290 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45035.8us
I20260709 13:15:05.714371 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:05.718712 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.718899 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 75783.1us
I20260709 13:15:05.719308 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.719731 22832 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:15:05.717566 22880 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.724764 22880 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.706235 30362 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:05.726099 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:05.726295 30362 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.727099 23457 raft_consensus.cc:2393] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 26: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:05.728515 22732 leader_election.cc:400] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19.
I20260709 13:15:05.730854 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:05.713014 22907 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
I20260709 13:15:05.731859 23462 raft_consensus.cc:2393] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 25: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:05.745352 22833 raft_consensus.cc:2468] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 19.
I20260709 13:15:05.746371 22732 leader_election.cc:403] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:05.730077 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:05.754665 22912 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
W20260709 13:15:05.754959 22880 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.755702 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.764164 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:05.702127 31085 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:05.770905 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:05.771071 22880 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.706346 30996 raft_consensus.cc:2804] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:05.774552 23464 raft_consensus.cc:2410] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:15:05.773416 22838 raft_consensus.cc:2468] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 29.
W20260709 13:15:05.775002 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:05.728068 31079 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:05.826377 30963 raft_consensus.cc:3060] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:05.828851 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:05.784605 31083 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:05.733093 30980 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.830505 30986 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:05.831405 31079 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.831557 30986 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:05.832130 30986 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:05.791059 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:05.834829 23462 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:15:05.797003 30362 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:05.787089 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:05.798502 22839 raft_consensus.cc:1275] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:15:05.848686 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61687.3us
I20260709 13:15:05.785000 31068 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:15:05.785404 31053 log.cc:927] Time spent T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:05.789716 31050 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:15:05.805718 31073 consensus_queue.cc:1059] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:15:05.806577 31076 raft_consensus.cc:3060] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:05.807663 31075 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:05.751423 30853 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:05.808945 31002 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:05.752367 31012 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:05.819207 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:05.822022 30996 raft_consensus.cc:697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:05.752610 31007 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:15:05.752133 31037 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:15:05.832180 30980 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:05.831934 31079 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:05.851902 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:05.849905 22880 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.847496 30896 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:05.786943 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:05.896414 22838 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:15:05.896028 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:05.867980 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:05.869714 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:05.882539 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.883426 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:05.871402 31084 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:05.884454 30996 consensus_queue.cc:237] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:05.885888 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57168.1us
I20260709 13:15:05.887813 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:05.888939 31036 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:05.856297 31068 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.894007 31075 raft_consensus.cc:3055] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:15:05.898784 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49379.5us
I20260709 13:15:05.901371 22832 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:15:05.913079 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:05.913905 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62115.7us
W20260709 13:15:05.926983 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.934294 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:05.935292 22880 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.936125 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.895339 31050 consensus_queue.cc:1243] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:15:05.939884 22880 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.920295 30999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.029s	sys 0.033s
I20260709 13:15:05.943205 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.938436 22879 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:05.945328 23462 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:15:05.969271 22833 raft_consensus.cc:1275] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:15:05.923866 30853 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:05.907819 31084 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.894802 31007 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:05.927218 31078 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:15:05.971192 22880 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:05.984288 22880 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.984962 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:05.971105 22879 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:05.986946 22880 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.987519 22880 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:05.965023 31080 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:05.927469 31075 raft_consensus.cc:740] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:05.971921 22732 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:05.978797 22840 raft_consensus.cc:1275] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:05.968612 31043 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:05.959980 31079 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:05.939651 30975 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:05.861622 30986 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:05.964450 30827 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:15:05.969956 30999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.029s	sys 0.034s
I20260709 13:15:05.972632 22730 leader_election.cc:400] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
I20260709 13:15:06.029886 22730 leader_election.cc:403] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 13:15:05.985399 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 223415us
I20260709 13:15:05.968977 31026 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:06.001734 22880 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:05.976008 30972 raft_consensus.cc:2804] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:06.012706 30974 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.017628 31028 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:05.981020 31084 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:05.983088 30937 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:06.028585 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3962) took 1752 ms. Trace:
I20260709 13:15:05.983227 31078 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.999308 31044 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.035681 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:06.019398 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 48383.5us
W20260709 13:15:06.039675 31055 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:15:06.039898 31075 consensus_queue.cc:260] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.017364 31043 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.019129 31036 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:06.026523 30853 raft_consensus.cc:2804] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:15:06.037902 30827 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:06.045426 22906 rpcz_store.cc:276] 0709 13:15:04.275928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.587860 (+311932us) service_pool.cc:224] Handling call
0709 13:15:04.614619 (+ 26759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:04.615115 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:15:04.662619 (+ 47504us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.906948 (+244329us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:04.907819 (+   871us) log.cc:844] Serialized 26 byte log entry
0709 13:15:04.907979 (+   160us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:15:04.913951 (+  5972us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.914269 (+   318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.914370 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.416769 (+502399us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b180035e660. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:05.432836 (+ 16067us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b180035e660. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:05.663744 (+230908us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:15:05.822936 (+159192us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b180035e660. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:06.028346 (+205410us) raft_consensus.cc:1685] finished
0709 13:15:06.028370 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.028519 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25023,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":107,"raft.queue_time_us":34226,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":20756,"spinlock_wait_cycles":50496128,"thread_start_us":212106,"threads_started":1,"wal-append.queue_time_us":391971}
W20260709 13:15:06.118769 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 117098us
I20260709 13:15:06.117455 31043 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:06.121858 30432 raft_consensus.cc:3060] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:06.122049 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:06.115821 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.155916 22880 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.118031 31078 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:06.122578 30862 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.164999 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:06.170589 22835 raft_consensus.cc:1275] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:15:06.159582 30986 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:06.124699 30653 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:06.116721 30972 raft_consensus.cc:697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:15:06.174356 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 207040us
W20260709 13:15:06.158159 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 295738us
I20260709 13:15:06.121181 31011 consensus_queue.cc:1048] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:15:06.164579 22880 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.148445 31036 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:06.148739 30853 raft_consensus.cc:697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 29 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:06.164984 31075 raft_consensus.cc:3060] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:15:06.177295 30981 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.505s	user 0.014s	sys 0.000s
I20260709 13:15:06.177028 30858 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:06.193007 31079 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:06.194051 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.194098 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.176736 31026 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:06.117684 31007 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.203022 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:06.210572 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:06.214567 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:06.188050 31025 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.217186 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:06.221060 23458 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:06.222033 22906 raft_consensus.cc:1240] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 25. Current term is 26. Ops: []
I20260709 13:15:06.179606 30862 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.201859 31026 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.218101 30978 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.218923 30974 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:06.224527 31037 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:06.219823 31076 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19.
I20260709 13:15:06.212064 30980 consensus_queue.cc:1048] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:15:06.212568 31028 consensus_queue.cc:1048] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:15:06.240106 31074 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:06.242844 31026 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:06.213225 30853 consensus_queue.cc:237] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 27.10, Last appended by leader: 10, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:06.278018 31088 log.cc:927] Time spent T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:15:06.296393 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:06.297360 31091 log.cc:927] Time spent T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:15:06.313835 31037 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:06.298086 31089 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:15:06.314504 31037 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.316829 31037 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:06.317638 30972 consensus_queue.cc:237] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.318846 30862 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:06.303375 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 152429us
W20260709 13:15:06.323449 22880 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.323899 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:06.324115 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.328549 22880 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.329555 22880 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.329968 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.331112 23458 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 26.
I20260709 13:15:06.229924 31025 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.346648 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:06.331552 31093 log.cc:927] Time spent T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:15:06.331709 31082 log.cc:927] Time spent T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:15:06.321833 30729 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.232118 30827 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.366686 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:06.373395 22910 raft_consensus.cc:3060] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:06.367180 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:06.273479 30978 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.343396 31079 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.380203 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:06.384506 23432 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:06.385620 31104 raft_consensus.cc:2804] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:06.386171 31104 raft_consensus.cc:697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:06.387307 31104 consensus_queue.cc:237] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.394527 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:06.396268 31105 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:06.397825 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:06.342480 30858 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:06.343541 31011 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.401968 31011 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.401525 30858 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.404440 31011 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:06.408408 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 112117us
I20260709 13:15:06.272614 31036 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.410224 22910 raft_consensus.cc:2468] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 26.
I20260709 13:15:06.360644 30896 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.406243 31109 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:06.413532 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:06.421321 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.438364 31101 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 13:15:06.423113 31095 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:15:06.440873 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 102074us
I20260709 13:15:06.443095 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:06.446707 23432 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:06.430222 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:06.446970 22880 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.448024 22880 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.453253 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:06.447703 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.454650 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.455698 22911 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:15:06.378216 30729 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.451725 23432 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:06.460757 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:06.463866 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:06.465301 22910 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
W20260709 13:15:06.465848 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.466724 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.470238 31111 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:15:06.471359 31094 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:15:06.472126 22906 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:15:06.470567 31110 log.cc:927] Time spent T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:15:06.471295 31070 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.474475 22910 raft_consensus.cc:1240] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [22.13-24.14]
I20260709 13:15:06.476763 22915 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:06.412426 31079 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:06.480005 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.494889 22880 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.495448 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.453756 31105 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:06.502120 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 49022.6us
I20260709 13:15:06.505647 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:06.506402 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:06.507606 22833 raft_consensus.cc:2410] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:15:06.462364 31031 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:06.512296 22914 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:06.459903 30896 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:06.517103 31069 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:06.512491 30972 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:06.523062 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57260.2us
I20260709 13:15:06.484004 30980 raft_consensus.cc:2804] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:06.473579 31012 consensus_queue.cc:1048] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:15:06.503835 30862 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.511231 31111 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:06.513326 31037 raft_consensus.cc:3060] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:06.505815 31076 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:06.545778 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 50927.9us
W20260709 13:15:06.531014 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.550894 22880 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.496613 31070 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:06.553922 22880 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.554117 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.560828 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:06.560909 22880 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.568326 22806 leader_election.cc:336] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.569206 22806 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.549028 30980 raft_consensus.cc:697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:15:06.572270 31099 log.cc:927] Time spent T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.283s	user 0.001s	sys 0.001s
I20260709 13:15:06.543718 30896 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 13:15:06.604199 31079 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:06.610638 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2995) took 1013 ms. Trace:
I20260709 13:15:06.611150 22837 rpcz_store.cc:276] 0709 13:15:05.596868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.598393 (+  1525us) service_pool.cc:224] Handling call
0709 13:15:05.616649 (+ 18256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:05.661015 (+ 44366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:05.661053 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:05.662294 (+  1241us) log.cc:844] Serialized 26 byte log entry
0709 13:15:06.031172 (+368878us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:15:06.031234 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.306095 (+274861us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b54001ff4d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:06.306168 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:06.610265 (+304097us) raft_consensus.cc:1685] finished
0709 13:15:06.610290 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.610554 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14816,"raft.queue_time_us":15609,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":114,"spinlock_wait_cycles":66734848,"thread_start_us":352148,"threads_started":1,"wal-append.queue_time_us":502053}
W20260709 13:15:06.617102 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.556926 31076 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.564383 22880 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:06.633613 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72729us
I20260709 13:15:06.621250 30729 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:06.580878 30996 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:06.630595 30963 consensus_meta.cc:287] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.802s	user 0.013s	sys 0.001s
W20260709 13:15:06.645128 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:06.652580 31043 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.657919 31025 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:06.659565 31114 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:15:06.696043 31121 log.cc:927] Time spent T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:15:06.660317 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.637318 30858 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:15:06.634501 31084 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.007s
W20260709 13:15:06.581581 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.647825 30963 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:06.645323 30980 consensus_queue.cc:237] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.650051 30653 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:06.719360 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 200076us
W20260709 13:15:06.683303 30432 consensus_meta.cc:287] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.003s
W20260709 13:15:06.683701 30998 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.983s	user 0.015s	sys 0.000s
W20260709 13:15:06.683852 22880 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.683956 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:06.702170 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 120669us
W20260709 13:15:06.721429 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:06.722370 22806 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.722819 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.715340 22805 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:06.681989 31078 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.002s
I20260709 13:15:06.721168 22908 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
W20260709 13:15:06.728881 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.731832 22806 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.732396 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:06.734249 22806 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.734787 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.726742 22805 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:06.741238 22880 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:06.756560 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 259203us
I20260709 13:15:06.758697 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:06.712811 30963 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:06.720080 30996 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:06.757738 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:06.769357 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:06.686661 31026 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.726929 30432 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
I20260709 13:15:06.727743 30937 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:06.750485 22880 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.744678 31023 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.743597 31079 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.774251 22837 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:15:06.744338 31065 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:06.744814 30999 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:06.778705 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:06.764732 31083 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:06.717633 31084 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:06.786669 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.748276 31078 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:06.855975 22908 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:06.811079 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
W20260709 13:15:06.809227 31075 consensus_meta.cc:287] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.006s
W20260709 13:15:06.918350 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.784628 31051 raft_consensus.cc:3055] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:06.857275 31083 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:06.790313 31007 raft_consensus.cc:3060] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:06.810547 31076 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:06.813446 31065 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:06.925575 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.808826 31127 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:06.941947 31127 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.942476 31127 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:06.828855 30432 consensus_queue.cc:1059] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:15:06.938644 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.808090 31036 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.128s
I20260709 13:15:06.788358 30937 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.923700 30963 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.957235 30963 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:06.957291 30999 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:06.959564 22806 leader_election.cc:336] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.960074 22806 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.778770 30996 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:06.926138 31051 raft_consensus.cc:740] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:06.936221 31075 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:15:06.966120 22806 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.934497 31135 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:06.967649 30862 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:06.975548 30986 consensus_meta.cc:287] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.796s	user 0.012s	sys 0.000s
W20260709 13:15:06.978613 31134 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:15:06.978906 31026 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:06.980520 31078 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:06.984221 31068 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:06.958598 31065 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:06.987566 31074 consensus_meta.cc:287] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.002s
I20260709 13:15:06.987881 31065 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:06.988175 31074 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:15:06.989889 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 178905us
I20260709 13:15:06.991778 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:06.950529 31131 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:06.995824 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:07.004355 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.964140 30999 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:07.017284 22806 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:06.990687 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:07.024507 22911 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:15:07.026796 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.026684 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:06.967845 31075 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:15:07.033354 31097 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.697s	user 0.000s	sys 0.005s
I20260709 13:15:07.036005 31075 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:07.037331 22841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39296 (request call id 2993) took 1441 ms. Trace:
I20260709 13:15:07.037746 22841 rpcz_store.cc:276] 0709 13:15:05.596276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.597863 (+  1587us) service_pool.cc:224] Handling call
0709 13:15:05.614040 (+ 16177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:05.614457 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:05.614489 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:05.615008 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:15:05.984992 (+369984us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:05.985066 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.985170 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:07.032577 (+1047407us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:15:07.036967 (+  4390us) raft_consensus.cc:1685] finished
0709 13:15:07.036997 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.037259 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":9406720,"thread_start_us":369504,"threads_started":1,"wal-append.queue_time_us":607190}
I20260709 13:15:07.042524 30937 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:07.045737 22806 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.035393 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 256684us
W20260709 13:15:07.051045 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 55323.9us
I20260709 13:15:07.013692 31139 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:07.052505 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:07.064707 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:07.027644 31074 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:07.048112 30998 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.060065 31130 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:15:07.055578 31065 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:07.082558 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.025108 31083 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.090907 30996 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.081898 30998 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.095592 30998 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:06.994488 31135 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.028582 30999 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.109076 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.132469 23465 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 18: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:07.130295 30998 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:07.134421 30998 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.136595 31135 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.079574 30729 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:07.061259 31051 consensus_queue.cc:260] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.130517 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:07.072079 31145 raft_consensus.cc:2804] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:15:07.081063 31037 consensus_meta.cc:287] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.530s	user 0.004s	sys 0.008s
I20260709 13:15:07.140771 22911 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:07.026003 31131 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:07.143507 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.144806 22914 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 13:15:07.148201 22880 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.150137 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 65385.3us
I20260709 13:15:07.152563 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.149009 22880 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.158269 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:07.171861 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:07.143857 30998 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.145112 31145 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.185860 31145 raft_consensus.cc:3060] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:07.195187 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 88809.4us
I20260709 13:15:07.147085 31037 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:07.200911 31142 log.cc:927] Time spent T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:15:07.174185 30729 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:07.174304 31026 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:07.174067 31078 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:07.151367 31084 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:07.184026 31146 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 13:15:07.185105 31051 raft_consensus.cc:3060] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:07.201870 22912 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:07.178622 31127 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.208869 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:07.216996 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.210991 31085 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.223837 31085 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.224856 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:07.227453 30963 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:15:07.210273 31114 log.cc:927] Time spent T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:15:07.221664 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 303260us
I20260709 13:15:07.228389 31075 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.237843 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.238260 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.239153 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.215854 30998 consensus_queue.cc:1059] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:15:07.240988 22908 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:15:07.242573 22880 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.247468 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.239984 30937 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:07.248185 22906 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 21.
I20260709 13:15:07.248389 30937 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.249676 30998 raft_consensus.cc:3055] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:15:07.250120 30998 raft_consensus.cc:740] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:07.251358 30937 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.215919 30986 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:07.252090 30986 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:07.252785 30986 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:07.245334 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.254494 30729 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:07.254827 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.252159 30996 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.257758 30996 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.245944 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.246297 22915 raft_consensus.cc:2393] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 26: Already voted for candidate acfda9b350734def9b6c621f66a5f247 in this term.
I20260709 13:15:07.260458 30996 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.261387 31025 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.261360 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:07.216291 31078 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:07.258342 30999 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.262480 31078 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:07.262233 31025 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.262694 31070 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.262944 22915 raft_consensus.cc:2368] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:07.264200 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:07.264505 31025 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.264855 22913 raft_consensus.cc:3060] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:07.269191 31131 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.270088 31131 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.270578 31139 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.270984 23465 raft_consensus.cc:1275] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:15:07.251076 30998 consensus_queue.cc:260] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.234948 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.279439 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.285053 22834 raft_consensus.cc:1275] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:15:07.286002 22880 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.288341 23465 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 13:15:07.289862 23458 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 30: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:07.291358 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:07.294094 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.295559 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:07.296378 22834 raft_consensus.cc:2368] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19.
W20260709 13:15:07.296108 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 84304.9us
W20260709 13:15:07.297271 22806 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.298276 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.298331 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:07.270685 31131 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.300115 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:07.300271 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.302613 22835 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:07.302664 30978 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.305095 31131 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.295550 30999 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:07.312762 30999 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.300812 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:07.314435 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:07.316185 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.317037 30978 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:07.317185 30999 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.317466 30978 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.326939 30978 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:07.300123 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.329815 23461 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:15:07.331398 23461 raft_consensus.cc:1240] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:07.299160 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:07.334273 30896 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:07.334332 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.334875 30896 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.335317 30896 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:07.335819 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.336438 23463 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:07.336989 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.338441 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.339522 23463 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:07.340695 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.318142 30996 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:07.271198 31139 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:07.295131 22880 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.342114 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47039.5us
W20260709 13:15:07.325975 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.378660 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.339679 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:07.350767 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.376103 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:07.374493 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.382248 22907 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 19.
W20260709 13:15:07.347910 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54819.4us
I20260709 13:15:07.383860 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.379426 22909 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:15:07.377609 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:07.387074 22730 connection.cc:256] RPC call timeout handler was delayed by -1.2350976709999486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 13:15:07.284545 31007 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:07.354044 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:07.387364 22840 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 13:15:07.401826 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 47921.9us
W20260709 13:15:07.387893 23431 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.559s (SENT)
I20260709 13:15:07.403721 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.344238 22732 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:07.405102 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 60963.8us
W20260709 13:15:07.397389 22730 connection.cc:256] RPC call timeout handler was delayed by -1.1801976710000872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 13:15:07.399312 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.381865 22836 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 30.
W20260709 13:15:07.405337 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.388270 22835 raft_consensus.cc:2468] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:15:07.276927 31084 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:07.411410 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.412776 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.414273 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:07.258286 31075 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:07.416263 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.424067 22832 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:07.428397 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:07.387637 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:07.420622 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:07.432026 22839 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 13:15:07.432965 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:07.364880 31070 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:07.433249 22906 raft_consensus.cc:2393] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 22: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:07.438905 23430 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:07.434624 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:07.439571 22839 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:15:07.416183 31144 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:07.386191 31068 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:07.435719 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:07.450048 22839 raft_consensus.cc:1275] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:15:07.450186 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 59928.5us
I20260709 13:15:07.295300 30963 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.138s
W20260709 13:15:07.451792 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 125862us
I20260709 13:15:07.416940 30432 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:07.430166 30999 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:07.429802 31054 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:07.469635 31054 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:07.457859 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.460434 22913 raft_consensus.cc:2468] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
W20260709 13:15:07.470824 22806 leader_election.cc:336] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.461633 22839 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 25. Current term is 26. Ops: []
I20260709 13:15:07.453330 22807 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:07.467281 22832 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 20.
W20260709 13:15:07.521989 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.430428 31083 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:07.471304 22806 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:07.530239 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76980.6us
W20260709 13:15:07.537945 23431 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.713s (SENT)
I20260709 13:15:07.469918 31127 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:07.541833 23431 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.742s (SENT)
W20260709 13:15:07.543982 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 73195.9us
W20260709 13:15:07.544783 22806 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.545740 22806 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.546304 22806 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.549048 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:07.547372 31083 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.470084 31054 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:07.554988 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 116192us
I20260709 13:15:07.456779 30998 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:07.456966 31065 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:07.431775 31084 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:07.591879 23431 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.809s (SENT)
I20260709 13:15:07.472703 31048 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 13:15:07.602208 31048 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.603744 31048 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Advancing to term 31
W20260709 13:15:07.526327 31094 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:15:07.554144 31023 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:07.529923 30432 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:07.610935 30432 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.563500 31139 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.553308 30999 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:07.615496 30999 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.616276 30999 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:07.459270 31007 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.129s
I20260709 13:15:07.560937 31131 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.662019 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.458993 31068 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:07.668872 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.888s (SENT)
I20260709 13:15:07.669584 23431 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:07.673367 23431 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.897s (SENT)
I20260709 13:15:07.673925 23431 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.594395 31065 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.632416 31127 raft_consensus.cc:3060] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:07.632073 30937 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.706606 22835 raft_consensus.cc:2410] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 13:15:07.673777 31025 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:15:07.716142 23431 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.915s (SENT)
I20260709 13:15:07.679332 31065 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:07.707866 31105 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:07.715027 30937 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.733997 30937 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:07.735513 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 186611us
I20260709 13:15:07.675983 31131 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:07.740199 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:07.738075 31051 consensus_meta.cc:287] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:15:07.760777 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:07.735978 31145 consensus_meta.cc:287] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.000s
W20260709 13:15:07.772488 21636 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:07.745689 31083 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:07.784240 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:07.786429 23431 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.010s (SENT)
I20260709 13:15:07.787353 23431 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:07.808463 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 95986.4us
I20260709 13:15:07.765578 30963 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.831099 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:07.832966 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:07.810174 31051 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:07.806943 31105 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.839619 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.766629 31078 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.770026 31145 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.855316 31026 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:07.856882 31026 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:15:07.858914 31028 consensus_queue.cc:1048] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:07.862802 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:07.864109 30937 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:15:07.865167 31028 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:15:07.868455 22879 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.870374 31028 raft_consensus.cc:2804] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:15:07.871421 31028 raft_consensus.cc:697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:07.872742 31028 consensus_queue.cc:237] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.879531 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:07.884057 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 51215.2us
I20260709 13:15:07.885846 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:07.838899 31160 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:07.895947 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:07.895659 30963 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:07.898481 31154 log.cc:927] Time spent T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:15:07.852249 31144 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:07.854562 31084 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:07.894068 31153 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 13:15:07.930945 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:07.877575 31157 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:07.933552 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54236.3us
I20260709 13:15:07.846397 31037 consensus_queue.cc:1059] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
W20260709 13:15:07.936823 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:07.941972 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:07.944701 23431 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.945479 23431 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:07.942224 31130 consensus_meta.cc:287] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.844s	user 0.010s	sys 0.005s
I20260709 13:15:07.940623 30975 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:07.950410 23465 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:15:07.945924 23431 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:07.924216 30963 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:15:07.957899 22841 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.012s	sys 0.014s
W20260709 13:15:07.959281 22841 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.013s	sys 0.014s
I20260709 13:15:07.908118 31159 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:07.979017 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:07.957404 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.985039 31131 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:07.955322 30853 consensus_queue.cc:1059] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
W20260709 13:15:07.948508 31151 log.cc:927] Time spent T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:15:07.986465 30853 consensus_queue.cc:1243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 13:15:07.988047 30853 raft_consensus.cc:3055] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:15:07.907315 31160 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:07.954988 31105 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.946802 30858 consensus_queue.cc:1048] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:15:07.949680 31144 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:15:07.959664 30896 consensus_meta.cc:287] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.005s
I20260709 13:15:07.940771 31051 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:07.988746 30853 raft_consensus.cc:740] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:15:07.981741 31150 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.265s	user 0.001s	sys 0.001s
I20260709 13:15:07.986791 22833 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1447) took 1237 ms. Trace:
I20260709 13:15:08.010753 22833 rpcz_store.cc:276] 0709 13:15:06.749061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.750301 (+  1240us) service_pool.cc:224] Handling call
0709 13:15:06.770551 (+ 20250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.918667 (+148116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:06.918718 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:06.918785 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:06.918831 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.986530 (+1067699us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400322dd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:07.986581 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.986726 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339746176}
W20260709 13:15:07.986153 23431 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:08.014437 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:07.975622 31158 log.cc:927] Time spent T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:15:08.019482 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:08.021718 22805 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:08.022239 23465 raft_consensus.cc:2410] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 28 because replica has last-logged OpId of term: 25 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 19.
I20260709 13:15:07.952108 31130 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:08.025060 22910 raft_consensus.cc:1275] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:15:08.026458 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:08.006840 30986 consensus_meta.cc:287] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.753s	user 0.012s	sys 0.002s
I20260709 13:15:08.031585 30986 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.031908 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:07.989794 31128 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 1.068s	user 0.004s	sys 0.006s
I20260709 13:15:08.033739 31177 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:07.992844 30896 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.038944 31084 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:08.041332 31036 raft_consensus.cc:3055] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:08.041826 31036 raft_consensus.cc:740] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:08.049728 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:07.989471 30937 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:15:08.056053 30978 consensus_meta.cc:287] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.005s
I20260709 13:15:08.059082 22906 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 30.
I20260709 13:15:08.062856 22805 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:08.036877 31048 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.069986 30937 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.094450 22807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.035s
W20260709 13:15:08.094915 22807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.007s	sys 0.035s
W20260709 13:15:08.095209 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 551094us
I20260709 13:15:08.056667 31036 consensus_queue.cc:260] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.096688 22807 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:08.099023 23463 raft_consensus.cc:2393] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 31: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:08.065282 30978 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:08.102957 22732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.010s	sys 0.009s
W20260709 13:15:08.103402 22732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.010s	sys 0.010s
I20260709 13:15:08.065958 31131 consensus_queue.cc:1048] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
W20260709 13:15:08.103682 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 697540us
I20260709 13:15:08.107170 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:08.077657 31084 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 13:15:08.118423 31164 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:08.112452 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:08.119855 30996 consensus_meta.cc:287] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.001s
I20260709 13:15:08.120285 30996 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:08.113188 30963 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:08.114777 22836 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:08.108567 31069 raft_consensus.cc:2804] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:08.121377 31069 raft_consensus.cc:697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:08.122654 31036 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:08.122515 31069 consensus_queue.cc:237] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.129408 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:08.130986 22839 raft_consensus.cc:2368] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:08.131178 22732 leader_election.cc:400] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:08.131592 22732 leader_election.cc:403] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:15:08.136798 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:08.137079 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:08.136880 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:08.137699 22841 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:08.138331 22911 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 23.
W20260709 13:15:08.126487 30729 consensus_meta.cc:287] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.871s	user 0.016s	sys 0.000s
I20260709 13:15:08.096143 30937 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.146648 22833 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:08.158232 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:08.168830 31174 log.cc:927] Time spent T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:15:08.147123 31157 raft_consensus.cc:3060] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:08.162925 31075 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.176443 31075 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.186187 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:08.194183 23457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1425) took 1479 ms. Trace:
I20260709 13:15:08.194885 23457 rpcz_store.cc:276] 0709 13:15:06.715095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.715335 (+   240us) service_pool.cc:224] Handling call
0709 13:15:06.726499 (+ 11164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:07.231777 (+505278us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:07.231815 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:07.231862 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:07.231895 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.193751 (+961856us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b5400603454. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:08.193825 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:08.194089 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161305472}
I20260709 13:15:08.160188 30729 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:08.164003 30432 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.218932 31181 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:08.218964 31068 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:08.223158 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:08.223853 23457 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:08.229060 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:08.235477 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:08.240515 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:08.240746 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:08.199739 30862 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:08.246186 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:08.253563 30862 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.255050 30937 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:08.256985 31176 raft_consensus.cc:493] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:08.257732 31176 raft_consensus.cc:515] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.260969 31075 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:08.262646 31048 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.263348 30978 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.212481 31065 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.264461 31160 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:08.265478 30999 consensus_meta.cc:287] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.648s	user 0.014s	sys 0.000s
I20260709 13:15:08.266023 30999 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:08.268813 31127 consensus_meta.cc:287] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.003s
I20260709 13:15:08.248754 31083 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:08.236193 30432 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.273785 31083 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:08.274456 31083 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:08.275040 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:08.276047 30432 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:08.251047 31070 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:15:08.269585 30862 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:08.214671 31165 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.268879 30999 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.322860 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:08.288347 31127 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:08.266052 31065 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.261214 31176 leader_election.cc:290] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:08.323578 31085 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.374178 23457 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:08.382974 22835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2532) took 1011 ms. Trace:
I20260709 13:15:08.384006 31051 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:08.384383 22838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46858 (request call id 1452) took 1049 ms. Trace:
I20260709 13:15:08.384774 22838 rpcz_store.cc:276] 0709 13:15:07.335325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.340835 (+  5510us) service_pool.cc:224] Handling call
0709 13:15:08.119986 (+779151us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b58000ff098. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:15:08.382841 (+262855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:08.383068 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:08.383093 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:08.383138 (+    45us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:15:08.383167 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.383223 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:08.384192 (+   969us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400411ed0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af08ad 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:08.384323 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614810112}
I20260709 13:15:08.383638 22835 rpcz_store.cc:276] 0709 13:15:07.371187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.431158 (+ 59971us) service_pool.cc:224] Handling call
0709 13:15:08.382658 (+951500us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b5400411ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:08.382891 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612793216}
I20260709 13:15:08.322521 31165 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.415860 31105 consensus_queue.cc:1048] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:15:08.421043 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:08.378137 31078 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.441347 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.442318 22908 raft_consensus.cc:3060] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:15:08.405333 30896 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.460497 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.461527 22906 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:08.443120 30827 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:15:08.456173 31152 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.009s	sys 0.021s
I20260709 13:15:08.485339 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:08.486433 22913 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
W20260709 13:15:08.491076 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:08.480643 31178 log.cc:927] Time spent T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
W20260709 13:15:08.500916 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 254800us
W20260709 13:15:08.482129 31152 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.009s	sys 0.022s
I20260709 13:15:08.474961 31164 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:08.514241 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.515599 22913 raft_consensus.cc:2393] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 21: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:08.523083 31145 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.525162 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.525939 22913 raft_consensus.cc:3060] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:08.529618 22653 maintenance_manager.cc:419] P 6e189717484b49e8bb20c98667e6d294: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033372
I20260709 13:15:08.536787 22906 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:08.484458 30827 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.548272 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.563267 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:08.564735 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:08.577646 22913 raft_consensus.cc:2468] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 22.
I20260709 13:15:08.596854 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:08.624176 22807 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:08.630662 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 137031us
I20260709 13:15:08.579893 30896 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:08.635864 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:08.573366 31145 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:08.579108 31078 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.639770 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.642297 22913 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:08.644187 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.644552 22807 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:08.640311 22730 leader_election.cc:400] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 18. Current term is 19.
I20260709 13:15:08.602288 31157 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233) with higher term. Message: Invalid argument: T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:08.673569 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:08.687592 22908 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 31.
I20260709 13:15:08.694336 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:08.665163 31054 consensus_meta.cc:287] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent flushing consensus metadata: real 1.032s	user 0.010s	sys 0.003s
I20260709 13:15:08.731688 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:08.732436 22908 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:08.640391 31162 raft_consensus.cc:2804] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:08.735288 22913 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:08.736271 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:08.773237 21636 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:08.665055 31037 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:08.641769 31145 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.663662 31130 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.673781 30896 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:08.801947 30862 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:08.801647 31176 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.820722 22880 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 127 similar messages]
I20260709 13:15:08.827230 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:08.828105 22833 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:08.704715 31085 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:08.854630 31192 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:15:08.783749 31165 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:08.668205 31078 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.782823 31026 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:08.814045 31145 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:08.886700 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:08.887816 22836 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 20.
I20260709 13:15:08.876744 31163 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:08.890196 31163 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.890702 31051 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.679297 31159 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.158s
I20260709 13:15:08.836015 31176 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:08.897745 31128 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:08.898510 30827 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:08.892514 31051 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:08.893644 31202 log.cc:927] Time spent T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:08.858922 23432 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:15:08.922405 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 63680.5us
W20260709 13:15:08.889341 31195 log.cc:927] Time spent T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:15:08.822224 31162 raft_consensus.cc:697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
W20260709 13:15:08.915971 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 242452us
W20260709 13:15:08.914685 22880 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:08.927497 22880 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:08.932296 22908 raft_consensus.cc:2468] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 20.
W20260709 13:15:08.900024 31177 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.865s	user 0.015s	sys 0.000s
I20260709 13:15:08.939415 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:08.940069 22839 raft_consensus.cc:1275] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 23 index: 9. Preceding OpId from leader: term: 29 index: 11. (index mismatch)
I20260709 13:15:08.940246 22836 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:08.933665 22730 leader_election.cc:400] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25.
I20260709 13:15:08.934281 22833 raft_consensus.cc:2468] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
W20260709 13:15:08.900718 31184 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:08.935760 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0670a9005da848cd93427b2d2b4aa41d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:08.937927 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.937963 22807 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:08.945016 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:08.791240 31054 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:09.063297 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 125435us
I20260709 13:15:08.915643 31026 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.059338 31037 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:15:09.085182 31162 consensus_queue.cc:237] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.087224 31176 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.093302 22910 raft_consensus.cc:1275] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:09.099110 30896 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.107388 31078 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.061197 31193 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.110296 31085 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:09.118520 31043 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:09.119191 31083 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.844s	user 0.010s	sys 0.005s
I20260709 13:15:09.122323 22908 raft_consensus.cc:1275] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:15:09.195016 22908 pending_rounds.cc:85] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 12
I20260709 13:15:08.933714 31165 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:08.934237 31048 raft_consensus.cc:2804] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 13:15:09.130103 31157 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:09.133718 30999 raft_consensus.cc:2804] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:09.059906 31177 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:09.141409 31163 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.196537 31083 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:08.827421 31130 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:08.832494 30862 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.210057 22832 raft_consensus.cc:1240] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 17. Current term is 18. Ops: []
W20260709 13:15:09.169695 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 225086us
W20260709 13:15:09.167128 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 368037us
W20260709 13:15:09.094297 31070 consensus_meta.cc:287] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Time spent flushing consensus metadata: real 0.816s	user 0.012s	sys 0.001s
I20260709 13:15:08.942188 30939 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:09.193713 23466 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:09.195308 22908 pending_rounds.cc:107] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:15:09.060708 31141 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.157994 31159 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.143321 31051 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.144310 31208 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:09.218466 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:09.243878 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:09.244668 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.220521 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:09.246054 22834 raft_consensus.cc:2368] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
W20260709 13:15:09.246395 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 293643us
I20260709 13:15:09.246886 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:09.247316 22806 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:09.227219 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:09.229457 22908 raft_consensus.cc:2889] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:09.233385 31078 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.240224 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.241370 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.250211 23462 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:15:09.247912 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:09.248602 22836 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:09.197288 31193 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.231825 30999 raft_consensus.cc:697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:09.231846 31070 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:15:09.243700 31165 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.227331 31131 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:09.252629 30986 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.243242 31141 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:09.375181 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86526.5us
W20260709 13:15:09.335598 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 82619.9us
I20260709 13:15:09.210839 31085 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.384073 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:09.385960 31211 log.cc:927] Time spent T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:15:09.386688 22840 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
W20260709 13:15:09.338220 31210 log.cc:927] Time spent T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:15:09.400410 31141 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.402457 31083 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.243476 31159 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.259042 31078 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.404866 30999 consensus_queue.cc:237] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.397473 31131 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.333384 31065 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:09.372805 31213 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.394811 31165 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.414103 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.433452 31159 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.453531 22837 raft_consensus.cc:2468] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 31.
I20260709 13:15:09.430567 31054 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.461303 22730 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:09.462430 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 231069us
I20260709 13:15:09.463832 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:09.443320 31159 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.439559 31048 raft_consensus.cc:697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:09.467552 31208 raft_consensus.cc:3060] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:09.467801 31209 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:15:09.468585 31214 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:15:09.459134 30986 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.459035 31074 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.459101 31157 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.479705 22835 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:15:09.528998 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:09.532341 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 71166.2us
I20260709 13:15:09.533926 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:09.462173 31037 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:09.536753 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.540719 22836 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:09.541983 22879 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:09.440158 31213 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.543855 22879 leader_election.cc:304] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:09.494076 31083 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.550119 31117 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:09.494429 31065 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.558667 22835 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:09.514160 31026 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:09.536017 30862 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.539332 31135 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:09.539563 31130 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.540787 30986 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.572908 31048 consensus_queue.cc:237] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.573935 30853 consensus_queue.cc:260] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.575172 30853 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:09.575425 30986 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.555065 30896 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:09.554458 31169 raft_consensus.cc:2749] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:09.553725 31223 raft_consensus.cc:2804] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:09.496249 31159 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.556756 31085 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.522929 31220 consensus_queue.cc:1048] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:15:09.540235 31157 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.547889 31037 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.594960 22914 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2650) took 1149 ms. Trace:
I20260709 13:15:09.600649 22914 rpcz_store.cc:276] 0709 13:15:08.445899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.447109 (+  1210us) service_pool.cc:224] Handling call
0709 13:15:08.465567 (+ 18458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:08.474679 (+  9112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:08.474716 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:08.482175 (+  7459us) log.cc:844] Serialized 26 byte log entry
0709 13:15:08.705609 (+223434us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:15:08.705677 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.705784 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:09.594687 (+888903us) raft_consensus.cc:1685] finished
0709 13:15:09.594716 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.594873 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19177,"raft.queue_time_us":14599,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":3556,"thread_start_us":173075,"threads_started":1,"wal-append.queue_time_us":738681}
I20260709 13:15:09.559422 31117 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:09.562793 31054 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.538975 31074 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.560196 31083 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.569746 31065 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.596637 31223 raft_consensus.cc:697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:09.606500 31117 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:09.599380 31193 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.580564 31135 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.614883 31078 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.615481 31213 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:09.632767 31225 log.cc:927] Time spent T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:15:09.638924 31074 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.626662 30896 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.621803 31085 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.624429 31131 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.648200 22907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2654) took 1190 ms. Trace:
I20260709 13:15:09.612490 31054 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.658768 31065 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.659058 31083 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.648679 31048 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.664724 31085 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.663143 31135 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.664084 31054 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.682993 31074 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.664157 31223 consensus_queue.cc:237] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.689028 31048 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.665313 31041 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.653980 22907 rpcz_store.cc:276] 0709 13:15:08.457875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.458041 (+   166us) service_pool.cc:224] Handling call
0709 13:15:08.458453 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:08.465098 (+  6645us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:08.640433 (+175335us) log.cc:844] Serialized 14 byte log entry
0709 13:15:09.177986 (+537553us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:09.179067 (+  1081us) log.cc:844] Serialized 52 byte log entry
0709 13:15:09.179261 (+   194us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:09.179519 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 13:15:09.179709 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.179812 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:09.403167 (+223355us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b180026eb40. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:09.647879 (+244712us) raft_consensus.cc:1685] finished
0709 13:15:09.647917 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.648120 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9611,"prepare.queue_time_us":37119,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":5699,"raft.queue_time_us":62135,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":6687,"spinlock_wait_cycles":47090816,"thread_start_us":522501,"threads_started":2,"wal-append.queue_time_us":528406}
I20260709 13:15:09.691326 31085 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.692451 31041 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.694859 31048 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.690332 31074 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.664628 31165 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.695914 31041 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:09.698338 31074 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:09.698609 31145 consensus_meta.cc:287] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.776s	user 0.009s	sys 0.004s
I20260709 13:15:09.694876 22832 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39292 (request call id 2539) took 1588 ms. Trace:
I20260709 13:15:09.653476 30896 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.678560 31159 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.660921 31157 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.712098 22832 rpcz_store.cc:276] 0709 13:15:08.106604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.107191 (+   587us) service_pool.cc:224] Handling call
0709 13:15:08.216059 (+108868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.210535 (+994476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.694587 (+484052us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b540013abd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:09.694804 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285532544}
I20260709 13:15:09.715943 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.713289 22652 maintenance_manager.cc:643] P 6e189717484b49e8bb20c98667e6d294: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 1.131s	user 0.046s	sys 0.004s Metrics: {"bytes_written":4377,"delete_count":0,"lbm_write_time_us":13788,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":4,"reinsert_count":0,"thread_start_us":25882,"threads_started":1,"update_count":158,"wal-append.queue_time_us":40996}
I20260709 13:15:09.719998 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.720669 22836 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:15:09.714882 31145 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.725750 31145 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.718293 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:09.728773 22832 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:09.713342 31065 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.736618 31065 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.735103 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.735968 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:09.738356 22841 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 18.
I20260709 13:15:09.738749 31048 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:09.739380 31048 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.739831 31048 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:09.740361 30896 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.743268 22915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2652) took 1292 ms. Trace:
I20260709 13:15:09.753244 31157 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.753994 31157 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.731487 31083 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.756462 31157 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.756891 31083 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.758208 31159 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.758796 31159 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.759837 31083 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.738929 31065 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.753353 31139 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.766898 31139 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:09.768111 23466 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43708 (request call id 1431) took 2431 ms (client timeout 2539 ms). Trace:
W20260709 13:15:09.768540 23466 rpcz_store.cc:269] 0709 13:15:07.336232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.347592 (+ 11360us) service_pool.cc:224] Handling call
0709 13:15:09.767787 (+2420195us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400740254. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:09.767970 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4061471616}
I20260709 13:15:09.772023 22837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46858 (request call id 1457) took 1499 ms. Trace:
I20260709 13:15:09.772490 22837 rpcz_store.cc:276] 0709 13:15:08.272956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.274402 (+  1446us) service_pool.cc:224] Handling call
0709 13:15:09.771815 (+1497413us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540028a0d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:09.771927 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2573069696}
W20260709 13:15:09.774252 21636 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:09.776456 30896 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.777073 30896 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.777844 22912 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 20. (term mismatch)
I20260709 13:15:09.778422 22912 pending_rounds.cc:85] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 19
I20260709 13:15:09.778672 22912 pending_rounds.cc:107] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 25.20
I20260709 13:15:09.778913 22912 raft_consensus.cc:2889] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:09.780285 30896 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.781168 31159 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.781641 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:09.743800 22915 rpcz_store.cc:276] 0709 13:15:08.451033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.451183 (+   150us) service_pool.cc:224] Handling call
0709 13:15:08.451608 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:08.579473 (+127865us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:15:08.602486 (+ 23013us) log.cc:844] Serialized 14 byte log entry
0709 13:15:08.675413 (+ 72927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:08.675513 (+   100us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:15:08.675582 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.742948 (+1067366us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001d1d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:09.743014 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.743200 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5303,"prepare.queue_time_us":41757,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":3322,"raft.queue_time_us":171,"spinlock_wait_cycles":348477312,"thread_start_us":72036,"threads_started":1,"wal-append.queue_time_us":288960}
I20260709 13:15:09.784916 30980 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:09.785372 30980 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.785743 30980 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:09.787312 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 389175us
I20260709 13:15:09.792503 31145 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.793305 31145 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.794777 31037 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.795655 31145 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.769340 31139 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.797209 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 4005) took 1286 ms. Trace:
I20260709 13:15:09.797640 22913 rpcz_store.cc:276] 0709 13:15:08.510237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.804547 (+294310us) service_pool.cc:224] Handling call
0709 13:15:08.804828 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.195551 (+390723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:09.195586 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:09.195629 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:09.195660 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.796939 (+601279us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54001e32d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:09.797001 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.797139 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898107776}
I20260709 13:15:09.800714 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.802935 31054 consensus_queue.cc:1048] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:09.804986 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:09.822892 23431 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.657s (SENT)
W20260709 13:15:09.823264 22840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39292 (request call id 2530) took 2452 ms (client timeout 2013 ms). Trace:
W20260709 13:15:09.823645 22840 rpcz_store.cc:269] 0709 13:15:07.370553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.420153 (+ 49600us) service_pool.cc:224] Handling call
0709 13:15:09.823027 (+2402874us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54000a5354. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af52fa 00007f68a6ab6b17 00007f68a00316d4 00007f68a0031662 00007f68a0031591 00007f68a003150c 00007f68a00307c2 00007f689fe2c48c 00007f689fe2b8c6 00007f689fe2b26f
0709 13:15:09.823184 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4455078656}
I20260709 13:15:09.827330 22839 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:09.830305 31208 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:09.834363 31191 log.cc:927] Time spent T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 1.277s	user 0.021s	sys 0.041s
I20260709 13:15:09.836261 30981 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:09.836468 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 30 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.836917 30981 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.837960 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.838056 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.839330 30981 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:09.840930 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.841717 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:09.842186 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.842584 30981 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.843011 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:09.843657 22913 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:09.843458 30981 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:09.844157 30998 consensus_queue.cc:1048] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:09.844975 30998 consensus_queue.cc:1243] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:09.846010 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.841850 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43708 (request call id 1436) took 1639 ms. Trace:
I20260709 13:15:09.848387 30981 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:09.837252 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.850292 22911 raft_consensus.cc:2410] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:15:09.854081 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.855538 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.858137 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.864989 22832 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:15:09.858027 22653 maintenance_manager.cc:419] P 6e189717484b49e8bb20c98667e6d294: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.220338
I20260709 13:15:09.854050 22805 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:09.868645 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.870669 22908 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:15:09.869887 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 3996) took 1735 ms. Trace:
I20260709 13:15:09.872671 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.872381 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:09.871559 22912 rpcz_store.cc:276] 0709 13:15:08.134300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.134583 (+   283us) service_pool.cc:224] Handling call
0709 13:15:08.135066 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.779521 (+1644455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.866783 (+ 87262us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54001e41d4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:09.869821 (+  3038us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3777251200}
I20260709 13:15:09.848703 23463 rpcz_store.cc:276] 0709 13:15:08.203005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.205305 (+  2300us) service_pool.cc:224] Handling call
0709 13:15:08.216244 (+ 10939us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:08.216652 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:08.216685 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:08.248131 (+ 31446us) log.cc:844] Serialized 52 byte log entry
0709 13:15:08.377542 (+129411us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:15:08.559570 (+182028us) log.cc:844] Serialized 14 byte log entry
0709 13:15:08.577492 (+ 17922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.577617 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:09.738252 (+1160635us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b180040c840. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea04522 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:09.800083 (+ 61831us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b180040c840. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9d22a6 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db
0709 13:15:09.833738 (+ 33655us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:15:09.841621 (+  7883us) raft_consensus.cc:1685] finished
0709 13:15:09.841649 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.841793 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28675,"prepare.queue_time_us":85229,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":4578,"raft.queue_time_us":24535,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":1497,"spinlock_wait_cycles":130306432,"thread_start_us":240367,"threads_started":2,"wal-append.queue_time_us":215431}
I20260709 13:15:09.939599 23465 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:09.942456 22913 raft_consensus.cc:2468] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
W20260709 13:15:09.945719 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 74232.3us
I20260709 13:15:09.952368 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:09.939553 31023 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:09.992934 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:09.867926 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:09.964968 31164 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:15:09.940335 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 69422.8us
I20260709 13:15:09.968911 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.998677 23457 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:09.969835 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.970181 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:09.999418 23462 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:09.971158 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:09.941031 31025 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:10.002712 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:09.995180 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 127281us
I20260709 13:15:09.983254 30980 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:09.953881 31026 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:09.985241 31141 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:09.991773 31054 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:09.975449 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.017704 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.987174 30853 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:15:10.008365 23431 leader_election.cc:336] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.009394 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.011615 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.015161 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.020351 22841 raft_consensus.cc:2410] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 13:15:10.020509 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.021049 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.022441 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.023309 22840 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:15:10.030184 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.030535 23466 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:15:10.031366 22838 raft_consensus.cc:2410] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:15:10.032826 22833 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:10.033083 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.033771 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.034595 22836 raft_consensus.cc:2410] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:15:10.035164 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.035629 22834 raft_consensus.cc:2410] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 13:15:10.037963 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.038551 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.043192 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.044159 23457 raft_consensus.cc:2393] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 19: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:10.045174 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.045027 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.046195 22806 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.001380 30981 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
I20260709 13:15:10.010440 31065 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:10.059388 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.057725 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:09.998860 31164 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.062935 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.064266 22836 raft_consensus.cc:2410] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:15:09.999441 31117 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.069288 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:10.072000 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.072376 22833 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: []
I20260709 13:15:10.072939 22913 raft_consensus.cc:1217] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 15.10->[16.11-23.13]   Dedup: 16.11->[21.12-23.13]
I20260709 13:15:10.038884 31085 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:10.073666 22913 pending_rounds.cc:85] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 11
I20260709 13:15:10.080780 22913 pending_rounds.cc:107] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:15:10.081123 22913 raft_consensus.cc:2889] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:10.070335 22835 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 18.
I20260709 13:15:10.039220 31208 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.078462 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:10.092389 22880 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.055733 31139 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:10.012015 31025 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.016506 31026 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.036113 30853 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:10.125437 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 501809us
I20260709 13:15:10.029770 31141 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.090070 30999 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:10.142508 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.144742 23462 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
W20260709 13:15:10.128523 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.161794 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66267.8us
I20260709 13:15:10.094503 31157 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:10.162978 22806 leader_election.cc:336] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.164453 22806 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.166653 22732 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:10.075431 31065 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:10.157730 22880 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.199846 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46063.5us
I20260709 13:15:10.218564 31139 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:10.222867 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:10.228402 22880 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.229219 22880 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.168352 31217 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:10.096410 31208 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.176913 31145 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.200882 31037 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:10.236840 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:15:10.234730 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 183217us
W20260709 13:15:10.248199 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.254652 22841 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
W20260709 13:15:10.256491 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 233821us
W20260709 13:15:10.257577 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:10.165705 31157 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.254606 23430 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:10.270557 22835 raft_consensus.cc:1275] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:15:10.228243 31048 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.264508 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.280710 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.246798 31028 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.244978 31221 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:10.275725 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.287057 22835 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:15:10.246654 31037 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:10.292498 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.290419 22730 leader_election.cc:400] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
I20260709 13:15:10.294355 22730 leader_election.cc:403] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:15:10.304572 31065 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:10.305585 31160 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:10.306154 31160 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.308916 31160 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:10.310484 31025 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.312675 31164 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.313560 30980 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:10.289171 31127 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:10.316002 31127 raft_consensus.cc:3060] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:10.318198 31236 log.cc:927] Time spent T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:15:10.318917 31048 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.319835 31037 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.241482 31145 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.310485 22836 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 13:15:10.315065 31026 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:10.268123 31139 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:10.325225 31145 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.289011 31054 raft_consensus.cc:2804] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:10.292323 31221 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.355424 30975 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.356213 30975 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.355473 31157 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.358920 30975 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:10.360342 22879 connection.cc:256] RPC call timeout handler was delayed by -1.2029498330000479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:15:10.361891 31223 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.362747 31223 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.364859 31223 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:10.365290 31244 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 13:15:10.366684 22912 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 13:15:10.369406 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 111075us
W20260709 13:15:10.370149 23431 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.371145 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:10.374915 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:10.377965 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:10.379149 22835 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:10.380646 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.382242 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.383499 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:10.388023 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.388736 22906 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:10.353368 31221 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:10.398676 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.399490 30980 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:10.400214 30980 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.402894 30980 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.339027 31065 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:10.407894 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.410436 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.416216 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.417129 22911 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 22.
I20260709 13:15:10.353109 31145 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.422849 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.297479 31028 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.432436 22835 raft_consensus.cc:2468] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 19.
I20260709 13:15:10.397013 31054 raft_consensus.cc:697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:10.416564 31223 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.451761 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.452927 22906 raft_consensus.cc:2393] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:10.459306 30853 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:10.422068 31065 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.462242 31065 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.463819 31141 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.467733 31054 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.469863 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.469590 31223 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.471148 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:10.471728 22911 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:10.472378 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.473308 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:10.474642 31240 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:15:10.474866 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.476087 22909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2653) took 2021 ms. Trace:
I20260709 13:15:10.472718 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.439632 31117 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:10.450963 23430 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:10.478567 31217 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:10.479125 31217 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:10.479243 31141 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:10.479930 31141 raft_consensus.cc:697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:10.480501 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.473352 22915 raft_consensus.cc:2393] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:10.485486 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.486040 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.490311 22914 raft_consensus.cc:1275] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:15:10.351023 31139 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.497063 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.497977 22906 raft_consensus.cc:1275] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:15:10.503787 22907 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:15:10.503875 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.510421 31208 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:10.476552 22909 rpcz_store.cc:276] 0709 13:15:08.454899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.455258 (+   359us) service_pool.cc:224] Handling call
0709 13:15:08.455669 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:08.461982 (+  6313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:08.462040 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:09.826961 (+1364921us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b2c00051d28. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a03eaa11 00007f68a03fa372 00007f68a1b2ca01 00007f68a1af143a 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2
0709 13:15:09.827465 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:15:10.154645 (+327180us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:15:10.154700 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.154798 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:10.475782 (+320984us) raft_consensus.cc:1685] finished
0709 13:15:10.475809 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:10.476024 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2642,"raft.queue_time_us":1519,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3310,"spinlock_wait_cycles":9038720,"thread_start_us":326347,"threads_started":1,"wal-append.queue_time_us":417026}
W20260709 13:15:10.514127 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 66511.9us
I20260709 13:15:10.483119 31028 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:10.472682 31223 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:10.480895 31141 consensus_queue.cc:237] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.517474 22911 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:15:10.520262 22839 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 24: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:10.435948 31145 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.526947 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:10.533382 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 247017us
I20260709 13:15:10.536713 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.539760 22835 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:10.541682 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.544040 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.539849 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.550746 22838 raft_consensus.cc:1275] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:10.551437 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.552484 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:10.552474 22835 raft_consensus.cc:2393] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 25: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:10.521235 31085 consensus_queue.cc:1048] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:15:10.563162 31208 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:10.543417 31131 consensus_queue.cc:1048] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:15:10.575059 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 412093us
W20260709 13:15:10.576187 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 275701us
W20260709 13:15:10.579295 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64148.8us
I20260709 13:15:10.577626 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.587477 31145 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:10.591759 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:10.590273 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.593933 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:10.597226 22879 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:10.592029 22880 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.588155 31208 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:10.612828 22806 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.616176 22806 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.617300 22806 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.618644 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.620296 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.620903 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.597309 22730 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:10.628659 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:10.631518 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.632688 31208 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:10.634038 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:10.635795 22837 raft_consensus.cc:2410] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 27 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:15:10.629478 22732 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:10.619848 31069 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:10.642688 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.630105 31248 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:10.636881 30896 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:10.661810 31252 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:10.662494 31252 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.662510 31253 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:10.662953 31252 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:10.633666 31213 consensus_queue.cc:1048] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:15:10.673274 31139 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:10.675460 31127 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) with higher term. Message: Invalid argument: T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 for earlier term 21. Current term is 22.
I20260709 13:15:10.679666 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.680620 23462 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 27.
I20260709 13:15:10.686489 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:10.652434 22732 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:10.688899 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 91692.5us
I20260709 13:15:10.693339 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.690006 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:10.703548 22880 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.686234 31234 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:10.698577 31159 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:10.723254 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.723541 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:15:10.744347 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54462.2us
I20260709 13:15:10.744843 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:10.747011 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 53808.3us
W20260709 13:15:10.774765 21636 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:10.786863 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.790282 22907 raft_consensus.cc:1275] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
W20260709 13:15:10.790535 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45107.3us
I20260709 13:15:10.721472 31234 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:10.799917 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.727110 31159 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.709973 31221 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.810367 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.812485 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.792037 31223 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:10.790198 30896 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:10.790198 30975 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:10.789889 31127 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.783258 30998 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:10.831470 23457 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 23. Current term is 24. Ops: []
I20260709 13:15:10.817703 31159 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:10.837936 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:10.859778 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47418.4us
I20260709 13:15:10.872714 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.872956 23458 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:10.803685 31234 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:10.866775 23462 raft_consensus.cc:1240] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 25. Current term is 26. Ops: []
I20260709 13:15:10.868899 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:10.815999 31208 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:15:10.886561 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.897616 23462 raft_consensus.cc:2410] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:15:10.815263 31145 consensus_queue.cc:1059] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 13:15:10.898644 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 104492us
I20260709 13:15:10.873594 31025 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:15:10.907058 22880 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:10.908264 22880 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.881477 31117 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:10.909008 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:10.909258 31117 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.909953 31221 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:10.910714 31041 consensus_queue.cc:1048] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 13:15:10.911501 31117 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:10.894106 31244 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:10.923058 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.894016 31043 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:10.925561 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.894740 31085 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:15:10.938375 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.939149 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:10.942960 22880 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.875607 31127 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:10.940549 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:10.943492 22880 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.875123 30998 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:10.928349 31145 consensus_queue.cc:1243] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:10.933048 31043 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.977846 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:10.920302 31160 consensus_queue.cc:1048] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:15:10.944214 31028 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:10.989621 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:11.007053 22907 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:11.010005 31157 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:10.930096 31217 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.014213 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:11.014961 23466 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
W20260709 13:15:10.990540 22880 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:10.994808 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:10.982656 31043 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:11.018501 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:10.983227 30853 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:15:10.982729 31244 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:11.034369 22907 raft_consensus.cc:2468] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:15:11.032876 30975 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:15:11.033824 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:11.002146 31145 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:15:11.005844 30896 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 13:15:11.040412 22880 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.042254 22880 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.042847 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:11.013427 31028 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:11.071290 31254 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 13:15:11.118389 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:11.124920 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 163197us
I20260709 13:15:11.125867 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:11.129367 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 203954us
I20260709 13:15:11.129806 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.130295 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:11.132535 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 92192.3us
I20260709 13:15:11.132625 22879 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:11.133562 22880 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:11.134959 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:11.135473 22880 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.145926 22880 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.135982 31235 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:11.148413 31167 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:11.146842 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:11.154631 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.136883 31208 raft_consensus.cc:3055] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:11.131893 31160 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:11.156121 31163 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:11.107030 30853 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:15:11.157037 31117 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:11.159420 31167 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:11.197837 22907 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:11.169597 31275 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:11.211700 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:11.210569 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 75116.4us
I20260709 13:15:11.169667 31208 raft_consensus.cc:740] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:11.172173 31290 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:11.214383 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 84173.6us
I20260709 13:15:11.233119 22730 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.192477 31289 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:11.208341 31167 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:11.252074 31276 log.cc:927] Time spent T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:15:11.214030 31127 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:11.254288 31266 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:15:11.256398 31280 log.cc:927] Time spent T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:15:11.257688 31277 log.cc:927] Time spent T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:15:11.242197 31028 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:11.262473 31085 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:11.263150 31085 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:15:11.263661 31279 log.cc:927] Time spent T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:15:11.266273 31213 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:11.266731 31213 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:11.267621 31213 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.269160 31213 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:11.241051 31275 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:11.274274 31275 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.278561 31167 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:11.286751 31208 consensus_queue.cc:260] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.287832 31289 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:11.297676 30998 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:11.285384 31282 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 13:15:11.288304 31127 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.016s
W20260709 13:15:11.304752 31294 log.cc:927] Time spent T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:11.238317 31290 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.305936 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:11.306514 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 358088us
I20260709 13:15:11.306591 31159 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.307432 31217 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:11.308665 31252 consensus_meta.cc:287] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.007s
W20260709 13:15:11.311923 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.352937 22806 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:11.325253 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 92220.6us
I20260709 13:15:11.328187 23432 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:11.333539 31043 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.343962 22840 raft_consensus.cc:1275] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:11.318809 31248 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:15:11.350279 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 90885us
W20260709 13:15:11.351754 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 117309us
I20260709 13:15:11.354458 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:11.365716 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 53814us
W20260709 13:15:11.367142 22806 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.367762 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.344343 31127 consensus_queue.cc:1243] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:15:11.335611 31208 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:15:11.378877 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 76337.5us
W20260709 13:15:11.393119 22806 leader_election.cc:336] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.393929 22806 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.395298 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.395958 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:11.402395 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.406184 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.361737 31048 raft_consensus.cc:2804] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:11.408258 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.371284 30996 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:15:11.410928 30981 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:11.429293 31145 raft_consensus.cc:3060] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:11.431015 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70216.6us
I20260709 13:15:11.406780 30998 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:11.400286 31300 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:11.363968 31252 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.409672 31037 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:11.410226 31302 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:11.450237 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:11.452502 31273 consensus_queue.cc:1059] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 13:15:11.454272 22879 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.502669 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:11.469821 31048 raft_consensus.cc:697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:11.440261 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:11.481261 22911 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:11.372849 31248 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.424853 31295 raft_consensus.cc:3060] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:11.502731 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:11.538986 22833 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:11.491336 31235 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:15:11.521622 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81478.4us
I20260709 13:15:11.544341 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.473157 31305 consensus_queue.cc:1048] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:11.527070 22910 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
W20260709 13:15:11.528604 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 126495us
W20260709 13:15:11.532648 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 43895.7us
W20260709 13:15:11.579445 22806 leader_election.cc:336] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.598362 31290 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:11.599428 31043 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:11.602707 31248 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:11.514779 31298 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 13:15:11.605633 31048 consensus_queue.cc:237] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.611011 22833 raft_consensus.cc:2468] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:15:11.613029 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 68847.7us
I20260709 13:15:11.618271 31145 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:11.587813 31234 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.001s
I20260709 13:15:11.704788 31234 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.707116 31234 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:11.707304 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:11.708037 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:11.708928 22840 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:11.709219 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:11.710261 22906 raft_consensus.cc:1240] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:11.711412 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:11.712000 22835 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:11.712832 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:11.634924 31159 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:11.635919 30975 consensus_meta.cc:287] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.004s
I20260709 13:15:11.646442 31300 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:11.592552 31037 raft_consensus.cc:2804] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:11.658983 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
W20260709 13:15:11.661764 22880 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.678006 22806 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.692974 23462 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:15:11.710167 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:11.712903 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:11.633894 31271 log.cc:927] Time spent T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:15:11.623060 31252 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:11.715327 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:11.751562 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 172175us
I20260709 13:15:11.748829 22880 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:11.753448 22806 leader_election.cc:336] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.751178 31037 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:11.753983 22806 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.740888 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:11.742782 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:11.754907 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 93198.1us
W20260709 13:15:11.746927 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 381998us
I20260709 13:15:11.738574 22837 raft_consensus.cc:1217] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Deduplicated request from leader. Original: 16.10->[22.11-24.12]   Dedup: 22.11->[24.12-24.12]
I20260709 13:15:11.755982 22837 raft_consensus.cc:1240] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 24. Current term is 25. Ops: [22.11-24.12]
W20260709 13:15:11.756163 22880 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.745044 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:11.757123 23457 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:15:11.673686 31085 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:11.773598 22806 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.774128 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:11.775434 21636 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:11.830405 23457 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
W20260709 13:15:11.838603 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61562.9us
I20260709 13:15:11.839238 22839 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:15:11.839824 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 67099.3us
I20260709 13:15:11.748593 31145 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.844121 22835 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
W20260709 13:15:11.758152 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:11.845335 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 87250.7us
W20260709 13:15:11.831700 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 58127.4us
I20260709 13:15:11.751688 30975 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:11.761051 31037 raft_consensus.cc:3060] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:11.874018 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 350065us
I20260709 13:15:11.759800 31023 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:15:11.887028 22806 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.895471 22806 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.770859 31028 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:15:11.894636 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54041.1us
I20260709 13:15:11.780942 30996 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:11.771145 31248 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:11.841393 30998 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:11.901398 22907 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: []
W20260709 13:15:11.893193 31213 consensus_meta.cc:287] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.003s
W20260709 13:15:11.897405 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:11.890655 31159 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.916165 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:11.894863 31252 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.924652 22879 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:11.898459 31315 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:11.949120 22806 leader_election.cc:336] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.949887 22806 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.950548 30896 raft_consensus.cc:3055] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:11.951004 30896 raft_consensus.cc:740] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:15:11.951154 22806 leader_election.cc:336] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.951663 22806 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.951901 30896 consensus_queue.cc:260] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:11.952839 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.952916 30998 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:11.891659 31145 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:11.958859 22806 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:11.921617 31300 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.923409 31179 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:11.966218 22806 leader_election.cc:336] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:11.969228 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:11.960358 31312 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.957310 31028 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:11.956533 31298 raft_consensus.cc:2804] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:11.908457 31248 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.988497 31248 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:11.957958 31023 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:11.976948 31304 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:11.944923 31252 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:11.975417 31167 consensus_meta.cc:287] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.000s
W20260709 13:15:11.992241 31308 log.cc:927] Time spent T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.302s	user 0.002s	sys 0.000s
I20260709 13:15:12.015601 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:11.938409 31159 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.020576 22910 raft_consensus.cc:2393] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:11.996133 31252 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:11.995282 31167 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:12.001168 31316 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:15:12.010870 31298 raft_consensus.cc:697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:11.979352 31302 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:11.972769 31318 consensus_queue.cc:1059] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:15:11.984627 31312 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.015523 31028 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.036116 22912 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2684) took 1095 ms. Trace:
I20260709 13:15:11.968670 31244 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:15:11.992292 31235 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:11.994678 31295 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:11.955757 31315 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:11.976131 31300 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.041208 22912 rpcz_store.cc:276] 0709 13:15:10.940973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.950039 (+  9066us) service_pool.cc:224] Handling call
0709 13:15:10.954203 (+  4164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.050446 (+ 96243us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:15:11.157753 (+107307us) log.cc:844] Serialized 14 byte log entry
0709 13:15:11.211648 (+ 53895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:11.211712 (+    64us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:11.211770 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.299017 (+ 87247us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b180027d600. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:12.035729 (+736712us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b5400125cd4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:12.035827 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.036033 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":892875,"prepare.queue_time_us":109308,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":76787,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":31828,"spinlock_wait_cycles":235433600,"thread_start_us":137790,"threads_started":2,"wal-append.queue_time_us":131541}
I20260709 13:15:12.054910 22908 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 13:15:12.041616 31305 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:12.056756 31085 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:12.092037 31145 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:12.103392 31300 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:12.103580 31167 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:12.104367 31309 log.cc:927] Time spent T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.400s	user 0.003s	sys 0.000s
I20260709 13:15:12.105455 31315 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:12.108058 31159 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:12.091245 31298 consensus_queue.cc:237] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:12.111615 31314 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.363s	user 0.000s	sys 0.003s
I20260709 13:15:12.080175 31305 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:12.103377 31288 log.cc:927] Time spent T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.970s	user 0.003s	sys 0.017s
I20260709 13:15:12.113932 31312 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:12.154673 22652 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.268s	user 0.000s	sys 0.001s
W20260709 13:15:12.172446 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 256134us
W20260709 13:15:12.065151 22880 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:12.191359 31028 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:11.975574 22806 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:12.214242 31157 raft_consensus.cc:3055] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:12.215155 31157 raft_consensus.cc:740] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
W20260709 13:15:12.216185 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 249859us
I20260709 13:15:12.220453 31323 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:12.229046 22913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43274 (request call id 4037) took 1550 ms. Trace:
I20260709 13:15:12.229839 22913 rpcz_store.cc:276] 0709 13:15:10.678065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.683671 (+  5606us) service_pool.cc:224] Handling call
0709 13:15:10.684025 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:10.684593 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:10.684622 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:10.685062 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:15:11.136509 (+451447us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:15:11.136560 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.136636 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.102654 (+966018us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
0709 13:15:12.228218 (+125564us) raft_consensus.cc:1685] finished
0709 13:15:12.228264 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.228789 (+   525us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"spinlock_wait_cycles":7539072,"thread_start_us":443406,"threads_started":1,"wal-append.queue_time_us":440849}
I20260709 13:15:12.237857 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:12.239660 23457 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 24: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:12.248550 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 47721.2us
I20260709 13:15:12.251955 31301 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.256922 31305 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:12.252148 31159 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.260746 22880 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:12.264402 31301 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.277264 31159 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.278055 22837 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 27. Current term is 28. Ops: []
I20260709 13:15:12.253230 31255 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:15:12.301214 22906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43286 (request call id 2687) took 1126 ms. Trace:
I20260709 13:15:12.301805 22906 rpcz_store.cc:276] 0709 13:15:11.174638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.196950 (+ 22312us) service_pool.cc:224] Handling call
0709 13:15:11.197277 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.710724 (+513447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.300484 (+589760us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54001d1d54. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1af1e01 00007f68a1af083e 00007f68a6ab5419 00007f68a002cbd4 00007f68a002cb62 00007f68a002ca91 00007f68a002ca0c 00007f68a002bcc2 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:12.300968 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179338624}
I20260709 13:15:12.314803 30896 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:12.323181 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.324530 22907 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
W20260709 13:15:12.331604 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 266484us
I20260709 13:15:12.281231 31159 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:12.366647 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:12.367190 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:12.305656 22806 leader_election.cc:336] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:12.375054 31320 log.cc:927] Time spent T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.137s	user 0.002s	sys 0.004s
I20260709 13:15:12.383893 31301 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:12.385566 22806 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:12.218282 31157 consensus_queue.cc:260] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:12.397740 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 64902.8us
I20260709 13:15:12.386276 31159 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:12.412381 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 114283us
I20260709 13:15:12.422178 31157 raft_consensus.cc:3060] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:12.424355 31085 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:12.443094 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:12.417469 31026 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:12.462042 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:15:12.462353 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 217378us
W20260709 13:15:12.463474 22806 leader_election.cc:336] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:12.464022 22806 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:12.465374 31065 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:12.471208 22880 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:12.499224 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:12.500653 31159 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:12.514113 22807 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:12.515344 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 55496.3us
I20260709 13:15:12.516660 31065 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:12.545266 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:12.545815 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:12.539131 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 62680.1us
I20260709 13:15:12.571923 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:12.575788 22906 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:12.539892 22907 pending_rounds.cc:85] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 12
I20260709 13:15:12.521433 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:12.584851 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 121503us
W20260709 13:15:12.553234 31037 consensus_meta.cc:287] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.010s
I20260709 13:15:12.576560 22907 pending_rounds.cc:107] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:15:12.591465 22907 raft_consensus.cc:2889] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:12.594465 31301 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.618340 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:12.625933 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:12.627238 22913 raft_consensus.cc:2393] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 23: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:12.631063 31065 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:12.536285 31159 consensus_queue.cc:1059] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.079s
W20260709 13:15:12.662120 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105456us
I20260709 13:15:12.635331 31065 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:12.679363 31065 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:12.679450 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.603649 31037 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.675155 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.675719 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:15:12.676599 31333 log.cc:927] Time spent T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:15:12.676786 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:12.677814 22911 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:15:12.683491 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:12.705976 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 184666us
I20260709 13:15:12.740355 23466 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:12.744954 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:12.747231 30981 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:15:12.750890 31028 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.751770 31028 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.758767 22914 raft_consensus.cc:1275] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:12.758623 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.768483 31141 consensus_queue.cc:1048] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:12.769799 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:12.769832 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.771906 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.684700 31301 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.772076 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:12.777745 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:12.786486 21636 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:12.783612 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.789717 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 29 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:12.791817 31301 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:12.794441 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 197223us
I20260709 13:15:12.781174 31255 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:12.790437 22914 raft_consensus.cc:2393] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 20: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:12.804847 31037 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:12.818186 22906 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:12.789921 31318 raft_consensus.cc:3055] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:15:12.826198 31235 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 13:15:12.828915 23466 raft_consensus.cc:2468] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:15:12.870349 31028 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:12.871009 23431 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.729s (SENT)
I20260709 13:15:12.873126 31223 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.873746 31223 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.875542 31157 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.876199 31157 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.876708 31223 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:12.879340 22880 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:12.880100 22880 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:12.881151 31235 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:15:12.882241 31157 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:12.871187 31028 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:15:12.876556 30996 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.831933 31318 raft_consensus.cc:740] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:12.888120 31028 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:15:12.903157 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 289767us
W20260709 13:15:12.919068 22880 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:12.923655 22880 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:12.911687 31157 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.904332 30996 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.954608 30996 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:12.956248 31028 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.958158 31028 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:15:12.957810 31157 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:12.960546 31157 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:12.961876 31235 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:15:12.963793 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:12.922310 31255 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:12.978300 31255 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:12.982301 31223 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:12.983305 31223 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:12.984117 22664 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 249) took 42420 ms (client timeout 15000 ms). Trace:
I20260709 13:15:12.919260 31037 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 13:15:12.988160 31037 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:12.994076 31065 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:12.984827 22664 rpcz_store.cc:269] 0709 13:14:30.564946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.785129 (+11220183us) service_pool.cc:224] Handling call
0709 13:15:12.983753 (+31198624us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:06.292721 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:06.293655 (+   934us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:06.293717 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:15:06.293734 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:06.539846 (+246112us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:06.563066 (+ 23220us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:06.565214 (+  2148us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:15:06.697226 (+132012us) tablet.cc:701] Row locks acquired
0709 13:15:06.697280 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:15:06.700664 (+  3384us) write_op.cc:270] Start()
0709 13:15:06.748476 (+ 47812us) write_op.cc:276] Timestamp: P: 1783602906697421 usec, L: 0
0709 13:15:06.751775 (+  3299us) op_driver.cc:348] REPLICATION: starting
0709 13:15:06.863707 (+111932us) log.cc:844] Serialized 6226 byte log entry
0709 13:15:07.183923 (+320216us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aece28 00007f68a5527581 00007f68a5526d58 00007f68a552b9f1 00007f68a552b9a9 00007f68a552b939 00007f68a552b901 00007f68a552ab2d 000000000038f354 000000000038f199
0709 13:15:07.705658 (+521735us) op_driver.cc:464] REPLICATION: finished
0709 13:15:08.231906 (+526248us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a5527ef0
0709 13:15:09.695270 (+1463364us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b180010c300. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:15:09.695475 (+   205us) write_op.cc:301] APPLY: starting
0709 13:15:09.695593 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:10.569281 (+873688us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:10.570658 (+  1377us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:11.523272 (+952614us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:11.602020 (+ 78748us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=64,mrs_lookups=0
0709 13:15:11.602084 (+    64us) write_op.cc:312] APPLY: finished
0709 13:15:11.602731 (+   647us) log.cc:844] Serialized 148 byte log entry
0709 13:15:11.747748 (+145017us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:11.747959 (+   211us) write_op.cc:454] Released schema lock
0709 13:15:11.748034 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:11.886536 (+138502us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:11.892553 (+  6017us) write_op.cc:454] Released schema lock
0709 13:15:12.110772 (+218219us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f68a168c86e 00007f68a168bbb6 00007f68a168b7d8 00007f68a1695221 00007f68a16951d9 00007f68a1695169
Metrics: {"mutex_wait_us":544034,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":815178,"apply.run_cpu_time_us":46318,"apply.run_wall_time_us":2131965,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"cfile_init":1,"lbm_read_time_us":26478,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":2,"num_ops":16,"prepare.queue_time_us":402107,"prepare.run_cpu_time_us":52112,"prepare.run_wall_time_us":909460,"raft.queue_time_us":304151,"raft.run_cpu_time_us":33274,"raft.run_wall_time_us":924115,"replication_time_us":911622,"spinlock_wait_cycles":56026496,"thread_start_us":1411472,"threads_started":5,"wal-append.queue_time_us":332021,"wal-append.run_cpu_time_us":18642,"wal-append.run_wall_time_us":1357371}]]}
I20260709 13:15:13.011103 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
W20260709 13:15:13.016644 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.019147 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 100112us
W20260709 13:15:13.022487 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 71264.6us
W20260709 13:15:13.049938 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 81941.5us
I20260709 13:15:13.051491 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:13.052661 23466 raft_consensus.cc:2393] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 24: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:13.049657 30996 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:15:13.054795 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.055178 30996 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.057319 31223 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:13.057669 31157 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:13.058281 31157 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:13.044867 31065 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:13.059525 31157 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:13.061995 22880 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.065943 31318 consensus_queue.cc:260] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.067008 30996 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.067729 31318 raft_consensus.cc:3060] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:13.070086 31255 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:13.098719 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44662.9us
I20260709 13:15:13.054107 31037 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.098785 31301 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.139919 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.139062 22652 maintenance_manager.cc:643] P 6e189717484b49e8bb20c98667e6d294: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 3.266s	user 0.118s	sys 0.076s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":25338,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":719310,"dirs.run_cpu_time_us":537,"dirs.run_wall_time_us":10110,"lbm_read_time_us":41145,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":18,"lbm_write_time_us":13586,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":8,"peak_mem_usage":257557,"reinsert_count":0,"thread_start_us":463866,"threads_started":2,"update_count":167}
I20260709 13:15:13.143440 31037 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:13.144960 22659 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 267) took 99124 ms (client timeout 15000 ms). Trace:
I20260709 13:15:13.145552 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.145879 31255 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:13.160014 30996 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.134711 30896 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.156747 22659 rpcz_store.cc:269] 0709 13:13:34.020742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.020958 (+   216us) service_pool.cc:224] Handling call
0709 13:13:34.867100 (+846142us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5b090 00007f68a1af9f3a 00007f68a78b01cc 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc 00007f68a5e6e392 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:13.144709 (+98277609us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121865,"reported_tablets":51,"spinlock_wait_cycles":112230144}
I20260709 13:15:13.159530 22653 maintenance_manager.cc:419] P 6e189717484b49e8bb20c98667e6d294: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:15:13.166172 22652 maintenance_manager.cc:643] P 6e189717484b49e8bb20c98667e6d294: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 13:15:13.170516 31348 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.171301 31348 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.172230 30896 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.174561 30896 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.175863 31301 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.159745 30981 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:13.159936 31139 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
W20260709 13:15:13.160038 22806 leader_election.cc:336] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.162964 30996 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.191596 22806 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:13.181607 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.196031 31301 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.196561 31065 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:13.197180 31348 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:13.201006 22732 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.203835 30996 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.160768 31255 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:13.213723 23431 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.117s (SENT)
I20260709 13:15:13.224727 31037 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:13.219422 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:13.231078 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.237203 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:13.239228 22836 raft_consensus.cc:1275] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:15:13.240523 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 46120.8us
I20260709 13:15:13.242009 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.243521 22910 raft_consensus.cc:2410] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:15:13.244617 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.245548 22837 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:15:13.216657 30981 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.256174 30981 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.230273 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:13.289902 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47301.5us
I20260709 13:15:13.305420 22911 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
W20260709 13:15:13.231020 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:13.308301 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77352us
I20260709 13:15:13.216356 31139 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:13.308931 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:13.325268 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 104581us
I20260709 13:15:13.315199 31139 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:13.327953 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:13.336961 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:13.340456 31037 consensus_queue.cc:1059] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:15:13.353049 31318 raft_consensus.cc:3055] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:13.359447 22656 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:13.361814 23466 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:15:13.361310 22656 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 16 to 19, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 19 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.362581 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:13.363270 22656 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 15 to 21, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 21 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.364228 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:13.365710 22656 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:13.367188 22656 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 15 to 18, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 18 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.369336 22656 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:15:13.369984 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.371321 22656 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 17 to 19, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 19 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.375712 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:13.397241 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:13.403654 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.404554 22834 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 22.
I20260709 13:15:13.358471 31318 raft_consensus.cc:740] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:13.407486 31255 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.407684 31318 consensus_queue.cc:260] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.415587 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:13.416579 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 256603us
W20260709 13:15:13.417675 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:13.417306 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:13.419164 23431 leader_election.cc:336] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:13.420706 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.408140 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:13.422497 31244 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:13.422789 23430 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:13.417016 23464 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 13:15:13.429857 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:13.432211 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.436923 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.437903 22839 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 28: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:13.438617 31255 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:13.439112 31255 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.439044 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.439610 31255 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:13.416792 31037 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.440459 31037 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.442018 31348 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.442574 31348 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.443403 31037 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:13.444787 31348 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.395099 22656 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:13.409426 31318 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:13.457016 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.467160 22879 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:13.464155 31298 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:13.472800 22834 raft_consensus.cc:1240] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:13.477054 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.475304 22656 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 19 to 21, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 21 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.479599 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.480268 22909 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 24.
I20260709 13:15:13.481052 31223 raft_consensus.cc:2804] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:13.481693 31223 raft_consensus.cc:697] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:13.480752 22656 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 18, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 18 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.486394 22656 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 15, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 15 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:13.483840 22881 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.482738 31223 consensus_queue.cc:237] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.426017 31157 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 13:15:13.497548 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 29 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:13.497648 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.501459 22839 raft_consensus.cc:2393] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 29: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:13.498567 22656 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 18 to 19, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 19 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:13.504259 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:13.505539 23464 raft_consensus.cc:2410] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 for term 24 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:15:13.502003 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:13.503803 22841 raft_consensus.cc:1240] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: []
I20260709 13:15:13.503846 22656 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 18 to 20, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 20 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.512279 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.513887 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.514123 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.513614 22656 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 17 to 19, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 19 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.515201 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.515849 22912 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:13.507230 23457 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:13.517256 22656 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:13.536171 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:13.537813 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.541877 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.542541 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.543529 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.544401 22907 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 28.
I20260709 13:15:13.546036 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.555955 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.556967 22907 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 23.
I20260709 13:15:13.534178 31353 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:13.577260 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:13.580742 22656 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 19 to 24, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 24 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.581609 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:13.578140 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.583974 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:13.591799 31366 raft_consensus.cc:493] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.592129 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56060.7us
I20260709 13:15:13.593492 22730 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:13.553892 31139 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.597419 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:13.592844 31366 raft_consensus.cc:515] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:13.609817 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.617231 31367 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:13.611371 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.613349 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:13.620591 22880 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.609827 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.614850 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.621685 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:13.619613 22807 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:13.622879 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:13.622068 22834 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:13.604925 31353 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:13.621263 22880 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.625835 30996 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:15:13.628855 30980 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:15:13.624114 22656 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 21 to 25, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 25 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:13.584482 31255 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:13.630149 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:13.635282 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.637782 31357 log.cc:927] Time spent T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:15:13.638888 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 45681.6us
I20260709 13:15:13.607491 31139 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:13.648984 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89357.4us
I20260709 13:15:13.654632 22879 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:13.629415 30980 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
W20260709 13:15:13.654029 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:13.632692 31350 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:15:13.666999 22806 leader_election.cc:336] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:13.668056 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.670979 31179 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.672426 31353 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.671859 22656 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 15, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 15 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:13.669236 30980 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:15:13.674125 31364 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:13.674577 31359 consensus_queue.cc:1048] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:15:13.681469 22806 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:13.677914 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:13.687820 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 69699us
I20260709 13:15:13.645727 31367 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:13.676573 31350 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.694223 22656 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:13.678840 31366 leader_election.cc:290] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.703938 22909 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:15:13.683857 30996 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:13.690126 31364 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:13.708516 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 49834.5us
W20260709 13:15:13.709355 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:13.709980 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:13.712380 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.713019 31255 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.713874 31371 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:13.714030 31346 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:13.715122 31139 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.715137 31355 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:13.716645 31037 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.721969 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:13.723652 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:13.724166 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:13.704141 31350 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:13.731050 30981 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:13.734628 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:13.745160 31298 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:13.725397 31366 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:15:13.747264 31364 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:13.717270 31037 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.696781 31179 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.752836 22656 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:13.754528 22656 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 18 to 20, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 20 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.758497 22656 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 19 to 22, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 22 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:13.776957 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 69041us
W20260709 13:15:13.787045 21636 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:13.815526 31179 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:13.837975 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 94166us
W20260709 13:15:13.839319 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 170757us
W20260709 13:15:13.834018 22880 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 161 similar messages]
I20260709 13:15:13.854383 31037 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:13.841779 22656 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 17 to 20, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 20 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.858493 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:13.858242 30996 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.859915 31141 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:13.860392 31141 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.860755 31141 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:13.816326 31367 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.856966 22656 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.867723 31367 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.868954 22656 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:15:13.879657 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.885746 31350 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.871472 31144 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:13.890965 31144 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:13.893361 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ba1db22a7176432c86e7264f87984ba3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:15:13.902521 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 262807us
I20260709 13:15:13.906755 30981 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:13.908051 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.870106 31367 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.919687 22912 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 24. Ops: []
W20260709 13:15:13.921473 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:13.922040 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.931083 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:13.931800 22907 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:15:13.935065 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 311536us
I20260709 13:15:13.942346 22730 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:13.938482 30980 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:13.955577 31144 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.963187 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:13.969341 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:13.973119 22911 raft_consensus.cc:1240] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 13:15:13.983673 31350 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:13.944998 31367 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.987262 31367 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:13.987848 22915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:13.964260 30980 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:13.995013 31380 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.995674 31380 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.000358 31364 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.001493 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:14.002161 22832 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:14.010799 31141 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.011533 22832 raft_consensus.cc:2468] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:15:14.014190 31141 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:14.015232 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.017023 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.021755 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.022075 22833 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 28: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:14.022550 22911 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 24.
I20260709 13:15:14.024606 31367 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:14.025708 31380 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:14.027407 22879 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:14.028429 31342 raft_consensus.cc:2804] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:15:14.029078 31342 raft_consensus.cc:697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:14.030140 31342 consensus_queue.cc:237] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.035053 31364 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:14.039499 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:14.040006 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 109660us
W20260709 13:15:14.041529 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:14.043062 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 114515us
I20260709 13:15:14.044786 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.045773 31380 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:14.046541 31380 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:14.048342 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:14.049880 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.060232 22907 raft_consensus.cc:2468] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 25.
I20260709 13:15:14.068141 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:14.072638 31355 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:14.073253 31355 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.073683 31355 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:14.085645 31380 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:15:14.102423 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54163us
I20260709 13:15:14.103626 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:14.107609 31380 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:14.109409 31380 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:14.109807 23464 raft_consensus.cc:1275] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:15:14.110502 31380 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:14.112295 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.112804 22806 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:14.114053 22806 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.115690 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.116032 31353 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:14.116770 31353 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:14.116863 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.117308 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.118173 31375 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:14.118462 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.118901 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.119863 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:14.134038 22910 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:14.121554 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.124974 31298 consensus_queue.cc:1048] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:14.126631 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:14.132578 22729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.146152 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:14.146862 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.147449 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.148962 31366 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:14.170665 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 44136.2us
I20260709 13:15:14.154083 30996 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:14.180464 22730 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.166282 31141 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.164263 31139 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:14.145817 31364 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 13:15:14.194563 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:14.200901 23432 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.201736 31376 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:14.209774 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:14.210592 22880 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.192773 31141 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.215773 31141 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:14.186446 31366 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.189289 31367 consensus_queue.cc:1059] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:15:14.218802 31367 raft_consensus.cc:3055] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:15:14.219180 31367 raft_consensus.cc:740] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:14.219429 31366 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:14.220428 31301 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:14.220047 31367 consensus_queue.cc:260] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.209872 31144 raft_consensus.cc:2804] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:15:14.223443 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:14.230705 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.232899 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:14.236804 22806 leader_election.cc:336] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.237263 22806 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.238365 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:14.241472 31139 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:14.243131 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:14.246714 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 72852.2us
I20260709 13:15:14.251030 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.251808 22906 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 26.
W20260709 13:15:14.252326 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.234026 31371 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:14.255343 31375 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:14.276141 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.255919 31144 raft_consensus.cc:697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:14.280416 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.286792 31157 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:14.299000 22911 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 28.
I20260709 13:15:14.304653 31144 consensus_queue.cc:237] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.308274 31371 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.311205 31371 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:14.319587 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 62032.2us
I20260709 13:15:14.288671 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.288794 22910 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:14.295799 31376 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.339195 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:14.294793 31367 raft_consensus.cc:3060] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:15:14.340126 22913 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 776baf137f8745dd866379ca15ab8cee in current term 25: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:14.344558 23432 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.350003 22732 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:14.348150 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:14.353168 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.354553 31360 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:14.354987 31360 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.355309 31360 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:14.355629 31355 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:14.359385 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 70806.3us
I20260709 13:15:14.360458 22730 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:14.361449 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:14.374326 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:14.347564 31366 raft_consensus.cc:3060] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:15:14.358775 30980 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:14.328590 31376 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.379698 22730 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:14.370373 22806 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:14.362267 31380 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:14.366645 31255 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 13:15:14.394176 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:14.388612 22806 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.378172 31364 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.387658 31157 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:14.410281 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.402035 31353 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:14.403797 31255 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:14.427207 31255 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.443532 23457 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:14.446923 31139 consensus_queue.cc:1059] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:14.447782 31139 consensus_queue.cc:1243] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Peer 74c0a02901bc45c891e42d0d1bbbf6a2 log is divergent from this leader: its last log entry 25.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:15:14.448388 31139 raft_consensus.cc:3055] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:15:14.448743 31139 raft_consensus.cc:740] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:14.450120 31139 consensus_queue.cc:260] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.462714 31355 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:14.466830 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:14.467823 22911 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:14.410797 31301 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:14.469470 22913 raft_consensus.cc:1240] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Rejecting Update request from peer 776baf137f8745dd866379ca15ab8cee for earlier term 24. Current term is 25. Ops: []
I20260709 13:15:14.482826 31360 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.485386 31360 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:14.450028 31391 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:15:14.511960 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 99899.1us
I20260709 13:15:14.511960 31139 raft_consensus.cc:3060] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:14.523712 31376 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:14.526607 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 54244us
I20260709 13:15:14.529165 22911 raft_consensus.cc:2468] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 25.
I20260709 13:15:14.530243 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:14.533123 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:14.529678 22806 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:14.534698 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 155111us
I20260709 13:15:14.510241 31380 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:14.524513 31366 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:15:14.535463 31382 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.555238 31350 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.558533 31350 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.551546 31380 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.552143 31382 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:14.575484 22807 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:14.561216 31350 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:14.577921 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 44790.9us
I20260709 13:15:14.586297 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:14.583395 31355 raft_consensus.cc:2804] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:15:14.593787 22838 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 23.
W20260709 13:15:14.643251 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:14.653805 31399 log.cc:927] Time spent T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
I20260709 13:15:14.592656 31382 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:14.680088 22913 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:15:14.684031 31353 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:14.694739 31376 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:14.744040 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
W20260709 13:15:14.750202 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 159873us
I20260709 13:15:14.749646 31139 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:14.787609 21636 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:14.756681 31355 raft_consensus.cc:697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 25 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:14.821540 22832 raft_consensus.cc:1275] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:15:14.823514 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 68492.1us
W20260709 13:15:14.839648 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 81565.5us
I20260709 13:15:14.856256 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:14.857201 22840 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:15:14.897131 31355 consensus_queue.cc:237] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:14.899618 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 2.124s (SENT)
I20260709 13:15:14.900300 23431 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:14.922914 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:14.776636 31139 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:14.922367 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:14.929649 22906 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:15:14.937253 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:14.950294 31346 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:14.954919 31346 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:14.955788 31346 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:14.968169 23458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:14.971144 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:14.920460 31360 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:14.973680 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 83871.2us
W20260709 13:15:14.977308 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 77761.5us
W20260709 13:15:14.977782 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 43979.7us
I20260709 13:15:14.978377 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:14.978962 23431 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Timed out: RequestConsensusVote RPC to 127.21.33.1:45095 timed out after 1.910s (SENT)
I20260709 13:15:14.977312 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:14.984531 23462 raft_consensus.cc:1240] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 26. Current term is 27. Ops: []
I20260709 13:15:14.994911 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:14.998370 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:14.998993 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.002785 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.008800 22838 raft_consensus.cc:1240] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 22. Current term is 24. Ops: []
I20260709 13:15:15.002015 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.008610 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e701ce8413c4f11afc5ca006f97eb50" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.005095 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:14.928788 31382 consensus_queue.cc:1048] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
W20260709 13:15:15.024061 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 101784us
I20260709 13:15:15.030464 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:14.986238 31380 raft_consensus.cc:493] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.031219 22838 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:15.009706 31398 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:15.034683 23431 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.001227 31387 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:15.020383 31367 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.056552 22841 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 13:15:15.041131 31301 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.040781 31412 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:15.073740 22832 raft_consensus.cc:1275] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 13:15:15.074216 22832 pending_rounds.cc:85] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 15
I20260709 13:15:15.074519 22832 pending_rounds.cc:107] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 13:15:15.072661 22730 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:15.045434 31387 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.074788 22832 raft_consensus.cc:2889] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:15.085355 22731 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.061448 31376 raft_consensus.cc:493] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:15.088804 22880 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:15.093007 22880 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.069435 31366 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.050271 31398 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.104789 22840 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:15:15.079175 31387 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:15.106228 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.070755 31367 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:15.064987 31380 raft_consensus.cc:515] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:15.107914 31360 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.028s
I20260709 13:15:15.097265 31301 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.138212 31367 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:15.138960 31301 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.122062 31396 raft_consensus.cc:3060] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:15.130143 31376 raft_consensus.cc:515] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.146579 31376 leader_election.cc:290] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.133306 31366 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.147192 31376 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.147691 31376 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.148824 31366 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:15.149708 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 43612.6us
W20260709 13:15:15.121469 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 55573.5us
I20260709 13:15:15.151399 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:15.128805 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105578us
I20260709 13:15:15.144469 31380 leader_election.cc:290] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:15.113762 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.171458 22838 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 26.
I20260709 13:15:15.162386 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:15.175233 23431 leader_election.cc:336] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.175063 22906 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 28.
W20260709 13:15:15.212306 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:15.217216 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 54916.7us
W20260709 13:15:15.120565 31416 log.cc:927] Time spent T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:15:15.217805 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 56540.8us
I20260709 13:15:15.129197 31398 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.233762 22914 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 13:15:15.237071 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:15.238422 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:15.090603 31412 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.193460 31366 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:15.242340 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:15.258116 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 105736us
W20260709 13:15:15.264340 31427 log.cc:927] Time spent T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:15:15.271903 31412 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.228147 31301 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:15.177313 31367 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.078s
I20260709 13:15:15.218979 31348 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
W20260709 13:15:15.288043 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:15.243875 31376 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:15.293627 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 232978us
I20260709 13:15:15.301442 31396 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:15.315399 31144 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 13:15:15.317441 31398 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:15.320358 31360 consensus_queue.cc:1048] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
W20260709 13:15:15.322461 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 94512.7us
I20260709 13:15:15.323683 22730 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:15.325692 31396 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:15.326542 31396 consensus_queue.cc:1059] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:15:15.328249 31301 raft_consensus.cc:3055] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:15:15.328647 31301 raft_consensus.cc:740] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:15.329609 31301 consensus_queue.cc:260] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:15.330925 31301 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:15.334621 31139 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:15.334530 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 55917us
W20260709 13:15:15.336207 23431 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:15.325256 22730 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:15.342023 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:15.342597 31346 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.346655 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:15.347441 31398 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:15.344944 31346 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:15.362470 22730 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:15.364328 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:15.375217 22806 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:15.378535 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 140178us
I20260709 13:15:15.396579 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:15.404536 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:15.405678 22833 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:15.416889 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.419251 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.420311 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.423066 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.425542 22906 raft_consensus.cc:2393] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 25: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:15.438457 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:15.438524 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.392663 31398 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:15.451228 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.438133 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.442070 31366 raft_consensus.cc:493] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.455520 23464 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:15:15.455869 31366 raft_consensus.cc:515] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.449265 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:15.396723 31371 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:15:15.393378 31396 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:15.480314 22833 raft_consensus.cc:2468] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 24.
I20260709 13:15:15.488713 31371 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:15:15.512187 22730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.514851 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.517266 22879 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:15.519624 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.519757 31235 raft_consensus.cc:2804] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:15:15.520569 31235 raft_consensus.cc:697] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:15.520736 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c406ce5efa449d4b877f6747d1f14ab" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.523272 22832 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:15:15.523732 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd464cd43c544f7abc90c38571db4d1c" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.524127 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.533783 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.534049 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a537de05aee48178205ec0e7e0d436e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.489360 31371 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:15:15.534910 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c6d83f2864eedb2775b615045955e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.540818 22730 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.524781 22836 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
W20260709 13:15:15.489236 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:15.560011 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 70837.5us
I20260709 13:15:15.511921 31380 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:15.561023 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:15.601912 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:15.573140 31398 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:15.602761 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:15.575572 31235 consensus_queue.cc:237] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:15.606678 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:15.607287 31366 leader_election.cc:290] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.543804 31425 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:15.574303 31426 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:15.620128 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 138865us
I20260709 13:15:15.638047 22914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.665324 22906 raft_consensus.cc:1240] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: []
W20260709 13:15:15.664537 22730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 123792us
I20260709 13:15:15.670377 22730 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.636544 31367 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.588802 31380 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.679237 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.679296 22910 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
W20260709 13:15:15.676505 22649 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0ad66b5c9842437d9320507b92b78fd5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 74c0a02901bc45c891e42d0d1bbbf6a2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:15.695168 22730 leader_election.cc:304] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.697960 22730 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.703586 31366 raft_consensus.cc:2749] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:15.694018 31367 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.706840 31367 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.707441 31367 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:15.717648 31366 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:15.684377 31426 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:15.716321 31437 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:15.733369 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:15.764963 22880 leader_election.cc:336] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:15.768718 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 252786us
W20260709 13:15:15.758291 31387 consensus_meta.cc:287] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.007s
W20260709 13:15:15.773694 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87067.7us
W20260709 13:15:15.776053 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:15.776886 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:15.777994 23431 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.780005 22730 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:15.788197 21636 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:15.789490 31367 raft_consensus.cc:493] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.794517 31367 raft_consensus.cc:515] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:15.780937 31387 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.797362 22908 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 26: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:15.796844 31367 leader_election.cc:290] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:15.707342 31425 consensus_queue.cc:1059] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:15.798724 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:15.790992 31366 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:15.783735 31396 consensus_queue.cc:1059] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:15:15.814906 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:15.814313 22807 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:15.822407 31407 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:15.825223 22730 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.816771 31425 consensus_queue.cc:1243] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 6545453e74344bb2957dd3729c5b3870 log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:15:15.814136 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:15.828425 31395 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:15.830916 22730 leader_election.cc:304] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.831826 31437 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:15.833741 31437 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:15.834451 31437 consensus_queue.cc:1243] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [LEADER]: Peer 776baf137f8745dd866379ca15ab8cee log is divergent from this leader: its last log entry 23.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:15.835934 31437 raft_consensus.cc:2749] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:15.838064 22730 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.839155 31425 raft_consensus.cc:3060] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:15.839340 31380 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:15.820103 31371 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:15.843695 31371 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:15.841686 22908 raft_consensus.cc:1275] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:15:15.846755 22730 leader_election.cc:304] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.847716 31301 raft_consensus.cc:2749] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:15.848297 23431 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.849098 22730 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:15.853751 31301 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:15:15.854008 23431 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.854789 23431 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:15.855031 22730 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.855348 31426 raft_consensus.cc:3055] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:15.855774 31426 raft_consensus.cc:740] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:15.855914 31380 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:15.856683 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.857239 31141 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:15:15.858609 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:15.859017 23431 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:15.859124 31367 raft_consensus.cc:3055] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:15.856688 31426 consensus_queue.cc:260] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:15.859582 31367 raft_consensus.cc:740] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:15.860369 31426 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:15.854214 31301 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.860535 31367 consensus_queue.cc:260] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:15.861348 31301 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:15.862339 31367 raft_consensus.cc:3060] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:15.854035 31387 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:15.863514 31434 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:15:15.871937 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
W20260709 13:15:15.862632 31443 log.cc:927] Time spent T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.064s	user 0.003s	sys 0.000s
I20260709 13:15:15.890978 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:15.891772 22839 raft_consensus.cc:3060] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:15.892341 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:15.893774 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8670cf017371429ea31529b6f247493e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.902170 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:15.882514 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:15.907619 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.910367 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55844ac3c9a4ceaa35b3d5bdadad40e" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:15.912891 22730 leader_election.cc:304] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.909770 22837 raft_consensus.cc:1217] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Deduplicated request from leader. Original: 16.10->[22.11-26.13]   Dedup: 22.11->[24.12-26.13]
I20260709 13:15:15.923830 22837 pending_rounds.cc:85] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Aborting all ops after (but not including) 11
I20260709 13:15:15.925158 22730 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.926568 31380 raft_consensus.cc:2749] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:15.928404 23466 raft_consensus.cc:1240] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Rejecting Update request from peer acfda9b350734def9b6c621f66a5f247 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:15:15.931340 22730 leader_election.cc:304] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:15.934054 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 244883us
I20260709 13:15:15.883577 31425 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:15.935384 22837 pending_rounds.cc:107] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 13:15:15.936851 22837 raft_consensus.cc:2889] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:15.933907 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:15.941434 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:15.905802 31395 raft_consensus.cc:493] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.937206 22659 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 24 to 29, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 29 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:15.935604 31396 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:15.953727 22659 catalog_manager.cc:5697] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.945461 31380 raft_consensus.cc:2749] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:15.983626 22840 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Rejecting Update request from peer 74c0a02901bc45c891e42d0d1bbbf6a2 for earlier term 28. Current term is 29. Ops: []
I20260709 13:15:15.979970 22659 catalog_manager.cc:5697] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 18 to 21, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 21 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.977825 31425 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:16.029161 22656 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 270) took 30373 ms (client timeout 15000 ms). Trace:
W20260709 13:15:16.029738 22656 rpcz_store.cc:269] 0709 13:14:45.655637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.383632 (+10727995us) service_pool.cc:224] Handling call
0709 13:15:16.029069 (+19645437us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:14.296180 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:14.296879 (+   699us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:14.296934 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:15:14.296953 (+    19us) tablet.cc:662] Decoding operations
0709 13:15:14.298966 (+  2013us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:14.299064 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:14.299092 (+    28us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:15:14.406076 (+106984us) tablet.cc:701] Row locks acquired
0709 13:15:14.406126 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:15:14.406395 (+   269us) write_op.cc:270] Start()
0709 13:15:14.406562 (+   167us) write_op.cc:276] Timestamp: P: 1783602914406320 usec, L: 0
0709 13:15:14.406595 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:15:14.407466 (+   871us) log.cc:844] Serialized 9660 byte log entry
0709 13:15:14.874342 (+466876us) op_driver.cc:464] REPLICATION: finished
0709 13:15:15.063760 (+189418us) write_op.cc:301] APPLY: starting
0709 13:15:15.063920 (+   160us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:15.353010 (+289090us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:15.353134 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:15.755394 (+402260us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:15.755966 (+   572us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:15:15.756042 (+    76us) write_op.cc:312] APPLY: finished
0709 13:15:15.756847 (+   805us) log.cc:844] Serialized 220 byte log entry
0709 13:15:15.779802 (+ 22955us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:15.794124 (+ 14322us) write_op.cc:454] Released schema lock
0709 13:15:15.794235 (+   111us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:15.795168 (+   933us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:15.795200 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":150053,"apply.run_cpu_time_us":41417,"apply.run_wall_time_us":749270,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"cfile_init":1,"lbm_read_time_us":534,"lbm_reads_lt_1ms":4,"num_ops":25,"prepare.queue_time_us":134748,"prepare.run_cpu_time_us":20937,"prepare.run_wall_time_us":271687,"raft.queue_time_us":218662,"raft.run_cpu_time_us":12734,"raft.run_wall_time_us":93245,"replication_time_us":467505,"thread_start_us":367244,"threads_started":4,"wal-append.queue_time_us":143554}]]}
I20260709 13:15:15.960837 31395 raft_consensus.cc:515] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.034446 22659 catalog_manager.cc:5697] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 16 to 22, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 22 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.036314 22659 catalog_manager.cc:5697] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 21 to 23, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 23 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:16.044812 31395 leader_election.cc:290] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.051195 31429 consensus_queue.cc:1048] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.102s
I20260709 13:15:16.068045 22839 raft_consensus.cc:2468] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:15:16.041706 22659 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:16.079481 22659 catalog_manager.cc:5697] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 14 to 22, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 22 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.097437 22807 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:16.097561 22840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.081189 31395 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.110610 22659 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 24 to 25, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 25 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.122604 31413 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:15:16.131803 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "961d7db1909a4e40a4b410a32dad4d64" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.132969 31376 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.131909 22879 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:16.133980 31395 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.136157 31395 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.136677 31395 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.136905 31451 raft_consensus.cc:2804] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:16.137545 31451 raft_consensus.cc:697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 26 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:16.138597 31449 raft_consensus.cc:493] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.138484 31451 consensus_queue.cc:237] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.139168 31449 raft_consensus.cc:515] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.120445 22659 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 10 to 23, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 23 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.143813 22732 leader_election.cc:304] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.133634 31376 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.151113 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.152927 22835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.144003 22659 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 15 to 18, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 18 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:16.141316 31449 leader_election.cc:290] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.137234 31395 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.169618 31437 raft_consensus.cc:2749] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:16.176234 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.186996 31437 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.187789 31437 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.190517 31437 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.192325 31395 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.194057 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e04261a736f42d2a1fd4e24bc787276" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.195277 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.195732 22834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.195874 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.194069 23457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:16.200132 31301 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.201988 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b88bd2b1b1d4ced918486ea751fbe25" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:16.203474 22880 leader_election.cc:336] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:16.204903 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:16.213562 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.216809 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a663ecdecbd14089a4074b2581e94517" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.225718 31380 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.226640 31380 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:16.228549 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:16.231056 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:16.234629 22730 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.237088 22730 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.181232 31376 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.238698 31376 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:16.253870 22659 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.256649 22659 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.260764 31380 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.261476 31367 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:16.271605 22840 raft_consensus.cc:1275] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:15:16.273701 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.275775 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:16.290400 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 308649us
I20260709 13:15:16.311694 22659 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:16.327198 22732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 53610.3us
I20260709 13:15:16.328073 22732 leader_election.cc:304] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.330379 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:16.332263 22732 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.341189 22659 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.329809 31380 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:16.366189 22659 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:16.378724 22659 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:16.379647 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 52461.6us
I20260709 13:15:16.346767 31398 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:16.357894 31376 raft_consensus.cc:2749] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:16.424118 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:16.425313 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45195.7us
I20260709 13:15:16.422971 31395 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.444212 31395 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.447775 31301 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.427738 31418 consensus_queue.cc:1048] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:15:16.450950 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.451403 31418 raft_consensus.cc:493] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:16.451972 31418 raft_consensus.cc:515] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.452710 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:16.453456 22839 raft_consensus.cc:2393] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate acfda9b350734def9b6c621f66a5f247 in current term 26: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:16.433722 31367 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:16.481029 31460 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:16.462302 31144 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: INVALID_TERM, Last received: 28.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.020s
I20260709 13:15:16.482954 31144 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 13:15:16.483376 31144 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:16.484263 31144 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:16.479516 31432 log.cc:927] Time spent T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:15:16.446738 31395 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.485419 31144 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:15:16.527783 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61298.9us
I20260709 13:15:16.544914 31418 leader_election.cc:290] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:16.591518 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:16.617761 31465 log.cc:927] Time spent T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:15:16.651994 22836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.656226 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:16.656746 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.672925 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09deb1f2792d4748b024f33e42736814" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:16.710511 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:16.711596 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:16.712180 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:16.733132 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:16.734992 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:16.735563 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:16.737058 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.739812 31367 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:15:16.739833 22833 raft_consensus.cc:1275] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:15:16.741842 31395 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:16.755095 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:16.766463 22879 leader_election.cc:304] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:16.767925 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:16.772907 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:16.773654 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:16.774492 31455 raft_consensus.cc:2749] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:16.788869 21636 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:16.796625 31455 consensus_queue.cc:1048] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:16.826042 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:16.829607 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:16.833786 31367 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.834470 31367 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.836796 31367 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.838474 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:16.839216 23464 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 27.
I20260709 13:15:16.840595 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:16.840934 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.845918 31367 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:15:16.846396 31367 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.846747 31367 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:15:16.850831 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:16.858744 31301 raft_consensus.cc:493] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.859421 31301 raft_consensus.cc:515] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.863368 31301 leader_election.cc:290] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.864795 31425 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.864967 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:16.865363 31425 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.866803 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adf76c1fde374ccbbb4a0edda126b851" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.867822 31425 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.868722 22730 leader_election.cc:304] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.868811 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:16.870059 31425 raft_consensus.cc:2749] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:16.870309 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:16.880461 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.883352 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:16.884593 31425 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:16.889818 31425 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.890733 31425 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.895309 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.898921 31367 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.899598 31425 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.907948 31425 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.908505 31425 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:16.911437 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:16.914032 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:16.914642 23466 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:15:16.925567 31367 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:16.925319 23466 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 28.
I20260709 13:15:16.931603 22730 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:16.932479 31367 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:15:16.934705 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.933269 31367 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 28 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:16.940637 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:16.941319 22913 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 28
I20260709 13:15:16.947455 22732 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.945158 31425 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:16.954931 31301 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:16.939076 31367 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:16.972527 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:16.981719 22732 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:16.984668 22913 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 28.
I20260709 13:15:16.987475 31396 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:16.998728 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:16.993717 31425 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.017956 31425 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.065841 31395 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.066501 31395 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:17.082180 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:17.093690 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.102082 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f12b50ff391746348739bd5f7a08a44a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:17.106769 31395 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:17.109148 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:17.110935 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.101809 31342 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:17.113600 22730 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:17.112927 31342 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.116772 31342 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:17.101956 31425 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:17.117373 22913 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 23.
I20260709 13:15:17.118535 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:17.119350 23466 raft_consensus.cc:2393] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 24: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
I20260709 13:15:17.121166 22732 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:17.121837 31425 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:17.125288 31395 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:17.125797 31395 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.126180 31395 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:17.124568 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:17.130465 23466 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 24.
I20260709 13:15:17.136776 31395 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.140050 31395 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:17.140091 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:17.141144 22913 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:15:17.141176 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:17.142205 23466 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:15:17.157385 23466 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:17.158804 22730 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247; no voters: 
I20260709 13:15:17.160300 31395 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:15:17.161209 31395 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Becoming Leader. State: Replica: acfda9b350734def9b6c621f66a5f247, State: Running, Role: LEADER
I20260709 13:15:17.168273 31396 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.168699 22913 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acfda9b350734def9b6c621f66a5f247 in term 25.
I20260709 13:15:17.168974 31396 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.175055 31396 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:17.177318 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:17.178198 22659 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 140) took 15329 ms (client timeout 15000 ms). Trace:
I20260709 13:15:17.177378 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6da5381f5554dc69fb15991d351fb20" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:17.177897 31395 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:17.186687 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:17.192854 22730 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.195818 31396 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:17.192072 22880 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:17.202456 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:17.184293 22659 rpcz_store.cc:269] 0709 13:15:01.848937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.164992 (+11316055us) service_pool.cc:224] Handling call
0709 13:15:17.178074 (+4013082us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:16.591922 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:16.592691 (+   769us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:16.592733 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:15:16.592769 (+    36us) tablet.cc:662] Decoding operations
0709 13:15:16.594713 (+  1944us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:16.594822 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:16.594858 (+    36us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:15:16.596393 (+  1535us) tablet.cc:701] Row locks acquired
0709 13:15:16.596417 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:15:16.596613 (+   196us) write_op.cc:270] Start()
0709 13:15:16.596772 (+   159us) write_op.cc:276] Timestamp: P: 1783602916596551 usec, L: 0
0709 13:15:16.596798 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:15:16.679706 (+ 82908us) log.cc:844] Serialized 6247 byte log entry
0709 13:15:16.769759 (+ 90053us) op_driver.cc:464] REPLICATION: finished
0709 13:15:16.786409 (+ 16650us) write_op.cc:301] APPLY: starting
0709 13:15:16.786531 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:16.855690 (+ 69159us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:16.855810 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:17.118917 (+263107us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:17.119224 (+   307us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:15:17.119271 (+    47us) write_op.cc:312] APPLY: finished
0709 13:15:17.119645 (+   374us) log.cc:844] Serialized 148 byte log entry
0709 13:15:17.120497 (+   852us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:17.120756 (+   259us) write_op.cc:454] Released schema lock
0709 13:15:17.120839 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:17.121297 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:17.121325 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58963,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":16345,"apply.run_cpu_time_us":26174,"apply.run_wall_time_us":334865,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":83309,"prepare.run_cpu_time_us":6339,"prepare.run_wall_time_us":109846,"replication_time_us":172762,"spinlock_wait_cycles":1783168,"thread_start_us":60835,"threads_started":2}]]}
I20260709 13:15:17.206930 31458 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:17.226225 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:17.226971 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:17.259088 22658 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.290349 31395 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.291059 31395 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:17.299113 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:17.302189 22650 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 89549.7us
I20260709 13:15:17.306381 31395 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:17.306953 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.306954 31396 raft_consensus.cc:493] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.308964 31396 raft_consensus.cc:515] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:17.310503 22732 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.313421 31425 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:17.313032 31395 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.320231 31396 leader_election.cc:290] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:17.360358 31395 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:17.366025 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:17.321801 22657 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 266) took 126833 ms (client timeout 15000 ms). Trace:
I20260709 13:15:17.323684 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 30 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.325449 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25ae06dc4f8940f1a419a39f791d6e18" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 30 candidate_status { last_received { term: 23 index: 9 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:17.371486 22657 rpcz_store.cc:269] 0709 13:13:10.488808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.489368 (+   560us) service_pool.cc:224] Handling call
0709 13:15:17.321686 (+126832318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254668,"reported_tablets":51}
I20260709 13:15:17.367838 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:17.375903 22730 leader_election.cc:304] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.377281 31396 raft_consensus.cc:2749] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:15:17.378907 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
W20260709 13:15:17.387425 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:17.388216 31396 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:17.378129 31395 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:17.434916 31395 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.435561 31395 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:17.438661 31395 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:17.440313 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.446795 31395 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.447432 31395 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.455209 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:17.459264 31395 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:17.469650 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:17.484460 22712 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:17.485342 22712 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:17.485769 22712 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:17.494504 31395 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.497264 23466 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 13:15:17.499310 31396 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:15:17.495570 31395 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.515467 31395 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:17.515733 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:17.525727 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 87437.7us
I20260709 13:15:17.534041 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:17.537292 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:17.540238 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:17.542061 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.543483 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.544519 31396 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:17.544879 31425 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:17.555636 22730 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.559028 31396 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:15:17.574623 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:17.583216 22658 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 245) took 76992 ms (client timeout 15000 ms). Trace:
I20260709 13:15:17.586966 22913 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 13:15:17.583794 22658 rpcz_store.cc:269] 0709 13:14:00.591024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.178420 (+9587396us) service_pool.cc:224] Handling call
0709 13:15:17.583124 (+67404704us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:17.324855 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:17.325442 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:17.325484 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:15:17.325521 (+    37us) tablet.cc:662] Decoding operations
0709 13:15:17.325881 (+   360us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:17.325961 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:17.325994 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:17.326187 (+   193us) tablet.cc:701] Row locks acquired
0709 13:15:17.326206 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:17.328790 (+  2584us) write_op.cc:270] Start()
0709 13:15:17.328930 (+   140us) write_op.cc:276] Timestamp: P: 1783602917326301 usec, L: 0
0709 13:15:17.328957 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:15:17.352468 (+ 23511us) log.cc:844] Serialized 535 byte log entry
0709 13:15:17.414353 (+ 61885us) op_driver.cc:464] REPLICATION: finished
0709 13:15:17.415150 (+   797us) write_op.cc:301] APPLY: starting
0709 13:15:17.415239 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:17.416124 (+   885us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:17.416222 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:17.416943 (+   721us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:17.417151 (+   208us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:17.417203 (+    52us) write_op.cc:312] APPLY: finished
0709 13:15:17.417421 (+   218us) log.cc:844] Serialized 25 byte log entry
0709 13:15:17.417865 (+   444us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:17.417961 (+    96us) write_op.cc:454] Released schema lock
0709 13:15:17.418000 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:17.418458 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:17.418488 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":18217,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":522,"apply.run_cpu_time_us":3035,"apply.run_wall_time_us":3255,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":31181,"prepare.run_cpu_time_us":3036,"prepare.run_wall_time_us":28038,"replication_time_us":85198,"thread_start_us":23087,"threads_started":1}]]}
I20260709 13:15:17.577975 22660 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 14 to 17, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 17 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.589017 31396 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
W20260709 13:15:17.652215 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:17.671456 22913 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:15:17.673056 31425 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:15:17.677354 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:17.700081 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:17.721346 23464 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Refusing update from remote peer acfda9b350734def9b6c621f66a5f247: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:15:17.723800 31425 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:15:17.721359 22660 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 248) took 77170 ms (client timeout 15000 ms). Trace:
W20260709 13:15:17.726217 22660 rpcz_store.cc:269] 0709 13:14:00.550766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.107920 (+1557154us) service_pool.cc:224] Handling call
0709 13:15:17.721255 (+75613335us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:17.628442 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:17.629056 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:17.629093 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:15:17.629127 (+    34us) tablet.cc:662] Decoding operations
0709 13:15:17.629488 (+   361us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:17.629569 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:17.629599 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:17.629818 (+   219us) tablet.cc:701] Row locks acquired
0709 13:15:17.629837 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:17.629998 (+   161us) write_op.cc:270] Start()
0709 13:15:17.630112 (+   114us) write_op.cc:276] Timestamp: P: 1783602917629936 usec, L: 0
0709 13:15:17.630135 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:17.630789 (+   654us) log.cc:844] Serialized 530 byte log entry
0709 13:15:17.632761 (+  1972us) op_driver.cc:464] REPLICATION: finished
0709 13:15:17.633265 (+   504us) write_op.cc:301] APPLY: starting
0709 13:15:17.633354 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:17.634363 (+  1009us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:17.634463 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:17.635215 (+   752us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:17.635427 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:17.635490 (+    63us) write_op.cc:312] APPLY: finished
0709 13:15:17.635683 (+   193us) log.cc:844] Serialized 25 byte log entry
0709 13:15:17.635854 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:17.635949 (+    95us) write_op.cc:454] Released schema lock
0709 13:15:17.635990 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:17.636404 (+   414us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:17.636432 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7654,"reported_tablets":51,"spinlock_wait_cycles":81536,"child_traces":[["op",{"apply.queue_time_us":259,"apply.run_cpu_time_us":3101,"apply.run_wall_time_us":3101,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":161,"prepare.run_cpu_time_us":2652,"prepare.run_wall_time_us":2652,"replication_time_us":2455}]]}
W20260709 13:15:17.749256 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:17.767155 31425 raft_consensus.cc:493] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.767899 31425 raft_consensus.cc:515] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:17.773358 31425 leader_election.cc:290] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:17.774350 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.775889 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b00e5613b4864bd6a0873aee1004ed22" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:17.782407 22730 leader_election.cc:304] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:17.787813 31425 raft_consensus.cc:2749] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:17.789480 21636 thread.cc:527] Waited for 11000ms trying to join with heartbeat (tid 22791)
W20260709 13:15:17.801983 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:17.806721 22658 catalog_manager.cc:5697] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 29 to 31, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 31 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.810117 22658 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 19 to 26. New cstate: current_term: 26 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.812737 22658 catalog_manager.cc:5697] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 14 to 17, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.814992 22658 catalog_manager.cc:5697] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 18 to 22, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 22 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.826295 22658 catalog_manager.cc:5697] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 18 to 23, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 23 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:17.841677 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:17.843019 22661 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 268) took 77621 ms (client timeout 15000 ms). Trace:
W20260709 13:15:17.843559 22661 rpcz_store.cc:269] 0709 13:14:00.221384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.221803 (+   419us) service_pool.cc:224] Handling call
0709 13:14:00.421376 (+199573us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5b090 00007f68a1af9f3a 00007f68a78b01cc 00007f68a79ae37e 00007f68a5e6f2a4 00007f68a5e6f232 00007f68a5e6f161 00007f68a5e6f0dc 00007f68a5e6e392 00007f689fe2c48c 00007f689fe2b8c6
0709 13:15:17.842726 (+77421350us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14000,"reported_tablets":51,"spinlock_wait_cycles":83920768}
I20260709 13:15:17.845825 22658 catalog_manager.cc:5697] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.847777 22658 catalog_manager.cc:5697] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 15 to 21, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 21 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.849426 22658 catalog_manager.cc:5697] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 23 to 27, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 27 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.850896 22658 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 18 to 20, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 20 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.853430 22658 catalog_manager.cc:5697] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 15 to 19, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 19 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.855283 22658 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 17 to 26, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 26 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.856945 22658 catalog_manager.cc:5697] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.858714 22658 catalog_manager.cc:5697] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.865419 22658 catalog_manager.cc:5697] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 15 to 19, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.868019 22658 catalog_manager.cc:5697] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:17.874734 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:17.876276 22658 catalog_manager.cc:5697] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.882071 22658 catalog_manager.cc:5697] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 19 to 20, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.883790 22658 catalog_manager.cc:5697] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 19 to 21, leader changed from 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 21 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.886020 22658 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:17.895665 22658 catalog_manager.cc:5697] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.913483 31425 raft_consensus.cc:493] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.914276 31425 raft_consensus.cc:515] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:17.918298 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:17.922617 31425 leader_election.cc:290] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:17.924225 31395 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.924808 31395 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:17.925665 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f984a583c84fbe9ef7b3c817b06c44" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:17.927177 31395 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:17.928990 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:17.944427 22732 leader_election.cc:304] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:17.946616 31396 raft_consensus.cc:2749] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:17.971237 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:17.995065 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 57717.8us
I20260709 13:15:17.997191 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:18.004835 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50925.5us
I20260709 13:15:17.993813 31423 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
W20260709 13:15:18.013861 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57170.3us
I20260709 13:15:18.011056 31395 raft_consensus.cc:493] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.017277 31395 raft_consensus.cc:515] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:18.019975 31395 leader_election.cc:290] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:18.014405 31423 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.023526 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.024241 31423 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:18.036770 31425 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.037468 31425 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:18.039851 31425 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:18.039907 23431 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.051378 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:18.060498 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 56466.8us
W20260709 13:15:18.064338 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.064489 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.065838 31425 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:18.068147 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.069155 22832 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 29: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:18.069898 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:18.070640 23466 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 29.
I20260709 13:15:18.074189 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb34c8c8aec0452ab45aaa676b75558a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:18.075191 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:18.078226 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:18.097889 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:18.093263 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:18.100558 31458 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:15:18.101343 31458 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:18.102000 31458 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:15:18.109974 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.110984 22732 leader_election.cc:304] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.112838 31425 raft_consensus.cc:2749] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:18.113796 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.114748 31425 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:18.119889 31458 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.122761 31458 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:18.126139 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:18.127056 22832 raft_consensus.cc:2393] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 29: Already voted for candidate 776baf137f8745dd866379ca15ab8cee in this term.
I20260709 13:15:18.129731 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:18.148018 23464 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 29.
I20260709 13:15:18.149813 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.151242 31458 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:15:18.153733 31458 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 29 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:18.154749 31458 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.163360 23464 consensus_queue.cc:237] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.169574 22658 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58552 (request call id 141) took 1223 ms. Trace:
I20260709 13:15:18.170192 22658 rpcz_store.cc:276] 0709 13:15:16.947170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.605892 (+658722us) service_pool.cc:224] Handling call
0709 13:15:18.169431 (+563539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:17.916646 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:17.917389 (+   743us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:17.917433 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:15:17.917445 (+    12us) tablet.cc:662] Decoding operations
0709 13:15:17.919135 (+  1690us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:17.919201 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:17.919237 (+    36us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:15:17.920877 (+  1640us) tablet.cc:701] Row locks acquired
0709 13:15:17.920907 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:15:17.921110 (+   203us) write_op.cc:270] Start()
0709 13:15:17.921252 (+   142us) write_op.cc:276] Timestamp: P: 1783602917921045 usec, L: 0
0709 13:15:17.921273 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:15:17.931900 (+ 10627us) log.cc:844] Serialized 7934 byte log entry
0709 13:15:17.934921 (+  3021us) op_driver.cc:464] REPLICATION: finished
0709 13:15:17.935452 (+   531us) write_op.cc:301] APPLY: starting
0709 13:15:17.935552 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:17.974599 (+ 39047us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:17.974749 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:18.085860 (+111111us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:18.086204 (+   344us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:15:18.086254 (+    50us) write_op.cc:312] APPLY: finished
0709 13:15:18.086729 (+   475us) log.cc:844] Serialized 180 byte log entry
0709 13:15:18.087581 (+   852us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:18.087899 (+   318us) write_op.cc:454] Released schema lock
0709 13:15:18.087965 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:18.088453 (+   488us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:18.088481 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37132,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":310,"apply.run_cpu_time_us":28895,"apply.run_wall_time_us":152934,"cfile_cache_hit":80,"cfile_cache_hit_bytes":83640,"num_ops":20,"prepare.queue_time_us":11658,"prepare.run_cpu_time_us":5928,"prepare.run_wall_time_us":15644,"replication_time_us":13462}]]}
W20260709 13:15:18.190359 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.223588 22906 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:18.227627 22833 raft_consensus.cc:1275] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:18.229113 22658 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:18.231156 22658 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 17 to 28, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 28 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:18.236303 31496 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:18.249564 31496 consensus_queue.cc:1048] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:15:18.312141 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:18.322605 31441 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 4 to 13, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:18.333953 22833 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 4 to 13, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:18.340941 31483 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.341672 31483 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.344017 31483 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:18.345940 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:18.346634 23464 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 29.
I20260709 13:15:18.348677 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:18.348987 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:18.350934 31440 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:15:18.351395 31440 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.351496 31425 raft_consensus.cc:493] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.351974 31440 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:15:18.352173 31425 raft_consensus.cc:515] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } attrs { replace: true } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.356343 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.357758 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:18.360388 22730 leader_election.cc:304] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.360548 31425 leader_election.cc:290] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:18.361304 31396 raft_consensus.cc:2749] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:18.361706 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f666ea4542d4951919f36de0b43b972" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:18.374536 22649 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f666ea4542d4951919f36de0b43b972 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:18.377048 31440 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:18.380582 22662 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 243) took 129171 ms (client timeout 15000 ms). Trace:
W20260709 13:15:18.390476 22662 rpcz_store.cc:269] 0709 13:13:09.209718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.210050 (+   332us) service_pool.cc:224] Handling call
0709 13:15:18.380347 (+129170297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38369,"reported_tablets":51,"spinlock_wait_cycles":3456}
W20260709 13:15:18.395073 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:18.395048 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:18.402819 31440 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 election: Requested vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:18.405344 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:18.408861 23464 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:15:18.430209 22906 raft_consensus.cc:2955] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 4 to 13, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:18.434016 23464 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 776baf137f8745dd866379ca15ab8cee in term 30.
W20260709 13:15:18.440691 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:18.445091 22805 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870
I20260709 13:15:18.446095 31440 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:15:18.446683 31440 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 30 LEADER]: Becoming Leader. State: Replica: 776baf137f8745dd866379ca15ab8cee, State: Running, Role: LEADER
I20260709 13:15:18.447865 31440 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.483098 22658 catalog_manager.cc:5697] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: config changed from index 4 to 13, VOTER acfda9b350734def9b6c621f66a5f247 (127.21.33.1) evicted. New cstate: current_term: 20 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:18.501737 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:18.506930 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:15:18.514690 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:18.524655 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:18.557755 31396 raft_consensus.cc:493] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.558408 31396 raft_consensus.cc:515] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:18.561599 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.562417 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a18b5b24775b48b589edb90581a5cad5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:18.564355 23464 raft_consensus.cc:1240] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Rejecting Update request from peer 6545453e74344bb2957dd3729c5b3870 for earlier term 29. Current term is 30. Ops: []
I20260709 13:15:18.564282 22732 leader_election.cc:304] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.565451 31425 raft_consensus.cc:2749] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:18.569927 31396 leader_election.cc:290] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:18.569070 31458 consensus_queue.cc:1059] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 13:15:18.570812 31458 raft_consensus.cc:3055] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 13:15:18.571204 31458 raft_consensus.cc:740] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:18.572203 31458 consensus_queue.cc:260] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.23, Last appended by leader: 23, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:18.574087 31458 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:15:18.580719 22663 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 265) took 149930 ms (client timeout 15000 ms). Trace:
W20260709 13:15:18.589367 22663 rpcz_store.cc:269] 0709 13:12:48.664244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.704919 (+ 40675us) service_pool.cc:224] Handling call
0709 13:15:18.580489 (+149875570us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8671,"reported_tablets":51}
I20260709 13:15:18.591876 22655 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 16 to 20, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 20 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.593794 22655 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.595362 22655 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 22 to 23, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 23 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.597579 22655 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 16 to 19, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.599587 22655 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 15 to 17, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 17 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:18.601333 31396 raft_consensus.cc:493] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.600926 22655 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:18.601956 31396 raft_consensus.cc:515] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:18.605729 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.608467 31396 leader_election.cc:290] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:18.609912 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ede022de6a346e284376d4212e1f6b7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:18.611116 22655 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.612043 22732 leader_election.cc:304] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.613448 31396 raft_consensus.cc:2749] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:18.617121 22655 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.629495 22655 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 18 to 21, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.655290 31396 raft_consensus.cc:493] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.655840 31396 raft_consensus.cc:515] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:18.658151 31396 leader_election.cc:290] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:18.664227 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.675027 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "072f69118119423dac7eba3f66879eee" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:18.681187 22732 leader_election.cc:304] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.683037 31396 raft_consensus.cc:2749] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:15:18.736824 22655 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 247) took 23001 ms (client timeout 15000 ms). Trace:
I20260709 13:15:18.744339 31396 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.745095 31396 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:18.748759 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:18.749032 31396 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:18.751641 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0818ffaecde64fc0af038b3c024b6008" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:18.758126 22730 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:18.759560 31396 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:18.737370 22655 rpcz_store.cc:269] 0709 13:14:55.843532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.750282 (+8906750us) service_pool.cc:224] Handling call
0709 13:15:18.736708 (+13986426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:18.636092 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:18.636613 (+   521us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:18.636652 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:15:18.636669 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:18.637595 (+   926us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:18.637664 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:18.637690 (+    26us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:15:18.638438 (+   748us) tablet.cc:701] Row locks acquired
0709 13:15:18.638457 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:18.638586 (+   129us) write_op.cc:270] Start()
0709 13:15:18.638702 (+   116us) write_op.cc:276] Timestamp: P: 1783602918638543 usec, L: 0
0709 13:15:18.638722 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:15:18.639702 (+   980us) log.cc:844] Serialized 3592 byte log entry
0709 13:15:18.641735 (+  2033us) op_driver.cc:464] REPLICATION: finished
0709 13:15:18.642092 (+   357us) write_op.cc:301] APPLY: starting
0709 13:15:18.642180 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:18.647846 (+  5666us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:18.647975 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:18.673938 (+ 25963us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:18.674201 (+   263us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:15:18.674247 (+    46us) write_op.cc:312] APPLY: finished
0709 13:15:18.674549 (+   302us) log.cc:844] Serialized 89 byte log entry
0709 13:15:18.681672 (+  7123us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:18.681819 (+   147us) write_op.cc:454] Released schema lock
0709 13:15:18.681870 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:18.682331 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:18.682360 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":51,"spinlock_wait_cycles":10624,"child_traces":[["op",{"apply.queue_time_us":125,"apply.run_cpu_time_us":12226,"apply.run_wall_time_us":40180,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37638,"num_ops":9,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":3642,"prepare.run_wall_time_us":5591,"replication_time_us":2886}]]}
I20260709 13:15:18.784101 22664 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 19 to 24, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 24 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:18.785820 22664 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 19 to 20, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.787341 22664 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 20 to 21, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 21 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:18.790035 21636 thread.cc:527] Waited for 12000ms trying to join with heartbeat (tid 22791)
I20260709 13:15:18.789462 22664 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.798843 22664 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.811790 22664 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 16 to 23, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 23 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.813822 22664 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 18 to 23, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 23 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.815694 22664 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 16 to 24, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 24 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.818173 22664 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 18 to 20, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 20 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:18.824595 22664 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 20 to 26. New cstate: current_term: 26 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.826762 22664 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 20 to 21, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 21 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.828213 22664 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 21 to 22, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 22 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:18.834040 22664 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 20 to 21, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 21 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.836376 22664 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 19 to 20, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:18.840746 22650 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer [suppressed 365 similar messages]
I20260709 13:15:18.850044 22664 catalog_manager.cc:5697] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.900429 22913 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 29 index: 23. Preceding OpId from leader: term: 30 index: 23. (term mismatch)
I20260709 13:15:18.901244 22913 pending_rounds.cc:85] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Aborting all ops after (but not including) 22
I20260709 13:15:18.901546 22913 pending_rounds.cc:107] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: Aborting uncommitted NO_OP operation due to leader change: 29.23
I20260709 13:15:18.901856 22913 raft_consensus.cc:2889] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:18.903998 31502 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 13:15:18.917111 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:18.927127 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:18.931900 22712 auto_rebalancer.cc:629] Failed to schedule move for tablet 4c3f341bf2b9433090bb7beb432566e1: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:18.934131 23464 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Refusing update from remote peer 776baf137f8745dd866379ca15ab8cee: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 13:15:18.941973 31502 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
W20260709 13:15:18.951251 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:19.047271 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 62276.8us
W20260709 13:15:19.052305 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 66036us
I20260709 13:15:19.040541 31396 raft_consensus.cc:493] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.058516 31396 raft_consensus.cc:515] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:19.084725 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 97884.5us
I20260709 13:15:19.111121 31396 leader_election.cc:290] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:19.117430 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:19.134303 22655 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 20 to 30. New cstate: current_term: 30 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:15:19.146078 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 48342.7us
W20260709 13:15:19.159322 22712 auto_rebalancer.cc:629] Failed to schedule move for tablet bac9a57b26684fe6a49a43a8a61151e6: Illegal state: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:15:19.165108 22712 auto_rebalancer.cc:629] Failed to schedule move for tablet bb34c8c8aec0452ab45aaa676b75558a: Illegal state: Replica 776baf137f8745dd866379ca15ab8cee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.166702 31396 raft_consensus.cc:493] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.167531 31396 raft_consensus.cc:515] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:19.170140 31396 leader_election.cc:290] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:19.179198 31396 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.180184 31396 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:19.183106 31396 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:19.187868 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:19.198588 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:19.217276 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 124240us
W20260709 13:15:19.221822 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:19.226702 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:19.227083 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 126330us
W20260709 13:15:19.233603 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:19.234495 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:19.235738 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.239017 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c3f341bf2b9433090bb7beb432566e1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:19.254714 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e19799c88cba4c9194a9308860a76da7" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:19.259268 22664 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 250) took 18222 ms (client timeout 15000 ms). Trace:
W20260709 13:15:19.262354 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:19.263221 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:19.268827 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:19.262286 22664 rpcz_store.cc:269] 0709 13:15:01.047782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.010179 (+11962397us) service_pool.cc:224] Handling call
0709 13:15:19.257227 (+6247048us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:18.858695 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:18.859375 (+   680us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:18.859415 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:15:18.859438 (+    23us) tablet.cc:662] Decoding operations
0709 13:15:18.861027 (+  1589us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:18.861096 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:18.861132 (+    36us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:15:18.886873 (+ 25741us) tablet.cc:701] Row locks acquired
0709 13:15:18.886903 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:15:18.887095 (+   192us) write_op.cc:270] Start()
0709 13:15:18.887244 (+   149us) write_op.cc:276] Timestamp: P: 1783602918887043 usec, L: 0
0709 13:15:18.887270 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:15:18.905284 (+ 18014us) log.cc:844] Serialized 5884 byte log entry
0709 13:15:18.908280 (+  2996us) op_driver.cc:464] REPLICATION: finished
0709 13:15:18.908775 (+   495us) write_op.cc:301] APPLY: starting
0709 13:15:18.909315 (+   540us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:18.948713 (+ 39398us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:18.949120 (+   407us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:18.969631 (+ 20511us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:18.969933 (+   302us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 13:15:18.969979 (+    46us) write_op.cc:312] APPLY: finished
0709 13:15:18.970447 (+   468us) log.cc:844] Serialized 139 byte log entry
0709 13:15:18.970632 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:18.970832 (+   200us) write_op.cc:454] Released schema lock
0709 13:15:18.970900 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:18.991464 (+ 20564us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:18.991994 (+   530us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":42901,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":284,"apply.run_cpu_time_us":21951,"apply.run_wall_time_us":77927,"cfile_cache_hit":60,"cfile_cache_hit_bytes":62730,"num_ops":15,"prepare.queue_time_us":1240,"prepare.run_cpu_time_us":5813,"prepare.run_wall_time_us":50291,"replication_time_us":20810}]]}
I20260709 13:15:19.267577 22656 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.294896 22656 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 20 to 22, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 22 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.297313 22656 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 19 to 20, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 20 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:19.329530 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 61823.7us
I20260709 13:15:19.329679 22656 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.327732 22730 leader_election.cc:304] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.334298 22656 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 14 to 17, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 17 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.341709 31425 raft_consensus.cc:2749] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:19.347162 22730 leader_election.cc:304] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.351918 31396 raft_consensus.cc:2749] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:19.351660 22656 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.360074 22730 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.371563 31425 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:19.376917 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:19.386821 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.413959 31525 raft_consensus.cc:493] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.414691 31525 raft_consensus.cc:515] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:19.418589 31525 leader_election.cc:290] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:19.420009 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:19.420746 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bac9a57b26684fe6a49a43a8a61151e6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:19.421676 22906 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:15:19.423700 22730 leader_election.cc:304] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.440392 31530 raft_consensus.cc:2749] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:19.475817 22837 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:15:19.476864 22879 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663): Couldn't send request to peer 74c0a02901bc45c891e42d0d1bbbf6a2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a7e34bc1d8d404aae87baaaa171e0b1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:19.483348 31506 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:19.484149 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:19.509110 22659 catalog_manager.cc:5697] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.511389 22659 catalog_manager.cc:5697] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.513585 22659 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.517863 22659 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 23 to 26, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 26 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
W20260709 13:15:19.519625 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:19.520658 22656 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51522 (request call id 271) took 11962 ms (client timeout 15000 ms). Trace:
I20260709 13:15:19.519867 31494 raft_consensus.cc:2955] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:15:19.521763 31530 raft_consensus.cc:493] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.521613 22837 raft_consensus.cc:2955] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } } }
I20260709 13:15:19.531433 31530 raft_consensus.cc:515] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:19.537460 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:19.538259 31530 leader_election.cc:290] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:19.538483 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf57270bf9649da86a032e1c9ce4bd6" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:19.540436 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.541678 22732 leader_election.cc:304] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.529166 22659 catalog_manager.cc:5697] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 19 to 20, leader changed from 6545453e74344bb2957dd3729c5b3870 (127.21.33.3) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.542986 31530 raft_consensus.cc:2749] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:19.529920 22656 rpcz_store.cc:269] 0709 13:15:07.587687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.154679 (+8566992us) service_pool.cc:224] Handling call
0709 13:15:19.520570 (+3365891us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:19.372280 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:19.372939 (+   659us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:19.372981 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:15:19.373003 (+    22us) tablet.cc:662] Decoding operations
0709 13:15:19.374028 (+  1025us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:19.374105 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:19.374136 (+    31us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:15:19.374678 (+   542us) tablet.cc:701] Row locks acquired
0709 13:15:19.374697 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:19.374859 (+   162us) write_op.cc:270] Start()
0709 13:15:19.374969 (+   110us) write_op.cc:276] Timestamp: P: 1783602919374808 usec, L: 0
0709 13:15:19.374989 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:15:19.375655 (+   666us) log.cc:844] Serialized 2446 byte log entry
0709 13:15:19.377910 (+  2255us) op_driver.cc:464] REPLICATION: finished
0709 13:15:19.379157 (+  1247us) write_op.cc:301] APPLY: starting
0709 13:15:19.379238 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:19.387145 (+  7907us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:19.387246 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:19.392560 (+  5314us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:19.392836 (+   276us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 13:15:19.392889 (+    53us) write_op.cc:312] APPLY: finished
0709 13:15:19.393198 (+   309us) log.cc:844] Serialized 65 byte log entry
0709 13:15:19.393389 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.393543 (+   154us) write_op.cc:454] Released schema lock
0709 13:15:19.393600 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:19.397226 (+  3626us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.397254 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5400,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":286,"apply.run_cpu_time_us":9410,"apply.run_wall_time_us":18050,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25092,"num_ops":6,"prepare.queue_time_us":3389,"prepare.run_cpu_time_us":3673,"prepare.run_wall_time_us":6196,"replication_time_us":2791}]]}
I20260709 13:15:19.550822 22659 catalog_manager.cc:5697] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 18 to 21, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.557340 22659 catalog_manager.cc:5697] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 17 to 19, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.564778 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:19.567915 22659 catalog_manager.cc:5697] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 22 to 23, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 23 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.569931 22659 catalog_manager.cc:5697] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 21, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.571870 22659 catalog_manager.cc:5697] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.576472 21636 tablet_replica.cc:333] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.582067 21636 raft_consensus.cc:2243] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.585650 21636 raft_consensus.cc:2272] T 4f79e81f486b4e7dbf2d845891b6b9c0 P acfda9b350734def9b6c621f66a5f247 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.574169 22659 catalog_manager.cc:5697] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 21 to 22, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 22 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.594784 22659 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 25 to 26, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 26 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.596628 22659 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 24 to 25, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 25 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.598212 22659 catalog_manager.cc:5697] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 19 to 20, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 20 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.600021 22659 catalog_manager.cc:5697] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.601935 21636 tablet_replica.cc:333] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.601619 22659 catalog_manager.cc:5697] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 22, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 22 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.610411 22659 catalog_manager.cc:5697] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 23 to 25, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 25 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.612836 22659 catalog_manager.cc:5697] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.615300 22659 catalog_manager.cc:5697] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:19.626502 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:19.617233 22659 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:19.627826 22659 catalog_manager.cc:5697] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 22, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 22 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.614434 21636 raft_consensus.cc:2243] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.630365 22659 catalog_manager.cc:5697] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 21 to 22, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 22 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.634128 22659 catalog_manager.cc:5697] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.635318 21636 pending_rounds.cc:70] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:19.636937 21636 pending_rounds.cc:77] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305637484095733760 op_type: NO_OP noop_request { }
I20260709 13:15:19.639853 21636 raft_consensus.cc:2889] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:19.635726 22659 catalog_manager.cc:5697] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 26 to 29, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 29 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.643642 21636 raft_consensus.cc:2272] T bb34c8c8aec0452ab45aaa676b75558a P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.648452 22659 catalog_manager.cc:5697] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 20 to 23, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 23 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:19.650460 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.654768 21636 tablet_replica.cc:333] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.674254 21636 raft_consensus.cc:2243] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.678869 21636 pending_rounds.cc:70] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:19.680800 21636 pending_rounds.cc:77] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 22 index: 12 } timestamp: 7305637480480268288 op_type: NO_OP noop_request { }
I20260709 13:15:19.681985 21636 raft_consensus.cc:2889] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:15:19.683090 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:19.683557 21636 raft_consensus.cc:2272] T bac9a57b26684fe6a49a43a8a61151e6 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.692972 21636 tablet_replica.cc:333] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
W20260709 13:15:19.701004 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.710985 21636 raft_consensus.cc:2243] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:19.713431 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.724493 31530 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.725593 21636 pending_rounds.cc:70] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:19.725936 21636 pending_rounds.cc:77] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 23 index: 13 } timestamp: 7305637442914312192 op_type: NO_OP noop_request { }
I20260709 13:15:19.726341 21636 raft_consensus.cc:2889] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:19.725248 31530 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:19.726845 21636 raft_consensus.cc:2272] T 0bf57270bf9649da86a032e1c9ce4bd6 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.730965 31530 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:19.731869 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:19.734891 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:19.735195 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:15:19.762921 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:19.762987 21636 tablet_replica.cc:333] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.765787 21636 raft_consensus.cc:2243] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.767364 21636 raft_consensus.cc:2272] T a663ecdecbd14089a4074b2581e94517 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.775923 21636 tablet_replica.cc:333] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.785946 22730 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:19.807898 31530 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:19.767953 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:19.809046 21636 raft_consensus.cc:2243] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.814241 21636 pending_rounds.cc:70] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:19.825675 21636 pending_rounds.cc:77] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305637463059886080 op_type: NO_OP noop_request { }
I20260709 13:15:19.832264 21636 raft_consensus.cc:2889] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:19.833557 21636 raft_consensus.cc:2272] T a18b5b24775b48b589edb90581a5cad5 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.837319 21636 tablet_replica.cc:333] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.863603 21636 raft_consensus.cc:2243] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:19.864914 21636 raft_consensus.cc:2272] T bf5430bef8774025a919427a6e425b65 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.874617 31530 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.878489 31530 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:19.895071 21636 tablet_replica.cc:333] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.896029 21636 raft_consensus.cc:2243] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.896648 21636 raft_consensus.cc:2272] T 961d7db1909a4e40a4b410a32dad4d64 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.905156 31525 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.905858 31525 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:19.908794 31525 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:19.910091 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:19.927224 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:19.943686 31530 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:19.945693 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:19.947892 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:19.949026 22732 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:19.951571 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eac8195b1d874d3aa822a4f96dee4738" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:19.949069 31530 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.956733 31530 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:19.959861 31530 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:19.960501 21636 tablet_replica.cc:333] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:19.960554 31530 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:19.964167 22730 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.964988 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:19.965200 31530 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:19.921837 31525 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.969441 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:19.965953 31525 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:19.973934 21636 raft_consensus.cc:2243] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:19.978549 22730 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:19.986127 31534 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.986940 31534 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:19.981678 31530 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:19.994048 21636 pending_rounds.cc:70] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:19.994639 21636 pending_rounds.cc:77] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 17 index: 10 } timestamp: 7305637464128995328 op_type: NO_OP noop_request { }
I20260709 13:15:19.995186 21636 raft_consensus.cc:2889] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:19.995630 21636 raft_consensus.cc:2272] T 3e701ce8413c4f11afc5ca006f97eb50 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:19.998910 31525 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:20.001586 21636 tablet_replica.cc:333] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.003334 21636 raft_consensus.cc:2243] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.004591 21636 raft_consensus.cc:2272] T 023c6d83f2864eedb2775b615045955e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.006117 31534 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:20.006809 31534 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.007340 31534 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:20.012701 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:20.014361 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
W20260709 13:15:20.030071 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46871.3us
I20260709 13:15:20.046397 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38001bb9b85f4814aff1beccb3ae9acb" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:20.054388 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:20.058640 22732 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:20.070540 22659 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51552 (request call id 248) took 6324 ms. Trace:
I20260709 13:15:20.071092 22659 rpcz_store.cc:276] 0709 13:15:13.746761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.228313 (+3481552us) service_pool.cc:224] Handling call
0709 13:15:20.070458 (+2842145us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:19.706841 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:19.707449 (+   608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:19.707491 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:15:19.707512 (+    21us) tablet.cc:662] Decoding operations
0709 13:15:19.709489 (+  1977us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:19.709599 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:19.709638 (+    39us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:15:19.713793 (+  4155us) tablet.cc:701] Row locks acquired
0709 13:15:19.713815 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:15:19.713987 (+   172us) write_op.cc:270] Start()
0709 13:15:19.714131 (+   144us) write_op.cc:276] Timestamp: P: 1783602919713935 usec, L: 0
0709 13:15:19.714158 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:15:19.731911 (+ 17753us) log.cc:844] Serialized 9704 byte log entry
0709 13:15:19.735243 (+  3332us) op_driver.cc:464] REPLICATION: finished
0709 13:15:19.736174 (+   931us) write_op.cc:301] APPLY: starting
0709 13:15:19.736281 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:19.738497 (+  2216us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5400295ed4. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a1a5a571 00007f68a1aece28 00007f68a5527581 00007f68a5526d58 00007f68a552b9f1 00007f68a552b9a9 00007f68a552b939 00007f68a552b901 00007f68a552ab2d 000000000038f354 000000000038f199
0709 13:15:19.795513 (+ 57016us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:19.795544 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:19.897758 (+102214us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:19.898077 (+   319us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:15:19.898118 (+    41us) write_op.cc:312] APPLY: finished
0709 13:15:19.898499 (+   381us) log.cc:844] Serialized 220 byte log entry
0709 13:15:19.898677 (+   178us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.898898 (+   221us) write_op.cc:454] Released schema lock
0709 13:15:19.898956 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:19.899343 (+   387us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.899367 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15963,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":659,"apply.run_cpu_time_us":32336,"apply.run_wall_time_us":163154,"cfile_cache_hit":100,"cfile_cache_hit_bytes":104550,"num_ops":25,"prepare.queue_time_us":137,"prepare.run_cpu_time_us":10372,"prepare.run_wall_time_us":31861,"replication_time_us":20938,"spinlock_wait_cycles":38754048}]]}
I20260709 13:15:20.061280 31525 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:20.078970 31535 ts_tablet_manager.cc:933] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: Initiating tablet copy from peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:20.083966 21636 tablet_replica.cc:333] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.085256 21636 raft_consensus.cc:2243] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.085287 22732 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:20.088515 31537 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:20.088670 21636 raft_consensus.cc:2272] T f55844ac3c9a4ceaa35b3d5bdadad40e P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.097236 21636 tablet_replica.cc:333] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.100298 21636 raft_consensus.cc:2243] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.102468 31534 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:20.112658 21636 pending_rounds.cc:70] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.115232 21636 pending_rounds.cc:77] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 23 index: 9 } timestamp: 7305637368647888896 op_type: NO_OP noop_request { }
I20260709 13:15:20.116099 21636 raft_consensus.cc:2889] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.117556 21636 raft_consensus.cc:2272] T 25ae06dc4f8940f1a419a39f791d6e18 P acfda9b350734def9b6c621f66a5f247 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.121093 21636 tablet_replica.cc:333] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
W20260709 13:15:20.130795 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.133812 31534 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.134981 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:20.134518 31534 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:20.138420 31534 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:20.140645 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:20.142722 31535 tablet_copy_client.cc:323] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.33.3:35233
I20260709 13:15:20.143467 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da64727f9cd84f6bb87d91e54eeac2d5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:20.147219 22732 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:20.149616 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:20.150933 31525 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:20.151757 22657 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 23 to 26, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to acfda9b350734def9b6c621f66a5f247 (127.21.33.1). New cstate: current_term: 26 leader_uuid: "acfda9b350734def9b6c621f66a5f247" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:20.164149 22657 catalog_manager.cc:5697] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 21 to 24, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 24 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:20.167505 22732 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:20.168624 21636 raft_consensus.cc:2243] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.197395 21636 raft_consensus.cc:2272] T adf76c1fde374ccbbb4a0edda126b851 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.169700 22925 tablet_copy_service.cc:140] P 6545453e74344bb2957dd3729c5b3870: Received BeginTabletCopySession request for tablet 6a7e34bc1d8d404aae87baaaa171e0b1 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 ({username='slave'} at 127.0.0.1:51142)
W20260709 13:15:20.183435 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:20.201906 22925 tablet_copy_service.cc:161] P 6545453e74344bb2957dd3729c5b3870: Beginning new tablet copy session on tablet 6a7e34bc1d8d404aae87baaaa171e0b1 from peer 74c0a02901bc45c891e42d0d1bbbf6a2 at {username='slave'} at 127.0.0.1:51142: session id = 74c0a02901bc45c891e42d0d1bbbf6a2-6a7e34bc1d8d404aae87baaaa171e0b1
W20260709 13:15:20.205109 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:20.208216 31525 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:20.198982 22657 catalog_manager.cc:5697] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 22 to 24, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 24 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:20.212637 22657 catalog_manager.cc:5697] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 23 to 25, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 776baf137f8745dd866379ca15ab8cee (127.21.33.2). New cstate: current_term: 25 leader_uuid: "776baf137f8745dd866379ca15ab8cee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:20.210999 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:20.201361 21636 tablet_replica.cc:333] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.226080 21636 raft_consensus.cc:2243] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.226941 21636 raft_consensus.cc:2272] T 8670cf017371429ea31529b6f247493e P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.230767 21636 tablet_replica.cc:333] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.232118 21636 raft_consensus.cc:2243] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.232852 21636 raft_consensus.cc:2272] T 11f984a583c84fbe9ef7b3c817b06c44 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.249480 22925 tablet_copy_source_session.cc:215] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:20.256067 31535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7e34bc1d8d404aae87baaaa171e0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:20.265661 21636 tablet_replica.cc:333] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.267060 21636 raft_consensus.cc:2243] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.267899 21636 raft_consensus.cc:2272] T 9951bee4cd6b4966bef18c1e0d25eaca P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.284426 31525 raft_consensus.cc:493] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.285123 31525 raft_consensus.cc:515] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:20.290921 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:20.291931 31525 leader_election.cc:290] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:20.294986 21636 tablet_replica.cc:333] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.296058 21636 raft_consensus.cc:2243] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:20.299062 21636 raft_consensus.cc:2272] T a9fc1639a8fa47ba91cd2fc158066fb2 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.305905 21636 tablet_replica.cc:333] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.307395 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76de9d7042974967a385da8f613f4cb5" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:20.308589 21636 raft_consensus.cc:2243] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:20.309597 22730 leader_election.cc:304] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:20.309937 21636 raft_consensus.cc:2272] T a17d5efe4abb490b8dc12b9e45d754ca P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.311275 31525 raft_consensus.cc:2749] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:20.313851 21636 tablet_replica.cc:333] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.317296 21636 raft_consensus.cc:2243] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.318358 21636 raft_consensus.cc:2272] T 0818ffaecde64fc0af038b3c024b6008 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.322861 21636 tablet_replica.cc:333] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.322185 22659 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: config changed from index -1 to 13, NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) added. New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.324025 21636 raft_consensus.cc:2243] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.324637 21636 raft_consensus.cc:2272] T e19799c88cba4c9194a9308860a76da7 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.326685 22659 catalog_manager.cc:5697] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 24 to 26, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 26 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.328092 21636 tablet_replica.cc:333] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.329257 21636 raft_consensus.cc:2243] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.330591 21636 pending_rounds.cc:70] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Trying to abort 2 pending ops.
I20260709 13:15:20.330874 21636 pending_rounds.cc:77] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305637417177391104 op_type: NO_OP noop_request { }
I20260709 13:15:20.331264 21636 raft_consensus.cc:2889] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.331607 21636 pending_rounds.cc:77] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 17 index: 11 } timestamp: 7305637367023857664 op_type: NO_OP noop_request { }
I20260709 13:15:20.331956 21636 raft_consensus.cc:2889] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.333560 21636 raft_consensus.cc:2272] T af39123de9434cc0b451f4b03d305310 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.337595 21636 tablet_replica.cc:333] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.338657 21636 raft_consensus.cc:2243] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:20.339985 21636 raft_consensus.cc:2272] T 11d0cdd09217481cbfc11c1d2d14a5a0 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.341777 31535 tablet_copy_client.cc:806] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:20.349800 31535 tablet_copy_client.cc:670] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:20.355872 31535 tablet_copy_client.cc:538] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:15:20.357381 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.343305 21636 tablet_replica.cc:333] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.364025 21636 raft_consensus.cc:2243] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.364729 21636 raft_consensus.cc:2272] T b00e5613b4864bd6a0873aee1004ed22 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.376986 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.382157 31525 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.382886 31525 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:20.386528 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:20.387159 21636 tablet_replica.cc:333] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.388352 21636 raft_consensus.cc:2243] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.389176 21636 raft_consensus.cc:2272] T 2a537de05aee48178205ec0e7e0d436e P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.393718 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.395237 31525 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:20.394462 31535 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap starting.
I20260709 13:15:20.397575 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "acfda9b350734def9b6c621f66a5f247" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:20.407688 22732 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acfda9b350734def9b6c621f66a5f247; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:20.411280 31525 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:20.421761 21636 tablet_replica.cc:333] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.422943 21636 raft_consensus.cc:2243] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.423550 21636 raft_consensus.cc:2272] T 76de9d7042974967a385da8f613f4cb5 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.438731 21636 tablet_replica.cc:333] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.439823 21636 raft_consensus.cc:2243] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.440577 21636 pending_rounds.cc:70] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.440861 21636 pending_rounds.cc:77] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 17 index: 12 } timestamp: 7305637403492323328 op_type: NO_OP noop_request { }
I20260709 13:15:20.441259 21636 raft_consensus.cc:2889] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:15:20.443991 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:20.443912 22657 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:51536 (request call id 251) took 3641 ms. Trace:
W20260709 13:15:20.453897 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:20.445812 22657 rpcz_store.cc:276] 0709 13:15:16.802996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.373104 (+570108us) service_pool.cc:224] Handling call
0709 13:15:20.443825 (+3070721us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:20.232785 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:20.233346 (+   561us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:20.233416 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:15:20.233434 (+    18us) tablet.cc:662] Decoding operations
0709 13:15:20.234083 (+   649us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:20.234148 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:20.234192 (+    44us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:15:20.234678 (+   486us) tablet.cc:701] Row locks acquired
0709 13:15:20.234693 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:15:20.234864 (+   171us) write_op.cc:270] Start()
0709 13:15:20.234996 (+   132us) write_op.cc:276] Timestamp: P: 1783602920234813 usec, L: 0
0709 13:15:20.235017 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:15:20.235746 (+   729us) log.cc:844] Serialized 1682 byte log entry
0709 13:15:20.258995 (+ 23249us) op_driver.cc:464] REPLICATION: finished
0709 13:15:20.259351 (+   356us) write_op.cc:301] APPLY: starting
0709 13:15:20.259489 (+   138us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:20.261440 (+  1951us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:20.261552 (+   112us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:20.268066 (+  6514us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:20.268255 (+   189us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:15:20.268296 (+    41us) write_op.cc:312] APPLY: finished
0709 13:15:20.268543 (+   247us) log.cc:844] Serialized 49 byte log entry
0709 13:15:20.271807 (+  3264us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:20.271915 (+   108us) write_op.cc:454] Released schema lock
0709 13:15:20.271959 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:20.289042 (+ 17083us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:20.289075 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":83734,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":113,"apply.run_cpu_time_us":5884,"apply.run_wall_time_us":29643,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16728,"num_ops":4,"prepare.queue_time_us":3271,"prepare.run_cpu_time_us":3269,"prepare.run_wall_time_us":3268,"replication_time_us":23801}]]}
I20260709 13:15:20.441671 21636 raft_consensus.cc:2272] T 0ad66b5c9842437d9320507b92b78fd5 P acfda9b350734def9b6c621f66a5f247 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.466815 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:20.469698 21636 tablet_replica.cc:333] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.490451 21636 raft_consensus.cc:2243] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.491200 21636 raft_consensus.cc:2272] T bd464cd43c544f7abc90c38571db4d1c P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.494650 21636 tablet_replica.cc:333] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.495707 21636 raft_consensus.cc:2243] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.496373 21636 raft_consensus.cc:2272] T 072f69118119423dac7eba3f66879eee P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.513543 21636 tablet_replica.cc:333] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.514635 21636 raft_consensus.cc:2243] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.515599 21636 raft_consensus.cc:2272] T 1c406ce5efa449d4b877f6747d1f14ab P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.519158 21636 tablet_replica.cc:333] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.520262 21636 raft_consensus.cc:2243] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.521245 21636 raft_consensus.cc:2272] T f2aaf3c1715e4d29954bd6ee38914a7a P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.540581 21636 tablet_replica.cc:333] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.541837 21636 raft_consensus.cc:2243] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:20.548317 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.542789 21636 pending_rounds.cc:70] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.561888 21636 pending_rounds.cc:77] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305637438492020736 op_type: NO_OP noop_request { }
I20260709 13:15:20.562335 21636 raft_consensus.cc:2889] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.562736 21636 raft_consensus.cc:2272] T eac8195b1d874d3aa822a4f96dee4738 P acfda9b350734def9b6c621f66a5f247 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.568886 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:20.570840 21636 tablet_replica.cc:333] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.572124 21636 raft_consensus.cc:2243] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.573064 21636 pending_rounds.cc:70] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.602623 21636 pending_rounds.cc:77] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 26 index: 16 } timestamp: 7305637516848201728 op_type: NO_OP noop_request { }
I20260709 13:15:20.603582 21636 raft_consensus.cc:2889] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.604035 21636 raft_consensus.cc:2272] T 38001bb9b85f4814aff1beccb3ae9acb P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.607815 21636 tablet_replica.cc:333] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.608855 21636 raft_consensus.cc:2243] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.609761 21636 raft_consensus.cc:2272] T 2e04261a736f42d2a1fd4e24bc787276 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.613488 21636 tablet_replica.cc:333] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.614528 21636 raft_consensus.cc:2243] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.615332 21636 raft_consensus.cc:2272] T 485fd0c9273e482692ddc7589cb09f02 P acfda9b350734def9b6c621f66a5f247 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.618742 21636 tablet_replica.cc:333] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.619614 21636 raft_consensus.cc:2243] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.620263 21636 raft_consensus.cc:2272] T 0b88bd2b1b1d4ced918486ea751fbe25 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.629554 31535 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:20.631845 21636 tablet_replica.cc:333] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
W20260709 13:15:20.637576 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.644363 21636 raft_consensus.cc:2243] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.645196 21636 pending_rounds.cc:70] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.645479 21636 pending_rounds.cc:77] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305637447607709696 op_type: NO_OP noop_request { }
I20260709 13:15:20.646019 21636 raft_consensus.cc:2889] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.646524 21636 raft_consensus.cc:2272] T 6d283692c3bb4f979788fa989a74bd80 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.648782 31535 tablet_bootstrap.cc:492] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: Bootstrap complete.
I20260709 13:15:20.649816 31535 ts_tablet_manager.cc:1403] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent bootstrapping tablet: real 0.260s	user 0.053s	sys 0.011s
I20260709 13:15:20.653699 31535 raft_consensus.cc:359] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
W20260709 13:15:20.654886 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:20.657680 21636 tablet_replica.cc:333] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.659152 21636 raft_consensus.cc:2243] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.659919 21636 raft_consensus.cc:2272] T 4c3f341bf2b9433090bb7beb432566e1 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.654790 31535 raft_consensus.cc:740] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Initialized, Role: LEARNER
I20260709 13:15:20.666517 31535 consensus_queue.cc:260] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: NON_VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: true } }
I20260709 13:15:20.667721 21636 tablet_replica.cc:333] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.669320 21636 raft_consensus.cc:2243] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.670223 21636 raft_consensus.cc:2272] T 6a7e34bc1d8d404aae87baaaa171e0b1 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.687552 21636 tablet_replica.cc:333] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.688634 21636 raft_consensus.cc:2243] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.689404 21636 raft_consensus.cc:2272] T 659f33da809544df8f5fc087bbe7cc17 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.692296 31535 ts_tablet_manager.cc:1434] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.002s
I20260709 13:15:20.693310 21636 tablet_replica.cc:333] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.694446 21636 raft_consensus.cc:2243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:20.696178 21636 raft_consensus.cc:2272] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P acfda9b350734def9b6c621f66a5f247 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.699810 22925 tablet_copy_service.cc:342] P 6545453e74344bb2957dd3729c5b3870: Request end of tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-6a7e34bc1d8d404aae87baaaa171e0b1 received from {username='slave'} at 127.0.0.1:51142
I20260709 13:15:20.700835 22925 tablet_copy_service.cc:434] P 6545453e74344bb2957dd3729c5b3870: ending tablet copy session 74c0a02901bc45c891e42d0d1bbbf6a2-6a7e34bc1d8d404aae87baaaa171e0b1 on tablet 6a7e34bc1d8d404aae87baaaa171e0b1 with peer 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:20.704931 21636 tablet_replica.cc:333] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.705919 21636 raft_consensus.cc:2243] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.706544 21636 raft_consensus.cc:2272] T 2f666ea4542d4951919f36de0b43b972 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.709857 21636 tablet_replica.cc:333] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.710918 21636 raft_consensus.cc:2243] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.711947 21636 pending_rounds.cc:70] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.712205 21636 pending_rounds.cc:77] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305637445663371264 op_type: NO_OP noop_request { }
I20260709 13:15:20.712639 21636 raft_consensus.cc:2889] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.713006 21636 raft_consensus.cc:2272] T 7ede022de6a346e284376d4212e1f6b7 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.729300 21636 tablet_replica.cc:333] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.730370 21636 raft_consensus.cc:2243] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.730935 21636 raft_consensus.cc:2272] T da64727f9cd84f6bb87d91e54eeac2d5 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.744485 21636 tablet_replica.cc:333] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.745451 21636 raft_consensus.cc:2243] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.746006 21636 raft_consensus.cc:2272] T c52c0b7017594a0f8651f1fe9e9f51a2 P acfda9b350734def9b6c621f66a5f247 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.748229 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:20.749522 21636 tablet_replica.cc:333] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.750420 21636 raft_consensus.cc:2243] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.751205 21636 raft_consensus.cc:2272] T a6da5381f5554dc69fb15991d351fb20 P acfda9b350734def9b6c621f66a5f247 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.761461 21636 tablet_replica.cc:333] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.762607 21636 raft_consensus.cc:2243] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.763199 21636 pending_rounds.cc:70] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.763437 21636 pending_rounds.cc:77] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305637436849528832 op_type: NO_OP noop_request { }
I20260709 13:15:20.763823 21636 raft_consensus.cc:2889] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.764189 21636 raft_consensus.cc:2272] T a71fa884e652460c841521a01aa8b5a0 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.780637 21636 tablet_replica.cc:333] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.781644 21636 raft_consensus.cc:2243] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.782306 21636 raft_consensus.cc:2272] T ba1db22a7176432c86e7264f87984ba3 P acfda9b350734def9b6c621f66a5f247 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.791420 21636 tablet_replica.cc:333] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.792308 21636 raft_consensus.cc:2243] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:20.793596 21636 raft_consensus.cc:2272] T e36250f751be4e7480619d5807252ac0 P acfda9b350734def9b6c621f66a5f247 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.796643 21636 tablet_replica.cc:333] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.797394 21636 raft_consensus.cc:2243] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.797968 21636 pending_rounds.cc:70] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Trying to abort 1 pending ops.
I20260709 13:15:20.798199 21636 pending_rounds.cc:77] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247: Aborting op as it isn't in flight: id { term: 22 index: 11 } timestamp: 7305637439603392512 op_type: NO_OP noop_request { }
I20260709 13:15:20.798554 21636 raft_consensus.cc:2889] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:20.798885 21636 raft_consensus.cc:2272] T cae1e4795b8046c9b632ce5907911c39 P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.803198 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:20.831473 21636 tablet_replica.cc:333] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.832463 21636 raft_consensus.cc:2243] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.833086 21636 raft_consensus.cc:2272] T f12b50ff391746348739bd5f7a08a44a P acfda9b350734def9b6c621f66a5f247 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:20.836319 21636 tablet_replica.cc:333] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247: stopping tablet replica
I20260709 13:15:20.837222 21636 raft_consensus.cc:2243] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:20.849694 21636 raft_consensus.cc:2272] T 4572187533dd409bad9adf0de57c6940 P acfda9b350734def9b6c621f66a5f247 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:20.862363 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:20.879212 22650 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 72056.7us
W20260709 13:15:20.891634 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
W20260709 13:15:20.947297 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:20.975019 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 61855.9us
I20260709 13:15:21.010749 23462 raft_consensus.cc:1217] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.12->[21.13-21.13]   Dedup: 21.13->[]
W20260709 13:15:21.018666 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:21.045899 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 46452.5us
I20260709 13:15:21.060122 31543 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.062294 31542 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.063068 31542 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:21.061100 31543 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:21.078539 31542 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:21.079509 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:21.083035 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:21.094394 31543 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:21.119252 31539 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.120052 31539 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.125383 31539 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:21.135673 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:21.137228 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:21.138139 23462 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 28.
I20260709 13:15:21.140167 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:21.140908 22913 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 28.
I20260709 13:15:21.148660 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.149585 31542 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:15:21.149976 31542 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.150364 31542 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:15:21.153070 22880 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:21.154307 22806 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:21.154838 22806 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.156458 31539 raft_consensus.cc:2749] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:21.160096 31542 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:21.162051 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.176091 31543 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:15:21.176648 31543 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.177124 31543 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:15:21.179301 31506 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.179930 31506 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.180291 31542 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:21.182332 31506 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:21.185065 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:21.187129 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:21.188017 22837 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 25.
I20260709 13:15:21.217593 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:21.224128 31506 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:21.224637 31506 raft_consensus.cc:493] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.225014 31506 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:21.275348 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
W20260709 13:15:21.277707 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:21.282143 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 64753.4us
W20260709 13:15:21.284577 22880 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:21.291585 31543 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:21.286947 31506 raft_consensus.cc:515] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.294669 31506 leader_election.cc:290] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:21.296079 22906 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in current term 29: Already voted for candidate 6545453e74344bb2957dd3729c5b3870 in this term.
I20260709 13:15:21.304445 31543 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:21.305475 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5430bef8774025a919427a6e425b65" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:21.306281 22832 raft_consensus.cc:3060] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:21.319839 22832 raft_consensus.cc:2468] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:15:21.332870 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.334549 31550 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:15:21.384279 22879 leader_election.cc:304] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:21.387290 31543 raft_consensus.cc:2804] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:21.388027 31543 raft_consensus.cc:697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 26 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:21.389072 31543 consensus_queue.cc:237] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.396425 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:21.397270 23462 raft_consensus.cc:2393] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6545453e74344bb2957dd3729c5b3870 in current term 29: Already voted for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in this term.
W20260709 13:15:21.407495 22880 leader_election.cc:336] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:21.418555 22880 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:21.427125 22657 catalog_manager.cc:5697] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 25 to 26, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 26 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:21.446393 22879 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.451920 31543 raft_consensus.cc:2749] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 13:15:21.456125 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:21.456851 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:21.469892 22880 connection.cc:769] client connection to 127.21.33.1:45095 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:15:21.480511 22806 connection.cc:573] client connection to 127.21.33.1:45095 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:15:21.498500 31522 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.502463 31522 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.509927 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:21.509723 31522 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:21.521581 31522 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.522490 31522 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.526777 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:21.535869 31522 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:21.546837 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:21.562954 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:21.568910 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:21.575169 22806 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:21.575739 22806 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.578573 31522 raft_consensus.cc:2749] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:15:21.579946 22806 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:21.580390 22806 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.592103 31522 raft_consensus.cc:2749] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:21.612709 31554 raft_consensus.cc:1064] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: attempting to promote NON_VOTER 74c0a02901bc45c891e42d0d1bbbf6a2 to VOTER
I20260709 13:15:21.615839 31554 consensus_queue.cc:237] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:21.648999 31543 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.649645 31543 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.651892 31543 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:21.665014 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:21.665943 22832 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
W20260709 13:15:21.667179 22880 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:21.668407 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.669548 31543 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:21.670029 31543 raft_consensus.cc:493] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.670408 31543 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:21.690711 23466 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEARNER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:15:21.692557 31513 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:21.717193 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:21.723045 22832 raft_consensus.cc:1275] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:15:21.724072 31522 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.728339 31522 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.729974 31554 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.730985 31554 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.731132 31513 consensus_queue.cc:1048] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:21.733238 31554 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:21.741153 22806 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:21.745549 22880 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:21.751139 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:21.755419 31522 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:21.754715 22832 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:15:21.756564 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:21.759541 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.759537 22807 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:21.762856 31539 raft_consensus.cc:2749] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:21.764225 31554 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:21.764722 31554 raft_consensus.cc:493] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.765097 31554 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:21.779995 31551 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.776290 31543 raft_consensus.cc:515] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.780759 31551 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.799226 31551 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:21.800179 22833 raft_consensus.cc:1275] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:15:21.801081 31543 leader_election.cc:290] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:21.802354 31554 raft_consensus.cc:515] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.804737 31554 leader_election.cc:290] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:21.805321 31513 consensus_queue.cc:1048] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:21.810444 31564 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.811250 31564 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.813113 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:21.813958 22833 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:15:21.817991 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d0cdd09217481cbfc11c1d2d14a5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:21.818693 22833 raft_consensus.cc:3060] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:15:21.820683 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:21.822640 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17d5efe4abb490b8dc12b9e45d754ca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:21.822899 31564 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:21.823352 22832 raft_consensus.cc:3060] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:21.824826 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:21.826164 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:21.862736 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47039.9us
I20260709 13:15:21.830431 22807 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:21.845754 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:21.866158 31539 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:15:21.856108 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:21.869923 31513 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:21.870388 31513 raft_consensus.cc:493] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.872014 31513 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:15:21.873358 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:21.878582 31543 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.879458 31543 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:21.882200 31543 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:21.881245 31554 raft_consensus.cc:2955] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:15:21.884646 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:21.885179 22880 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:21.906111 31551 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:21.906893 31551 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.907951 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:21.911819 31551 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:21.890386 22833 raft_consensus.cc:2468] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
W20260709 13:15:21.925974 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:21.930392 22879 leader_election.cc:304] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: 
I20260709 13:15:21.933254 22832 raft_consensus.cc:2468] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:15:21.939476 31551 raft_consensus.cc:2804] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:21.940109 31551 raft_consensus.cc:697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:21.941030 31551 consensus_queue.cc:237] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:21.944329 22832 raft_consensus.cc:2955] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
W20260709 13:15:21.950281 22880 leader_election.cc:336] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:21.951144 22880 leader_election.cc:336] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:21.951958 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:21.952392 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:21.955996 22841 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee" is_pre_election: true
I20260709 13:15:21.956949 22841 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
W20260709 13:15:22.011461 22880 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:22.012038 22880 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:22.015935 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 79338.7us
I20260709 13:15:22.031067 31506 raft_consensus.cc:2749] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:22.028813 22657 catalog_manager.cc:5697] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee reported cstate change: config changed from index 13 to 14, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:22.001668 31513 raft_consensus.cc:515] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:22.042567 22880 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:22.041723 31513 leader_election.cc:290] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:22.085589 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:22.086517 22880 leader_election.cc:336] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:22.124099 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:22.124801 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:22.144995 22879 leader_election.cc:304] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:22.147123 31554 raft_consensus.cc:2804] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:22.147683 31554 raft_consensus.cc:697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:22.148306 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:22.148615 31554 consensus_queue.cc:237] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:22.149183 31569 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:22.149708 31569 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:22.150182 31569 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:22.176725 22832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9fc1639a8fa47ba91cd2fc158066fb2" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:22.177613 22832 raft_consensus.cc:3060] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:15:22.178412 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:22.189087 31569 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:22.206429 23466 raft_consensus.cc:2955] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } } }
I20260709 13:15:22.215766 22659 catalog_manager.cc:5697] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 26 to 27, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 27 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:22.221463 31569 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:22.231406 31570 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:22.251449 31570 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:22.242897 22880 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:22.264516 31570 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:22.269889 23431 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:22.269896 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:22.271620 22911 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 25.
I20260709 13:15:22.274628 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:22.278934 31550 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:15:22.279520 31550 raft_consensus.cc:493] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader acfda9b350734def9b6c621f66a5f247)
I20260709 13:15:22.280140 31550 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:22.249477 22832 raft_consensus.cc:2468] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:15:22.298944 22833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "776baf137f8745dd866379ca15ab8cee"
I20260709 13:15:22.299664 22833 raft_consensus.cc:3060] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:22.313143 31550 raft_consensus.cc:515] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:22.323956 22879 leader_election.cc:304] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:22.325093 31554 raft_consensus.cc:2804] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:22.325752 31554 raft_consensus.cc:697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:22.326782 31554 consensus_queue.cc:237] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:22.338697 31550 leader_election.cc:290] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:22.340061 23431 leader_election.cc:336] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:22.408679 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 68444.2us
I20260709 13:15:22.425261 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e36250f751be4e7480619d5807252ac0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:22.427271 22913 raft_consensus.cc:3060] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:15:22.326799 22833 raft_consensus.cc:2468] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
I20260709 13:15:22.464555 22879 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee; no voters: acfda9b350734def9b6c621f66a5f247
W20260709 13:15:22.497030 21636 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:22.500522 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 49847.5us
W20260709 13:15:22.519045 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 54924.6us
I20260709 13:15:22.521348 31551 raft_consensus.cc:2804] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:22.522193 31551 raft_consensus.cc:697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:22.523840 31551 consensus_queue.cc:237] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:22.533157 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:22.541805 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:22.578280 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 107369us
W20260709 13:15:22.589366 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:22.590612 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 45661.7us
I20260709 13:15:22.594053 22913 raft_consensus.cc:2468] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 26.
W20260709 13:15:22.592706 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:22.595152 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:22.595458 22837 raft_consensus.cc:1275] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 13:15:22.606446 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:22.631855 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 76113.9us
I20260709 13:15:22.632833 23432 leader_election.cc:304] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:22.635170 31583 raft_consensus.cc:2804] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:15:22.635787 31583 raft_consensus.cc:697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
W20260709 13:15:22.637805 22807 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 146326us
I20260709 13:15:22.637182 31583 consensus_queue.cc:237] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:22.676208 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:22.680336 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 47872.3us
W20260709 13:15:22.681167 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:22.684091 31567 consensus_queue.cc:1048] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:15:22.677234 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:22.697147 22656 catalog_manager.cc:5697] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 26 to 27, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 27 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:22.729400 23430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 50131.8us
W20260709 13:15:22.743885 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:22.760411 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:22.780398 22656 catalog_manager.cc:5697] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 26 to 27, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 27 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:22.749210 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:22.852828 22659 catalog_manager.cc:5697] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 25 to 26, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 26 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:22.869274 22838 raft_consensus.cc:1275] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
W20260709 13:15:22.909415 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:22.925999 31571 consensus_queue.cc:1048] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:22.968855 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:22.986300 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:23.004185 22837 raft_consensus.cc:1275] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 13:15:23.012959 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:23.015668 31567 consensus_queue.cc:1048] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:15:23.025698 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:23.026444 22837 raft_consensus.cc:1275] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:15:23.036931 31569 consensus_queue.cc:1048] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
W20260709 13:15:23.060832 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.090153 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:23.108170 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:23.142977 31583 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.143791 31583 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:23.144457 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:23.155141 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:23.166553 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:23.168319 22880 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:23.168744 31583 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:23.165927 22657 catalog_manager.cc:5697] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee reported cstate change: term changed from 26 to 27, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 27 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } }
I20260709 13:15:23.189621 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:23.190603 22907 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 29.
W20260709 13:15:23.194439 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:23.205122 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:23.206589 31583 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:15:23.207113 31583 raft_consensus.cc:493] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.207535 31583 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:15:23.240134 31583 raft_consensus.cc:515] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:23.257675 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:23.269361 23431 leader_election.cc:336] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:23.271078 31583 leader_election.cc:290] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 election: Requested vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:23.273182 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f79e81f486b4e7dbf2d845891b6b9c0" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:23.273904 22913 raft_consensus.cc:3060] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:15:23.287319 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:23.294273 22907 raft_consensus.cc:1275] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:15:23.295624 22913 raft_consensus.cc:2468] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 30.
I20260709 13:15:23.297889 23432 leader_election.cc:304] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: acfda9b350734def9b6c621f66a5f247
I20260709 13:15:23.299268 31586 consensus_queue.cc:1048] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:15:23.300304 31583 raft_consensus.cc:2804] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:15:23.301049 31583 raft_consensus.cc:697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:23.303304 31583 consensus_queue.cc:237] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:23.375732 22656 catalog_manager.cc:5697] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 28 to 30, leader changed from acfda9b350734def9b6c621f66a5f247 (127.21.33.1) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 30 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:23.376835 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:23.497836 21636 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:23.512178 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:23.546761 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:23.622370 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:23.684614 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:23.763924 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:23.812094 22907 raft_consensus.cc:1275] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
W20260709 13:15:23.819628 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:23.823308 31583 consensus_queue.cc:1048] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 13:15:23.854492 22880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111) [suppressed 463 similar messages]
W20260709 13:15:23.911280 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:23.984994 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:23.985697 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.071728 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.103170 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.111722 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.233407 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:24.292763 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.316731 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.338466 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.354772 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:24.384838 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:24.419421 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:24.488031 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:24.498404 21636 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:24.527482 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:24.577165 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:24.751432 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.771083 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:24.790880 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:24.925772 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:24.973315 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.012606 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.014983 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.039826 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.040768 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:25.082728 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.101516 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.208935 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.303539 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.421594 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.431205 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:25.434873 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.438346 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.475569 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.478926 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.498971 21636 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:25.505146 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.543861 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.558703 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.574050 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:25.621842 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.653856 22880 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:25.708315 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.711968 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:25.875584 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:25.996860 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:26.037847 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.084951 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:26.149942 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.186625 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:26.190560 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.214648 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.237921 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:26.342132 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:26.473740 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.500032 21636 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:26.624513 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.740694 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:26.751535 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.846262 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.864128 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:26.865278 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:26.881685 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.882548 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:26.919013 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:27.061033 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:27.094533 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:27.110553 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:27.151535 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:27.204736 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:27.245673 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:27.330094 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:27.352108 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:27.392787 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:27.397799 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:27.420152 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:27.438066 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:27.500905 21636 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:27.608680 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:27.662128 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:27.682282 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:27.765040 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:27.790182 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:27.792666 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:28.118709 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 52238.4us
W20260709 13:15:28.117249 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:28.150749 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:28.195156 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 51189.6us
W20260709 13:15:28.254038 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.282737 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 83825.7us
W20260709 13:15:28.335103 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.337795 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.338830 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.339238 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:28.341861 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.342664 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:28.343660 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:28.352602 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.366464 22880 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.501520 21636 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 22787)
W20260709 13:15:28.597920 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:28.615676 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:28.720413 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:28.729632 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.732414 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.759802 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:28.809962 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:28.858887 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:28.875236 22880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111) [suppressed 491 similar messages]
W20260709 13:15:28.894315 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:28.924403 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.073390 22806 consensus_peers.cc:597] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:29.127806 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002110a0 after lost signal to thread 21639
W20260709 13:15:29.129204 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08003100c0 after lost signal to thread 22698
W20260709 13:15:29.130873 23487 debug-util.cc:398] Leaking SignalData structure 0x7b080029b9e0 after lost signal to thread 22865
W20260709 13:15:29.134063 23487 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec6e0 after lost signal to thread 22939
W20260709 13:15:29.139290 23487 debug-util.cc:398] Leaking SignalData structure 0x7b0800029140 after lost signal to thread 23490
I20260709 13:15:29.146903 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
W20260709 13:15:29.265041 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:29.271478 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.332118 22806 consensus_peers.cc:597] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:29.398838 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.489030 22806 consensus_peers.cc:597] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:29.512418 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.534658 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:29.584879 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:29.657523 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
W20260709 13:15:29.665920 23430 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.677362 22806 consensus_peers.cc:597] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.685621 23430 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.725086 22879 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.733888 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.736131 22879 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.739388 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:29.756687 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:29.795179 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:29.801889 22879 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.803936 22879 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.805976 22805 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 81015.6us
W20260709 13:15:29.816175 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:29.823845 23430 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.842520 23430 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.880014 23430 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.898517 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:37663, user_credentials={real_user=slave}} blocked reactor thread for 75475.2us
W20260709 13:15:29.907595 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:29.912683 23430 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.921999 23430 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.924674 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:29.930111 22879 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.932997 22879 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.938798 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:29.943765 23430 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.960129 22879 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.961119 22879 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.961910 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.962549 22879 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.963196 22879 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.963866 22879 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.968443 22879 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.969200 22879 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.970227 22879 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:29.969825 23432 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:35233, user_credentials={real_user=slave}} blocked reactor thread for 57092.9us
W20260709 13:15:29.993973 22806 consensus_peers.cc:597] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:30.004045 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:30.028967 23430 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.028093 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:15:30.041088 22879 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.042011 21636 tablet_replica.cc:333] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.047196 21636 raft_consensus.cc:2243] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.048026 21636 raft_consensus.cc:2272] T bb34c8c8aec0452ab45aaa676b75558a P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.050051 23430 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.051831 21636 tablet_replica.cc:333] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.053081 21636 raft_consensus.cc:2243] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.058020 21636 raft_consensus.cc:2272] T 0670a9005da848cd93427b2d2b4aa41d P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.065591 22806 consensus_peers.cc:597] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:30.074527 22879 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:30.075311 22879 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.075369 21636 tablet_replica.cc:333] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.077863 21636 raft_consensus.cc:2243] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.085879 21636 raft_consensus.cc:2272] T bac9a57b26684fe6a49a43a8a61151e6 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.089141 22879 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.089430 21636 tablet_replica.cc:333] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.091357 21636 raft_consensus.cc:2243] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.094163 21636 raft_consensus.cc:2272] T a18b5b24775b48b589edb90581a5cad5 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.096854 22879 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:30.097963 22879 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:30.098552 22879 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:30.108954 22879 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:30.109187 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:30.122641 22879 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.127501 21636 tablet_replica.cc:333] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
W20260709 13:15:30.134513 22879 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.136861 21636 raft_consensus.cc:2243] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.146770 21636 raft_consensus.cc:2272] T a663ecdecbd14089a4074b2581e94517 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.150421 22879 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.152557 21636 tablet_replica.cc:333] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.167639 21636 raft_consensus.cc:2243] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.168804 21636 raft_consensus.cc:2272] T f55844ac3c9a4ceaa35b3d5bdadad40e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.174330 21636 tablet_replica.cc:333] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
W20260709 13:15:30.169116 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:30.185834 21636 raft_consensus.cc:2243] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:30.178527 23430 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.193640 21636 raft_consensus.cc:2272] T 25ae06dc4f8940f1a419a39f791d6e18 P 776baf137f8745dd866379ca15ab8cee [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.197085 21636 tablet_replica.cc:333] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.199139 21636 raft_consensus.cc:2243] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.208575 21636 raft_consensus.cc:2272] T 0bf57270bf9649da86a032e1c9ce4bd6 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.208405 22879 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.212787 21636 tablet_replica.cc:333] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.225907 21636 raft_consensus.cc:2243] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.226886 21636 raft_consensus.cc:2272] T 7ede022de6a346e284376d4212e1f6b7 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.230300 21636 tablet_replica.cc:333] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.242419 21636 raft_consensus.cc:2243] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.244091 21636 raft_consensus.cc:2272] T 2f666ea4542d4951919f36de0b43b972 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.247689 21636 tablet_replica.cc:333] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.249992 21636 raft_consensus.cc:2243] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.251432 21636 raft_consensus.cc:2272] T 17f68c054a034150b6ed9a31489ec653 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.254691 21636 tablet_replica.cc:333] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.257395 21636 raft_consensus.cc:2243] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.270699 21636 raft_consensus.cc:2272] T bf5430bef8774025a919427a6e425b65 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.262145 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:30.274365 21636 tablet_replica.cc:333] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.275444 21636 raft_consensus.cc:2243] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.285990 21636 raft_consensus.cc:2272] T 961d7db1909a4e40a4b410a32dad4d64 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.289554 21636 tablet_replica.cc:333] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.290603 21636 raft_consensus.cc:2243] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.291538 21636 raft_consensus.cc:2272] T 3e701ce8413c4f11afc5ca006f97eb50 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.295689 21636 tablet_replica.cc:333] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.296708 21636 raft_consensus.cc:2243] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.299026 21636 raft_consensus.cc:2272] T 023c6d83f2864eedb2775b615045955e P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.302409 21636 tablet_replica.cc:333] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.320706 21636 raft_consensus.cc:2243] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.321861 21636 raft_consensus.cc:2272] T adf76c1fde374ccbbb4a0edda126b851 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.335532 21636 tablet_replica.cc:333] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.336596 21636 raft_consensus.cc:2243] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.337911 21636 raft_consensus.cc:2272] T 8670cf017371429ea31529b6f247493e P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.344177 21636 tablet_replica.cc:333] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.345055 21636 raft_consensus.cc:2243] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.367594 21636 raft_consensus.cc:2272] T 09deb1f2792d4748b024f33e42736814 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.371327 21636 tablet_replica.cc:333] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.382335 21636 raft_consensus.cc:2243] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.383278 21636 raft_consensus.cc:2272] T a17d5efe4abb490b8dc12b9e45d754ca P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.397938 21636 tablet_replica.cc:333] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
W20260709 13:15:30.399856 22806 consensus_peers.cc:597] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:30.404732 21636 raft_consensus.cc:2243] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:15:30.405918 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:30.411257 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:30.406600 21636 raft_consensus.cc:2272] T 0818ffaecde64fc0af038b3c024b6008 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.447932 21636 tablet_replica.cc:333] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.471939 21636 raft_consensus.cc:2243] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.473981 21636 raft_consensus.cc:2272] T a9fc1639a8fa47ba91cd2fc158066fb2 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.493930 21636 tablet_replica.cc:333] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.498423 21636 raft_consensus.cc:2243] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.499828 21636 raft_consensus.cc:2272] T e19799c88cba4c9194a9308860a76da7 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.497277 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:30.514235 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:30.532245 21636 tablet_replica.cc:333] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.549157 21636 raft_consensus.cc:2243] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.555904 21636 raft_consensus.cc:2272] T b00e5613b4864bd6a0873aee1004ed22 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.560179 21636 tablet_replica.cc:333] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.577208 21636 raft_consensus.cc:2243] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.582397 21636 raft_consensus.cc:2272] T 2a537de05aee48178205ec0e7e0d436e P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.563024 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:30.611253 21636 tablet_replica.cc:333] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.612933 31644 raft_consensus.cc:493] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.630918 31644 raft_consensus.cc:515] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:30.632947 21636 raft_consensus.cc:2243] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.633826 21636 raft_consensus.cc:2272] T 76de9d7042974967a385da8f613f4cb5 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.623567 31650 raft_consensus.cc:493] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:30.642275 31644 leader_election.cc:290] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:30.643466 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4572187533dd409bad9adf0de57c6940" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:30.613570 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:30.647030 31650 raft_consensus.cc:515] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:30.645561 22806 leader_election.cc:336] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.655023 31650 leader_election.cc:290] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:30.655148 22807 leader_election.cc:304] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:30.657287 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:30.663905 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:30.668076 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c26954cc1042d884cd8813a9346a27" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.672585 21636 tablet_replica.cc:333] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.679742 31650 raft_consensus.cc:2749] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:30.682853 21636 raft_consensus.cc:2243] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:15:30.693783 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:30.693866 22806 consensus_peers.cc:597] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:30.689088 21636 raft_consensus.cc:2272] T af39123de9434cc0b451f4b03d305310 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.697805 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:30.698875 21636 tablet_replica.cc:333] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.700820 22805 leader_election.cc:304] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:30.715269 21636 raft_consensus.cc:2243] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.719367 21636 raft_consensus.cc:2272] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.715965 31650 raft_consensus.cc:2749] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:30.725220 21636 tablet_replica.cc:333] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
W20260709 13:15:30.721213 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:30.727065 21636 raft_consensus.cc:2243] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.728010 21636 raft_consensus.cc:2272] T 072f69118119423dac7eba3f66879eee P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.731590 21636 tablet_replica.cc:333] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.732478 21636 raft_consensus.cc:2243] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:15:30.734196 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:30.779024 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 57436us
I20260709 13:15:30.780412 21636 raft_consensus.cc:2272] T eac8195b1d874d3aa822a4f96dee4738 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.785041 21636 tablet_replica.cc:333] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.788607 31644 raft_consensus.cc:493] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:30.758086 31650 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.789559 31650 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:30.792451 31650 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:30.793166 21636 raft_consensus.cc:2243] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.794314 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.795153 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad66b5c9842437d9320507b92b78fd5" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.796269 21636 raft_consensus.cc:2272] T a6da5381f5554dc69fb15991d351fb20 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.797593 22805 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
W20260709 13:15:30.751228 22806 consensus_peers.cc:597] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:30.798658 22806 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47490.8us
I20260709 13:15:30.799852 31644 raft_consensus.cc:515] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:30.807128 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fd0c9273e482692ddc7589cb09f02" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.801851 31650 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:30.809830 31643 raft_consensus.cc:493] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.810132 31650 raft_consensus.cc:493] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.810424 31643 raft_consensus.cc:515] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:30.811885 31650 raft_consensus.cc:515] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:30.812661 31643 leader_election.cc:290] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:30.813972 31644 leader_election.cc:290] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:30.820606 22806 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.823868 31650 leader_election.cc:290] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:30.825168 22906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae1e4795b8046c9b632ce5907911c39" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:30.826516 22806 leader_election.cc:336] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.828449 22806 leader_election.cc:304] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:30.830724 31650 raft_consensus.cc:2749] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:30.817909 21636 tablet_replica.cc:333] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.814790 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d283692c3bb4f979788fa989a74bd80" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.844832 31644 raft_consensus.cc:493] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:30.845564 31644 raft_consensus.cc:515] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:30.847991 31644 leader_election.cc:290] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:30.849716 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659f33da809544df8f5fc087bbe7cc17" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.852118 21636 raft_consensus.cc:2243] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.853554 21636 raft_consensus.cc:2272] T 1c406ce5efa449d4b877f6747d1f14ab P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.854809 22806 leader_election.cc:336] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.855759 22806 leader_election.cc:304] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:30.856544 31650 raft_consensus.cc:2749] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:30.860270 22806 leader_election.cc:336] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.861740 31650 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.862501 31650 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:30.866618 21636 tablet_replica.cc:333] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.867084 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.867739 21636 raft_consensus.cc:2243] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.868597 21636 raft_consensus.cc:2272] T 0ad66b5c9842437d9320507b92b78fd5 P 776baf137f8745dd866379ca15ab8cee [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.870323 31650 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:30.870539 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7e34bc1d8d404aae87baaaa171e0b1" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.872061 22805 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:30.873077 31650 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:30.871219 22806 leader_election.cc:304] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:30.874413 21636 tablet_replica.cc:333] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.875414 21636 raft_consensus.cc:2243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.875900 31650 raft_consensus.cc:2749] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:30.876958 21636 raft_consensus.cc:2272] T 4dd4e5f26303415aa699a9fd74ac17e8 P 776baf137f8745dd866379ca15ab8cee [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.878989 22806 leader_election.cc:336] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.879627 22806 leader_election.cc:304] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:30.880445 21636 tablet_replica.cc:333] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.880399 31650 raft_consensus.cc:2749] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:30.881441 21636 raft_consensus.cc:2243] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:30.884532 22806 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.889278 21636 raft_consensus.cc:2272] T 26c26954cc1042d884cd8813a9346a27 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.895329 21636 tablet_replica.cc:333] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.896129 21636 raft_consensus.cc:2243] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:15:30.896153 22880 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:30.897665 21636 raft_consensus.cc:2272] T bd464cd43c544f7abc90c38571db4d1c P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.901777 21636 tablet_replica.cc:333] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.902544 21636 raft_consensus.cc:2243] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.903944 21636 raft_consensus.cc:2272] T 0b88bd2b1b1d4ced918486ea751fbe25 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.908013 21636 tablet_replica.cc:333] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.910247 21636 raft_consensus.cc:2243] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.911298 21636 raft_consensus.cc:2272] T 4c3f341bf2b9433090bb7beb432566e1 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.916051 21636 tablet_replica.cc:333] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.917025 21636 raft_consensus.cc:2243] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.918016 31644 raft_consensus.cc:493] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.918623 31644 raft_consensus.cc:515] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:30.920874 31644 leader_election.cc:290] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:30.917851 21636 raft_consensus.cc:2272] T 6d283692c3bb4f979788fa989a74bd80 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.922701 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9318e6ae5d8a4923a2bc67c1c2f6a754" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.926218 21636 tablet_replica.cc:333] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.927183 21636 raft_consensus.cc:2243] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:15:30.928364 21636 raft_consensus.cc:2272] T d6a04bfd7f184dd7a86718230ea0680e P 776baf137f8745dd866379ca15ab8cee [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.932330 21636 tablet_replica.cc:333] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
W20260709 13:15:30.932817 22806 leader_election.cc:336] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:30.933059 31643 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.934082 31643 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:30.939906 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.943722 31643 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:30.943933 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1db22a7176432c86e7264f87984ba3" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.944463 22806 leader_election.cc:304] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:30.945612 22805 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:30.948215 31643 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:30.933212 21636 raft_consensus.cc:2243] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:30.949146 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:30.949720 21636 raft_consensus.cc:2272] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.951447 31644 raft_consensus.cc:670] T 2e04261a736f42d2a1fd4e24bc787276 P 776baf137f8745dd866379ca15ab8cee: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping [suppressed 2 similar messages]
I20260709 13:15:30.952486 31643 raft_consensus.cc:2749] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:30.960522 21636 tablet_replica.cc:333] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.962639 31643 raft_consensus.cc:493] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:30.963912 31643 raft_consensus.cc:515] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:30.962743 21636 raft_consensus.cc:2243] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.966781 21636 raft_consensus.cc:2272] T 485fd0c9273e482692ddc7589cb09f02 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.969967 21636 tablet_replica.cc:333] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.970522 31644 raft_consensus.cc:493] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:30.974756 31644 raft_consensus.cc:515] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:30.975883 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c52c0b7017594a0f8651f1fe9e9f51a2" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.977135 31644 leader_election.cc:290] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:30.978664 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:30.981328 21636 raft_consensus.cc:2243] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.983820 21636 raft_consensus.cc:2272] T 4572187533dd409bad9adf0de57c6940 P 776baf137f8745dd866379ca15ab8cee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:30.982174 31643 leader_election.cc:290] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:30.981590 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f76532c7d04012a58fce30b62b344d" candidate_uuid: "776baf137f8745dd866379ca15ab8cee" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:30.991569 22807 leader_election.cc:304] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2
I20260709 13:15:30.993041 21636 tablet_replica.cc:333] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:30.993045 31644 raft_consensus.cc:2749] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 13:15:30.994086 21636 raft_consensus.cc:2243] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:30.998318 21636 raft_consensus.cc:2272] T cae1e4795b8046c9b632ce5907911c39 P 776baf137f8745dd866379ca15ab8cee [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:30.999907 22806 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:31.019259 21636 tablet_replica.cc:333] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.020447 21636 raft_consensus.cc:2243] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.022301 21636 raft_consensus.cc:2272] T ba1db22a7176432c86e7264f87984ba3 P 776baf137f8745dd866379ca15ab8cee [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.021359 22806 leader_election.cc:336] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:31.025981 21636 tablet_replica.cc:333] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.026955 21636 raft_consensus.cc:2243] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:31.028205 21636 raft_consensus.cc:2272] T b86ad75e17d742e2926f8af53f082682 P 776baf137f8745dd866379ca15ab8cee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.023171 22806 leader_election.cc:304] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 776baf137f8745dd866379ca15ab8cee; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:31.037014 31643 raft_consensus.cc:2749] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:31.037993 21636 tablet_replica.cc:333] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.039068 21636 raft_consensus.cc:2243] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 25 LEADER]: Raft consensus shutting down.
W20260709 13:15:31.044855 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:31.045708 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:31.048344 21636 raft_consensus.cc:2272] T da64727f9cd84f6bb87d91e54eeac2d5 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.052011 21636 tablet_replica.cc:333] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.052855 21636 raft_consensus.cc:2243] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.053920 21636 raft_consensus.cc:2272] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 776baf137f8745dd866379ca15ab8cee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.056154 21636 tablet_replica.cc:333] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.060287 21636 raft_consensus.cc:2243] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.064106 21636 raft_consensus.cc:2272] T 659f33da809544df8f5fc087bbe7cc17 P 776baf137f8745dd866379ca15ab8cee [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.103463 21636 tablet_replica.cc:333] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.106153 21636 raft_consensus.cc:2243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.110646 21636 raft_consensus.cc:2272] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 776baf137f8745dd866379ca15ab8cee [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:31.115672 21636 tablet_replica.cc:333] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.116886 21636 raft_consensus.cc:2243] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.118232 31641 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.118861 31641 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:31.121160 31641 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:31.125428 22879 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:31.126536 21636 raft_consensus.cc:2272] T c52c0b7017594a0f8651f1fe9e9f51a2 P 776baf137f8745dd866379ca15ab8cee [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.130229 22880 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:31.130739 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:31.131534 31641 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.134968 21636 tablet_replica.cc:333] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee: stopping tablet replica
I20260709 13:15:31.135985 21636 raft_consensus.cc:2243] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:31.136933 21636 raft_consensus.cc:2272] T 82f76532c7d04012a58fce30b62b344d P 776baf137f8745dd866379ca15ab8cee [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:31.150884 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:31.175302 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:31.177418 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:31.192641 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:31.315271 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:31.328027 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:31.341400 31646 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.342195 31646 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:31.344671 31646 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:31.353619 22879 leader_election.cc:336] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:15:31.354492 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:31.358173 22879 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:31.359863 31641 raft_consensus.cc:2749] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:15:31.379045 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:31.445813 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:31.514796 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 61115.5us
I20260709 13:15:31.601147 31646 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.603406 31646 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:31.607024 31646 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:31.645427 31613 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.646234 31613 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:31.649612 31613 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:31.663749 23430 leader_election.cc:336] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:31.674186 22879 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:31.698801 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:31.699653 22907 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 26.
I20260709 13:15:31.705286 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:31.706740 31613 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:31.707309 31613 raft_consensus.cc:493] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.707780 31613 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:31.750764 31613 raft_consensus.cc:515] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:31.752877 22880 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:31.764294 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:31.765918 31646 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:31.775233 23430 leader_election.cc:336] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:31.776921 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:31.784744 31613 leader_election.cc:290] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:31.812312 31646 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.813409 31646 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:31.818814 31646 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:31.822651 22907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f68c054a034150b6ed9a31489ec653" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:31.823378 22907 raft_consensus.cc:3060] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:31.832823 22907 raft_consensus.cc:2468] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 27.
W20260709 13:15:31.843490 22879 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:31.837786 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:31.909660 31613 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.910594 31613 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:31.913460 31647 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.914151 31647 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:31.917098 31647 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:31.940369 31613 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:31.946105 23432 leader_election.cc:304] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:31.947606 31647 raft_consensus.cc:2804] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:31.948191 31647 raft_consensus.cc:697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:31.949225 31647 consensus_queue.cc:237] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:31.968147 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:31.970814 23430 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:31.986085 31613 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:31.986938 31613 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.039631 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:32.044137 22656 catalog_manager.cc:5697] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 26 to 27, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 27 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:32.046056 31613 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:32.069275 22879 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.090931 22879 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.094363 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:32.092708 31657 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.096005 31657 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.098382 31657 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:32.101658 23430 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.102258 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:32.103186 31647 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:32.110293 31641 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.111164 31641 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.114217 31657 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.114845 31657 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.117975 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:32.121670 31657 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:32.117233 22880 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:32.124737 31641 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:32.127542 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:32.127282 22879 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.133245 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:32.128332 23430 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.134270 23430 leader_election.cc:336] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.135610 23430 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.136739 23430 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.137774 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:32.139201 31660 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:32.127627 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:32.130565 22879 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.141381 31659 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:32.141670 31641 raft_consensus.cc:2749] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:32.146983 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.148224 31647 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:15:32.150169 23430 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.153997 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:32.156189 31660 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:32.158795 31641 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.159358 31641 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:32.159668 23432 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.160633 31613 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:15:32.161587 31641 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:32.163674 31659 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.164216 31659 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.165644 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:32.172617 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
W20260709 13:15:32.173444 23430 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.173467 22913 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:15:32.173038 31659 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:32.186568 22879 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.186934 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:32.187647 23462 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:15:32.195255 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:32.197688 22879 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.231904 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:32.234179 22880 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:32.245641 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:32.247509 31662 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:32.250715 31662 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:15:32.251053 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:32.251297 31662 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:32.253815 22879 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.254877 31662 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:32.251668 22880 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:32.259845 22879 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.260429 31662 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:32.268882 31662 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
W20260709 13:15:32.269711 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:32.270344 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:32.269470 31662 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:32.284950 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:32.286751 31662 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:32.274559 31641 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.287686 31641 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.290069 31641 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:32.279278 31647 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.293431 31647 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.293874 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:32.294778 23462 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 26.
I20260709 13:15:32.295934 31647 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:32.298166 22880 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:32.299819 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
W20260709 13:15:32.302348 23430 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.313926 22880 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:32.314081 23430 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.314438 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:32.316565 22879 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.317039 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:32.317303 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:32.317811 31641 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:15:32.318076 22913 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 20.
I20260709 13:15:32.318285 31641 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.318755 31641 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:32.319332 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.320878 31646 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:32.320880 31647 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:32.322790 31647 raft_consensus.cc:493] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.323182 31647 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:32.330479 31641 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.331539 31659 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:32.332376 31647 raft_consensus.cc:515] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.333151 22879 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.339498 22879 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.344869 22879 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.345326 23430 leader_election.cc:336] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.345618 31647 leader_election.cc:290] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:32.358222 31641 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:32.361039 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dd4e5f26303415aa699a9fd74ac17e8" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:32.363042 23462 raft_consensus.cc:3060] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:15:32.364390 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b86ad75e17d742e2926f8af53f082682" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870"
I20260709 13:15:32.365021 22913 raft_consensus.cc:3060] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:32.371891 23462 raft_consensus.cc:2468] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 27.
W20260709 13:15:32.372653 22879 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.374619 22913 raft_consensus.cc:2468] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
I20260709 13:15:32.375772 22879 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.377396 31659 raft_consensus.cc:2804] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:15:32.377947 31659 raft_consensus.cc:697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
I20260709 13:15:32.379191 23432 leader_election.cc:304] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.378942 31659 consensus_queue.cc:237] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:32.381096 31647 raft_consensus.cc:2804] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:32.382511 31647 raft_consensus.cc:697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Becoming Leader. State: Replica: 74c0a02901bc45c891e42d0d1bbbf6a2, State: Running, Role: LEADER
I20260709 13:15:32.383618 31647 consensus_queue.cc:237] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.386708 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:32.390900 22879 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.410386 22656 catalog_manager.cc:5697] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 26 to 27, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 27 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:32.420348 31659 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.421048 31659 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.422849 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:32.423467 31659 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:32.432027 23430 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:32.432471 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:32.433189 23462 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 30.
I20260709 13:15:32.435547 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 
I20260709 13:15:32.438364 31641 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 13:15:32.438747 31641 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.439098 31641 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Advancing to term 31
I20260709 13:15:32.439499 22659 catalog_manager.cc:5697] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 reported cstate change: term changed from 20 to 21, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4). New cstate: current_term: 21 leader_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:32.442562 22879 leader_election.cc:336] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.447901 22879 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.448784 23430 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.451618 31641 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.467248 23430 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.468689 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6a04bfd7f184dd7a86718230ea0680e" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2"
I20260709 13:15:32.469411 23462 raft_consensus.cc:3060] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Advancing to term 31
W20260709 13:15:32.475071 22879 leader_election.cc:336] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:32.479300 22879 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.484059 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:32.484661 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:32.488633 23462 raft_consensus.cc:2468] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 31.
I20260709 13:15:32.490509 22879 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:32.492105 31659 raft_consensus.cc:2804] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 13:15:32.492789 31659 raft_consensus.cc:697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 31 LEADER]: Becoming Leader. State: Replica: 6545453e74344bb2957dd3729c5b3870, State: Running, Role: LEADER
W20260709 13:15:32.493721 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:32.497992 31659 consensus_queue.cc:237] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 30.23, Last appended by leader: 23, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:32.503388 22879 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.504554 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:32.467072 31641 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 31 election: Requested vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:32.549363 21636 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22862)
W20260709 13:15:32.568405 22879 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.569485 22879 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.570210 22879 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.624183 22879 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.629415 23430 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.654495 31656 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:32.655755 31656 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:32.665004 23430 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.670878 22907 raft_consensus.cc:1275] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:15:32.672210 31656 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:32.697851 31613 consensus_queue.cc:1048] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:15:32.715487 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:32.716482 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:32.732905 22880 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:32.746919 23430 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.738011 22879 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.767938 22879 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.770479 22879 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.771821 22879 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.774427 22879 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.786015 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.785251 22656 catalog_manager.cc:5697] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 reported cstate change: term changed from 30 to 31, leader changed from 776baf137f8745dd866379ca15ab8cee (127.21.33.2) to 6545453e74344bb2957dd3729c5b3870 (127.21.33.3). New cstate: current_term: 31 leader_uuid: "6545453e74344bb2957dd3729c5b3870" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:32.794499 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:32.800068 23430 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.800786 23430 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.803155 22879 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:32.806974 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:32.804280 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:32.810484 31672 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:32.838645 23430 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:32.855113 22879 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.860343 22879 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.864692 22879 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.892827 22913 raft_consensus.cc:1275] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Refusing update from remote peer 74c0a02901bc45c891e42d0d1bbbf6a2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:15:32.894431 22879 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:32.899631 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:32.908414 31647 consensus_queue.cc:1048] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:15:32.927594 22879 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.942502 23462 raft_consensus.cc:1275] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 30 index: 23. Preceding OpId from leader: term: 31 index: 24. (index mismatch)
W20260709 13:15:32.942957 22879 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:32.955543 31672 consensus_queue.cc:1048] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
W20260709 13:15:32.958516 22879 consensus_peers.cc:597] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:33.032060 23462 raft_consensus.cc:1275] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Refusing update from remote peer 6545453e74344bb2957dd3729c5b3870: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 13:15:33.033759 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:33.044502 22879 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.072600 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:33.092265 31672 consensus_queue.cc:1048] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 13:15:33.132268 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:33.138214 22879 consensus_peers.cc:597] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:33.137491 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:33.192451 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:33.254973 22879 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:33.269222 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:33.282893 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:33.328279 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:33.360400 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:33.379334 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:33.467273 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:33.551744 21636 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22862)
W20260709 13:15:33.689872 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:33.746186 31657 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:33.747027 31657 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:33.754102 31657 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:33.763171 23430 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:33.766454 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:33.766983 23431 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:33.768105 31613 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:33.807777 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:33.833652 31656 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:33.834615 31656 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:33.839725 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:33.842684 31656 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:33.844530 22879 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:33.851462 31656 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:33.852319 31656 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:33.855124 31656 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:33.860385 22879 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:33.899405 23430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111) [suppressed 779 similar messages]
I20260709 13:15:33.908649 31672 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:33.937286 31672 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:33.946627 31672 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:34.552341 21636 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22862)
I20260709 13:15:34.843048 31657 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:34.843918 31657 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:34.847379 31657 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:34.872907 31672 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:34.873739 31672 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:34.876739 31672 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:34.882591 31672 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:34.883734 31672 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:34.892925 31672 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:34.895728 31683 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:34.897230 31683 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:34.899863 31683 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:34.894704 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:34.900661 23430 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:34.902945 23430 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:34.904838 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:34.905774 31680 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:34.929998 31687 raft_consensus.cc:493] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:34.931042 31687 raft_consensus.cc:515] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:34.933583 31687 leader_election.cc:290] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:34.935184 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:34.936353 23464 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
I20260709 13:15:34.943459 31687 raft_consensus.cc:493] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:34.944986 31689 raft_consensus.cc:493] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 74c0a02901bc45c891e42d0d1bbbf6a2)
I20260709 13:15:34.945742 31689 raft_consensus.cc:515] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:34.948386 31689 leader_election.cc:290] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:34.944221 31687 raft_consensus.cc:515] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:34.956045 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:34.957823 31687 leader_election.cc:290] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
W20260709 13:15:34.959405 22880 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:34.960632 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:34.961225 22880 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:34.961760 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:34.963240 22880 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:34.963852 31687 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:34.964934 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:34.966027 31687 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:34.966588 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:34.967574 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:34.968192 22880 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:34.966006 22879 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:34.978962 22879 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:34.979576 22879 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:34.980552 31687 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:34.989228 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9951bee4cd6b4966bef18c1e0d25eaca" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:34.992597 22880 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:34.993297 22880 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:35.000311 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71fa884e652460c841521a01aa8b5a0" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:35.000536 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:35.002424 31687 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:35.001881 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:35.003574 22880 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:35.006623 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2aaf3c1715e4d29954bd6ee38914a7a" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
W20260709 13:15:35.007788 22880 leader_election.cc:336] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:35.008796 22880 leader_election.cc:304] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:35.011498 31687 raft_consensus.cc:2749] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:35.012282 22879 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:35.013762 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:35.012782 22880 leader_election.cc:336] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:35.014705 22880 leader_election.cc:336] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:35.018443 31687 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:35.020376 22879 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:35.020998 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:35.021950 31687 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:35.028378 22879 leader_election.cc:304] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:35.029345 31687 raft_consensus.cc:2749] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:35.030874 22879 leader_election.cc:304] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 74c0a02901bc45c891e42d0d1bbbf6a2, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:35.031772 31687 raft_consensus.cc:2749] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:35.099647 31687 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:35.100685 31687 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:35.108516 22880 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:35.115281 22879 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:35.115821 22879 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:35.116699 31689 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:35.116972 31687 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:35.177361 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:35.258366 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800325b40 after lost signal to thread 21639
W20260709 13:15:35.259789 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800375360 after lost signal to thread 22698
W20260709 13:15:35.261394 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800366780 after lost signal to thread 22939
W20260709 13:15:35.262814 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800034320 after lost signal to thread 23490
W20260709 13:15:35.347275 23431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 86539.1us
W20260709 13:15:35.422535 22879 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.425796 22879 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.442466 23430 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.482903 23430 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.492098 23430 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.500152 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:35.524045 23430 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.528664 22879 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.548177 23430 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.553001 21636 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22862)
W20260709 13:15:35.569865 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:35.577004 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:35.582521 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:35.598801 22879 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.607725 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:35.621452 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:35.668890 23430 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.700950 23430 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:35.721076 22879 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.736455 22879 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.738327 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:35.755503 22879 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.762595 23430 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.774595 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:35.819340 22879 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.821954 23430 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.827975 22879 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.855986 22879 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.887298 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:35.937987 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:35.967092 22879 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.970860 22879 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.978818 23430 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.982811 23430 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:35.983390 23430 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.985195 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:35.991217 22879 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:35.997556 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.011816 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:36.013821 22879 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.016440 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.018186 22879 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.028996 22879 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.032246 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:36.033699 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:36.035063 22879 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.057459 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:36.069083 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.070556 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.077809 22879 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.103333 22879 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.120859 22879 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.123588 22879 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.124110 22879 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.151542 22879 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.179765 22879 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.242350 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:36.251452 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:36.275046 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:36.339979 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.357477 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:36.413592 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:36.495196 22879 consensus_peers.cc:597] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:36.556938 21636 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22862)
I20260709 13:15:36.573269 31680 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:36.574005 31680 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:36.587095 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:36.588266 23430 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:36.589634 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:36.588887 31680 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:36.590600 31682 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:36.600600 22879 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.625079 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.637990 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.639559 22879 consensus_peers.cc:597] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:36.642230 22879 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:36.761581 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:15:36.782428 31688 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:36.783176 31688 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:36.791676 31688 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:36.800238 22879 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:36.850382 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:36.853766 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:36.859503 22880 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:36.860226 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:36.862069 31688 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:36.881671 31688 raft_consensus.cc:493] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:36.882417 31688 raft_consensus.cc:515] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
W20260709 13:15:36.887269 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:36.905748 22879 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:36.884979 31688 leader_election.cc:290] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:36.977541 31688 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:36.978233 31688 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:36.980732 31688 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:37.005460 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:37.007681 22879 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:37.009730 23430 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:37.012133 22880 leader_election.cc:336] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:37.018072 22880 leader_election.cc:304] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:37.026573 31688 raft_consensus.cc:2749] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:37.074684 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:37.078161 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:37.103135 22880 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:37.113080 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:37.115942 31688 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:37.133158 31657 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:37.133925 31657 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:37.142836 31657 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:37.147496 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:37.149129 23430 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:37.149959 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:37.151141 31657 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:37.166579 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:37.557565 21636 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 22862)
W20260709 13:15:37.843403 22879 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:41075, user_credentials={real_user=slave}} blocked reactor thread for 670840us
W20260709 13:15:37.846841 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 676672us
I20260709 13:15:37.867518 31700 raft_consensus.cc:493] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:37.868389 31700 raft_consensus.cc:515] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:37.872157 31700 leader_election.cc:290] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:37.906071 31689 raft_consensus.cc:493] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:37.906490 31695 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:37.906876 31689 raft_consensus.cc:515] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:37.907323 31695 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:37.909687 31689 leader_election.cc:290] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:37.910493 31695 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 74c0a02901bc45c891e42d0d1bbbf6a2 (127.21.33.4:37663)
I20260709 13:15:37.919807 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "6545453e74344bb2957dd3729c5b3870" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" is_pre_election: true
I20260709 13:15:37.920718 23462 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6545453e74344bb2957dd3729c5b3870 in term 21.
W20260709 13:15:37.933912 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:38.030246 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:38.031236 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.032130 22880 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:38.036798 22879 leader_election.cc:336] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:38.037640 22879 leader_election.cc:304] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:38.043754 31689 raft_consensus.cc:2749] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:38.033214 22880 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:38.046270 22880 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:38.046411 22879 leader_election.cc:336] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:38.047071 22879 leader_election.cc:304] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:38.048324 31689 raft_consensus.cc:2749] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:38.055811 22879 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:38.057693 22879 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:38.058060 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:38.058710 31689 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:38.070709 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:38.118487 31680 raft_consensus.cc:493] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:38.119225 31680 raft_consensus.cc:515] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
W20260709 13:15:38.136435 23431 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:38.139878 23430 leader_election.cc:336] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:38.143836 23430 leader_election.cc:304] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:38.140587 31680 leader_election.cc:290] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:38.145449 31657 raft_consensus.cc:2749] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:38.191349 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800197680 after lost signal to thread 21639
W20260709 13:15:38.192821 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800091180 after lost signal to thread 22698
W20260709 13:15:38.194543 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800366e20 after lost signal to thread 22939
W20260709 13:15:38.196102 22695 debug-util.cc:398] Leaking SignalData structure 0x7b0800034600 after lost signal to thread 23490
W20260709 13:15:38.246683 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:38.301753 23430 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:38.319295 31701 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:38.320057 31701 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:38.323343 31701 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:38.348047 22880 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.358949 22879 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:38.426677 22880 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.428231 22880 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:38.433736 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:38.435786 31695 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:38.451666 23430 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.473083 22879 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.487105 22880 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.492180 23430 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.505632 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:38.514251 23430 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.527752 22880 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:38.557318 22880 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.558327 21636 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 22862)
W20260709 13:15:38.568387 22880 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:38.569010 23430 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.572077 22879 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.578794 22879 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.583518 22880 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.725914 22880 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:38.749619 22879 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.752168 23430 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.767292 22879 consensus_peers.cc:597] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.769935 22879 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.803447 22879 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.804824 22879 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.845515 22879 consensus_peers.cc:597] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.847210 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:38.854394 23430 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.879374 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:38.900959 23430 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.901156 22879 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.909636 22879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111) [suppressed 646 similar messages]
W20260709 13:15:38.911550 22879 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.915311 23430 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.916150 22879 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.916425 22880 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:45095, user_credentials={real_user=slave}} blocked reactor thread for 47218.3us
I20260709 13:15:38.924325 31689 raft_consensus.cc:493] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:38.925904 31689 raft_consensus.cc:515] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:38.932754 31697 raft_consensus.cc:493] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:38.933439 31697 raft_consensus.cc:515] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:38.935250 31689 leader_election.cc:290] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
I20260709 13:15:38.936632 31697 leader_election.cc:290] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:38.937984 22879 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:38.957628 23431 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:38.975924 22880 leader_election.cc:336] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:38.977571 22880 leader_election.cc:304] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:38.978502 31689 raft_consensus.cc:2749] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:38.980396 22879 consensus_peers.cc:597] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.981002 22879 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:38.979939 22880 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:38.995502 22879 consensus_peers.cc:597] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.003091 22880 consensus_peers.cc:597] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.005625 22880 consensus_peers.cc:597] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:39.004971 22879 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.014959 23430 leader_election.cc:336] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:39.016940 23430 leader_election.cc:304] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:39.019156 31680 raft_consensus.cc:2749] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:39.020375 22879 consensus_peers.cc:597] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.023083 22880 consensus_peers.cc:597] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.034809 22880 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:39.035630 22880 consensus_peers.cc:597] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:39.047137 23430 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:39.050905 22879 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.053376 23430 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:39.074481 23430 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.105096 22879 consensus_peers.cc:597] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.116755 22879 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.125414 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:39.178463 22879 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.220257 22879 consensus_peers.cc:597] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.232000 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:39.317700 22879 consensus_peers.cc:597] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.333349 22879 consensus_peers.cc:597] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.381016 22880 consensus_peers.cc:597] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:39.386358 22879 consensus_peers.cc:597] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:39.472963 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:39.495209 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:39.496618 22880 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.510555 22880 consensus_peers.cc:597] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:39.526260 31704 raft_consensus.cc:493] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:39.526955 31704 raft_consensus.cc:515] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:39.533037 31704 leader_election.cc:290] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:39.578735 22879 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:39.603310 22880 leader_election.cc:336] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:39.604285 22880 leader_election.cc:304] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:39.608500 31709 raft_consensus.cc:2749] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:39.614326 22880 consensus_peers.cc:597] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.670265 22879 consensus_peers.cc:597] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.681782 22879 consensus_peers.cc:597] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:15:39.887915 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
W20260709 13:15:39.894646 23430 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:39.910873 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:39.923461 22879 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:39.926873 22880 consensus_peers.cc:597] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.935160 22880 consensus_peers.cc:597] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:15:39.943353 22880 consensus_peers.cc:597] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:15:39.944126 22880 consensus_peers.cc:597] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.961122 22880 consensus_peers.cc:597] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:39.996649 22879 consensus_peers.cc:597] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:39.998417 31697 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:39.999248 31697 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:40.006533 31697 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:40.012385 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:40.016125 22908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af39123de9434cc0b451f4b03d305310" candidate_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6545453e74344bb2957dd3729c5b3870" is_pre_election: true
I20260709 13:15:40.017091 22908 raft_consensus.cc:2468] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74c0a02901bc45c891e42d0d1bbbf6a2 in term 21.
W20260709 13:15:40.025483 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:40.069979 23430 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:40.074234 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870, 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:40.075294 31697 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:40.073647 22880 consensus_peers.cc:597] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:40.082520 22880 consensus_peers.cc:597] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:40.157963 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:15:40.177152 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
W20260709 13:15:40.235400 23432 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:40.255749 23432 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:40.261983 23432 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:40.299290 22880 consensus_peers.cc:597] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:40.299851 22880 consensus_peers.cc:597] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:15:40.300915 23432 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:40.323648 22880 consensus_peers.cc:597] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:15:40.347342 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:40.351553 21636 tablet_replica.cc:333] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.353132 21636 raft_consensus.cc:2243] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.355363 21636 raft_consensus.cc:2272] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 6545453e74344bb2957dd3729c5b3870 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.359404 21636 tablet_replica.cc:333] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.360651 21636 raft_consensus.cc:2243] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.362452 21636 raft_consensus.cc:2272] T bb34c8c8aec0452ab45aaa676b75558a P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.366060 21636 tablet_replica.cc:333] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.367601 21636 raft_consensus.cc:2243] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.369202 21636 raft_consensus.cc:2272] T 0670a9005da848cd93427b2d2b4aa41d P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.372478 21636 tablet_replica.cc:333] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.373461 21636 raft_consensus.cc:2243] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.375211 21636 raft_consensus.cc:2272] T bac9a57b26684fe6a49a43a8a61151e6 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.378705 21636 tablet_replica.cc:333] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.380007 21636 raft_consensus.cc:2243] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.381798 21636 raft_consensus.cc:2272] T a18b5b24775b48b589edb90581a5cad5 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.385087 21636 tablet_replica.cc:333] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.386236 21636 raft_consensus.cc:2243] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.387981 21636 raft_consensus.cc:2272] T 0bf57270bf9649da86a032e1c9ce4bd6 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.391551 21636 tablet_replica.cc:333] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.392588 21636 raft_consensus.cc:2243] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.393700 21636 raft_consensus.cc:2272] T a663ecdecbd14089a4074b2581e94517 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.400064 21636 tablet_replica.cc:333] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.401305 21636 raft_consensus.cc:2243] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.403085 21636 raft_consensus.cc:2272] T 8670cf017371429ea31529b6f247493e P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.406833 21636 tablet_replica.cc:333] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.407826 21636 raft_consensus.cc:2243] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.410215 21636 raft_consensus.cc:2272] T bf5430bef8774025a919427a6e425b65 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.414191 21636 tablet_replica.cc:333] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.414980 21636 raft_consensus.cc:2243] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.416507 21636 raft_consensus.cc:2272] T 961d7db1909a4e40a4b410a32dad4d64 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.420233 21636 tablet_replica.cc:333] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.421975 21636 raft_consensus.cc:2243] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.424155 21636 raft_consensus.cc:2272] T 3e701ce8413c4f11afc5ca006f97eb50 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.427713 21636 tablet_replica.cc:333] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
W20260709 13:15:40.428639 23432 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.428741 21636 raft_consensus.cc:2243] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.430596 21636 raft_consensus.cc:2272] T 023c6d83f2864eedb2775b615045955e P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.433384 21636 tablet_replica.cc:333] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.434245 21636 raft_consensus.cc:2243] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.435617 21636 raft_consensus.cc:2272] T 25ae06dc4f8940f1a419a39f791d6e18 P 6545453e74344bb2957dd3729c5b3870 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.440490 21636 tablet_replica.cc:333] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.441587 21636 raft_consensus.cc:2243] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.442634 21636 raft_consensus.cc:2272] T 17f68c054a034150b6ed9a31489ec653 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.445814 21636 tablet_replica.cc:333] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.446758 21636 raft_consensus.cc:2243] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.449283 21636 raft_consensus.cc:2272] T a17d5efe4abb490b8dc12b9e45d754ca P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.453902 21636 tablet_replica.cc:333] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.454814 21636 raft_consensus.cc:2243] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.455497 21636 raft_consensus.cc:2272] T 0818ffaecde64fc0af038b3c024b6008 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.458942 21636 tablet_replica.cc:333] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.460713 21636 raft_consensus.cc:2243] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.462157 21636 raft_consensus.cc:2272] T 11f984a583c84fbe9ef7b3c817b06c44 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.468230 21636 tablet_replica.cc:333] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.473692 21636 raft_consensus.cc:2243] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.475106 21636 raft_consensus.cc:2272] T 9951bee4cd6b4966bef18c1e0d25eaca P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.478646 21636 tablet_replica.cc:333] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.479617 21636 raft_consensus.cc:2243] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Raft consensus shutting down.
W20260709 13:15:40.481980 23432 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.483363 21636 raft_consensus.cc:2272] T a9fc1639a8fa47ba91cd2fc158066fb2 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:40.491524 22880 consensus_peers.cc:597] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:15:40.491211 21636 tablet_replica.cc:333] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.493086 21636 raft_consensus.cc:2243] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.495074 21636 raft_consensus.cc:2272] T b00e5613b4864bd6a0873aee1004ed22 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.498363 21636 tablet_replica.cc:333] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.499302 21636 raft_consensus.cc:2243] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.501911 21636 raft_consensus.cc:2272] T 76de9d7042974967a385da8f613f4cb5 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.507115 21636 tablet_replica.cc:333] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
W20260709 13:15:40.508353 23432 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.508386 21636 raft_consensus.cc:2243] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.510283 21636 raft_consensus.cc:2272] T af39123de9434cc0b451f4b03d305310 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.513571 21636 tablet_replica.cc:333] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.514708 21636 raft_consensus.cc:2243] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.516068 21636 raft_consensus.cc:2272] T 11d0cdd09217481cbfc11c1d2d14a5a0 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:40.516443 23432 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:40.519950 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:15:40.520032 21636 tablet_replica.cc:333] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.522370 21636 raft_consensus.cc:2243] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.523811 21636 raft_consensus.cc:2272] T 072f69118119423dac7eba3f66879eee P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.527251 21636 tablet_replica.cc:333] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.528141 21636 raft_consensus.cc:2243] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.528882 21636 raft_consensus.cc:2272] T 1c406ce5efa449d4b877f6747d1f14ab P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.532037 21636 tablet_replica.cc:333] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.532980 21636 raft_consensus.cc:2243] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.533576 31712 raft_consensus.cc:493] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:40.534399 21636 raft_consensus.cc:2272] T f2aaf3c1715e4d29954bd6ee38914a7a P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:40.534292 23432 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.534237 31712 raft_consensus.cc:515] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } }
I20260709 13:15:40.537216 31712 leader_election.cc:290] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075)
W20260709 13:15:40.542328 22879 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:40.542829 22880 leader_election.cc:336] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:40.543229 22880 leader_election.cc:304] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6545453e74344bb2957dd3729c5b3870; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:40.545293 31712 raft_consensus.cc:2749] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:40.548460 21636 tablet_replica.cc:333] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
W20260709 13:15:40.548547 23432 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.549724 21636 raft_consensus.cc:2243] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.550595 21636 raft_consensus.cc:2272] T eac8195b1d874d3aa822a4f96dee4738 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.553346 21636 tablet_replica.cc:333] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.554065 21636 raft_consensus.cc:2243] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.556725 21636 raft_consensus.cc:2272] T 0ad66b5c9842437d9320507b92b78fd5 P 6545453e74344bb2957dd3729c5b3870 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.560302 21636 tablet_replica.cc:333] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.561137 21636 raft_consensus.cc:2243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.562561 21636 raft_consensus.cc:2272] T 4dd4e5f26303415aa699a9fd74ac17e8 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.565936 21636 tablet_replica.cc:333] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.566613 21636 raft_consensus.cc:2243] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.567139 21636 raft_consensus.cc:2272] T 26c26954cc1042d884cd8813a9346a27 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.569873 21636 tablet_replica.cc:333] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.570648 21636 raft_consensus.cc:2243] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.571902 21636 raft_consensus.cc:2272] T bd464cd43c544f7abc90c38571db4d1c P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.575768 21636 tablet_replica.cc:333] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.576711 21636 raft_consensus.cc:2243] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.577744 21636 raft_consensus.cc:2272] T a71fa884e652460c841521a01aa8b5a0 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:40.577867 23432 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.588536 21636 tablet_replica.cc:333] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.589690 21636 raft_consensus.cc:2243] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.591625 21636 raft_consensus.cc:2272] T 6d283692c3bb4f979788fa989a74bd80 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.596647 21636 tablet_replica.cc:333] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.597527 21636 raft_consensus.cc:2243] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.599078 21636 raft_consensus.cc:2272] T 4c3f341bf2b9433090bb7beb432566e1 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.612119 21636 tablet_replica.cc:333] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.613035 21636 raft_consensus.cc:2243] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.619622 21636 raft_consensus.cc:2272] T 38001bb9b85f4814aff1beccb3ae9acb P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.623754 21636 tablet_replica.cc:333] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.624603 21636 raft_consensus.cc:2243] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.626250 21636 raft_consensus.cc:2272] T 2e04261a736f42d2a1fd4e24bc787276 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:40.628087 23432 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.630453 21636 tablet_replica.cc:333] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.631397 21636 raft_consensus.cc:2243] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 31 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.632655 21636 raft_consensus.cc:2272] T d6a04bfd7f184dd7a86718230ea0680e P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.635545 21636 tablet_replica.cc:333] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
W20260709 13:15:40.635896 23430 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:40.636474 21636 raft_consensus.cc:2243] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.638028 21636 raft_consensus.cc:2272] T c52c0b7017594a0f8651f1fe9e9f51a2 P 6545453e74344bb2957dd3729c5b3870 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.641302 21636 tablet_replica.cc:333] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.642160 21636 raft_consensus.cc:2243] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.642933 21636 raft_consensus.cc:2272] T 82f76532c7d04012a58fce30b62b344d P 6545453e74344bb2957dd3729c5b3870 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.646111 21636 tablet_replica.cc:333] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.647195 21636 raft_consensus.cc:2243] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.648747 21636 raft_consensus.cc:2272] T ba1db22a7176432c86e7264f87984ba3 P 6545453e74344bb2957dd3729c5b3870 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.652593 21636 tablet_replica.cc:333] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.653465 21636 raft_consensus.cc:2243] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.655212 21636 raft_consensus.cc:2272] T e36250f751be4e7480619d5807252ac0 P 6545453e74344bb2957dd3729c5b3870 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.658017 21636 tablet_replica.cc:333] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.658612 21636 raft_consensus.cc:2243] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.660234 21636 raft_consensus.cc:2272] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.663358 21636 tablet_replica.cc:333] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.668449 21636 raft_consensus.cc:2243] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.670392 21636 raft_consensus.cc:2272] T 9318e6ae5d8a4923a2bc67c1c2f6a754 P 6545453e74344bb2957dd3729c5b3870 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.674190 21636 tablet_replica.cc:333] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.675118 21636 raft_consensus.cc:2243] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.676158 21636 raft_consensus.cc:2272] T b86ad75e17d742e2926f8af53f082682 P 6545453e74344bb2957dd3729c5b3870 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.680050 21636 tablet_replica.cc:333] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.680979 21636 raft_consensus.cc:2243] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.681955 21636 raft_consensus.cc:2272] T 2f666ea4542d4951919f36de0b43b972 P 6545453e74344bb2957dd3729c5b3870 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.684818 21636 tablet_replica.cc:333] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.685606 21636 raft_consensus.cc:2243] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.687227 21636 raft_consensus.cc:2272] T 7ede022de6a346e284376d4212e1f6b7 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.690644 21636 tablet_replica.cc:333] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.691596 21636 raft_consensus.cc:2243] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.692970 21636 raft_consensus.cc:2272] T cae1e4795b8046c9b632ce5907911c39 P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.695881 21636 tablet_replica.cc:333] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.696580 21636 raft_consensus.cc:2243] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.697790 21636 raft_consensus.cc:2272] T f12b50ff391746348739bd5f7a08a44a P 6545453e74344bb2957dd3729c5b3870 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.700320 21636 tablet_replica.cc:333] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.701002 21636 raft_consensus.cc:2243] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.701764 21636 raft_consensus.cc:2272] T 09deb1f2792d4748b024f33e42736814 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.704104 21636 tablet_replica.cc:333] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.704712 21636 raft_consensus.cc:2243] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:40.706060 21636 raft_consensus.cc:2272] T 4572187533dd409bad9adf0de57c6940 P 6545453e74344bb2957dd3729c5b3870 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:40.708835 21636 tablet_replica.cc:333] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870: stopping tablet replica
I20260709 13:15:40.709393 21636 raft_consensus.cc:2243] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:40.710471 21636 raft_consensus.cc:2272] T da64727f9cd84f6bb87d91e54eeac2d5 P 6545453e74344bb2957dd3729c5b3870 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:40.720558 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:40.737370 23432 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:40.855763 23430 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:40.955430 23430 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:40.983376 23431 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:41.072194 23430 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:41.128468 23430 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:41.178800 23430 consensus_peers.cc:597] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:41.215473 23431 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:41.286144 23430 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:41.413347 23431 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:41.517798 23430 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:41.521662 23430 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:41.524966 23430 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:41.562677 23430 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:41.566093 23430 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:15:41.703575 31713 raft_consensus.cc:493] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:41.704574 31713 raft_consensus.cc:515] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:41.705256 31680 raft_consensus.cc:493] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:41.706137 31680 raft_consensus.cc:515] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:41.708964 31680 leader_election.cc:290] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
I20260709 13:15:41.710055 31713 leader_election.cc:290] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:41.718993 23431 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:41.719689 23432 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
I20260709 13:15:41.720216 23432 leader_election.cc:304] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:41.720840 23430 leader_election.cc:336] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:41.721072 31713 raft_consensus.cc:2749] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:41.721045 23432 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
W20260709 13:15:41.723284 23431 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:41.723798 23431 leader_election.cc:304] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:41.724632 31680 raft_consensus.cc:2749] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:41.726105 23430 leader_election.cc:336] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:41.739182 31680 raft_consensus.cc:493] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:41.739807 31680 raft_consensus.cc:515] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:41.742129 31680 leader_election.cc:290] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:41.748204 23432 leader_election.cc:336] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
W20260709 13:15:41.749269 23430 leader_election.cc:336] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:41.749743 23430 leader_election.cc:304] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:41.750516 31680 raft_consensus.cc:2749] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 13:15:41.768987 23431 consensus_peers.cc:597] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:15:41.780849 31680 raft_consensus.cc:493] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:41.781558 31680 raft_consensus.cc:515] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:41.783927 31680 leader_election.cc:290] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:41.789728 23431 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:41.790490 23432 leader_election.cc:336] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
I20260709 13:15:41.790990 23432 leader_election.cc:304] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:41.791946 31680 raft_consensus.cc:2749] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:41.818977 23431 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:41.849594 21636 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22936)
W20260709 13:15:41.900064 23431 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:15:41.983642 31713 raft_consensus.cc:493] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:41.984275 31713 raft_consensus.cc:515] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:41.987952 31713 leader_election.cc:290] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:41.994207 23430 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:41.995224 23432 leader_election.cc:336] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
I20260709 13:15:41.995685 23432 leader_election.cc:304] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, 776baf137f8745dd866379ca15ab8cee
I20260709 13:15:41.996690 31713 raft_consensus.cc:2749] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:15:42.043696 31713 raft_consensus.cc:493] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:42.044449 31713 raft_consensus.cc:515] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } attrs { replace: true } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:42.048018 31713 leader_election.cc:290] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:42.054315 23432 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
W20260709 13:15:42.055783 23431 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
I20260709 13:15:42.056221 23431 leader_election.cc:304] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:42.057849 31713 raft_consensus.cc:2749] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:42.062332 23431 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:42.062685 23430 leader_election.cc:336] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
W20260709 13:15:42.111613 23430 consensus_peers.cc:597] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:42.113261 31713 raft_consensus.cc:493] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6545453e74344bb2957dd3729c5b3870)
I20260709 13:15:42.114569 31713 raft_consensus.cc:515] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:42.119699 31713 leader_election.cc:290] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:42.126647 23431 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:42.132357 23432 leader_election.cc:336] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
I20260709 13:15:42.132969 23432 leader_election.cc:304] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 6545453e74344bb2957dd3729c5b3870, acfda9b350734def9b6c621f66a5f247
I20260709 13:15:42.144805 31713 raft_consensus.cc:2749] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:15:42.462991 23431 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:42.744174 23432 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:42.796326 23431 consensus_peers.cc:597] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:15:42.864154 23432 consensus_peers.cc:597] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:42.885826 23430 consensus_peers.cc:597] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:42.936185 23432 consensus_peers.cc:597] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:42.958644 23432 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:42.969177 23432 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:42.992110 31718 raft_consensus.cc:493] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 776baf137f8745dd866379ca15ab8cee)
I20260709 13:15:42.993728 31718 raft_consensus.cc:515] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "acfda9b350734def9b6c621f66a5f247" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 45095 } } peers { permanent_uuid: "776baf137f8745dd866379ca15ab8cee" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41075 } } peers { permanent_uuid: "6545453e74344bb2957dd3729c5b3870" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 35233 } attrs { replace: true } } peers { permanent_uuid: "74c0a02901bc45c891e42d0d1bbbf6a2" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 37663 } attrs { promote: false } }
I20260709 13:15:42.997341 31718 leader_election.cc:290] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095), 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075), 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233)
W20260709 13:15:43.008365 23431 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:15:43.011153 23430 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111)
I20260709 13:15:43.012247 23430 leader_election.cc:304] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 74c0a02901bc45c891e42d0d1bbbf6a2; no voters: 776baf137f8745dd866379ca15ab8cee, acfda9b350734def9b6c621f66a5f247
W20260709 13:15:43.012851 23431 consensus_peers.cc:597] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:15:43.013319 31680 raft_consensus.cc:2749] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:43.013269 23432 leader_election.cc:336] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
W20260709 13:15:43.019241 23432 consensus_peers.cc:597] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:43.023478 23431 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:15:43.030512 23432 consensus_peers.cc:597] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:43.079139 23432 consensus_peers.cc:597] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:43.088158 23432 consensus_peers.cc:597] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:43.103538 23432 consensus_peers.cc:597] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:43.105159 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
W20260709 13:15:43.110396 23432 consensus_peers.cc:597] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:43.139987 23432 consensus_peers.cc:597] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:43.157429 23430 consensus_peers.cc:597] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:43.183401 23432 consensus_peers.cc:597] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): Couldn't send request to peer 6545453e74344bb2957dd3729c5b3870. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:43.294402 21636 tablet_server.cc:179] TabletServer@127.21.33.4:0 shutting down...
W20260709 13:15:43.335644 23430 consensus_peers.cc:597] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer 776baf137f8745dd866379ca15ab8cee (127.21.33.2:41075): Couldn't send request to peer 776baf137f8745dd866379ca15ab8cee. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:41075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:43.366978 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:43.368213 21636 tablet_replica.cc:333] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.369755 21636 raft_consensus.cc:2243] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.371503 21636 raft_consensus.cc:2272] T 2f666ea4542d4951919f36de0b43b972 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.375356 21636 tablet_replica.cc:333] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.376514 21636 raft_consensus.cc:2243] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.378105 21636 raft_consensus.cc:2272] T 6a7e34bc1d8d404aae87baaaa171e0b1 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.382985 21636 tablet_replica.cc:333] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.383971 21636 raft_consensus.cc:2243] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.386121 21636 raft_consensus.cc:2272] T 659f33da809544df8f5fc087bbe7cc17 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.390002 21636 tablet_replica.cc:333] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.390868 21636 raft_consensus.cc:2243] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.392679 21636 raft_consensus.cc:2272] T 82f76532c7d04012a58fce30b62b344d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.396729 21636 tablet_replica.cc:333] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.398051 21636 raft_consensus.cc:2243] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.400287 21636 raft_consensus.cc:2272] T f2aaf3c1715e4d29954bd6ee38914a7a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.404031 21636 tablet_replica.cc:333] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.405108 21636 raft_consensus.cc:2243] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Raft consensus shutting down.
W20260709 13:15:43.406823 23431 consensus_peers.cc:597] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 -> Peer acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): Couldn't send request to peer acfda9b350734def9b6c621f66a5f247. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:43.406842 21636 raft_consensus.cc:2272] T 17f68c054a034150b6ed9a31489ec653 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.410940 21636 tablet_replica.cc:333] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.412118 21636 raft_consensus.cc:2243] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.413918 21636 raft_consensus.cc:2272] T a71fa884e652460c841521a01aa8b5a0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.417368 21636 tablet_replica.cc:333] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.418360 21636 raft_consensus.cc:2243] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.420390 21636 raft_consensus.cc:2272] T ba1db22a7176432c86e7264f87984ba3 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.424084 21636 tablet_replica.cc:333] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.425014 21636 raft_consensus.cc:2243] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.426800 21636 raft_consensus.cc:2272] T e36250f751be4e7480619d5807252ac0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.430485 21636 tablet_replica.cc:333] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.431356 21636 raft_consensus.cc:2243] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.432291 21636 raft_consensus.cc:2272] T f12b50ff391746348739bd5f7a08a44a P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.435760 21636 tablet_replica.cc:333] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.436610 21636 raft_consensus.cc:2243] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.438148 21636 raft_consensus.cc:2272] T f55844ac3c9a4ceaa35b3d5bdadad40e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.441431 21636 tablet_replica.cc:333] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.442384 21636 raft_consensus.cc:2243] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.443202 21636 raft_consensus.cc:2272] T 0b88bd2b1b1d4ced918486ea751fbe25 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.446631 21636 tablet_replica.cc:333] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.447530 21636 raft_consensus.cc:2243] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.451423 21636 raft_consensus.cc:2272] T b86ad75e17d742e2926f8af53f082682 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.455226 21636 tablet_replica.cc:333] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.456154 21636 raft_consensus.cc:2243] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.458150 21636 raft_consensus.cc:2272] T 4f79e81f486b4e7dbf2d845891b6b9c0 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.461560 21636 tablet_replica.cc:333] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.462421 21636 raft_consensus.cc:2243] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.463275 21636 raft_consensus.cc:2272] T 4dd4e5f26303415aa699a9fd74ac17e8 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.466625 21636 tablet_replica.cc:333] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.468371 21636 raft_consensus.cc:2243] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.469872 21636 raft_consensus.cc:2272] T 9951bee4cd6b4966bef18c1e0d25eaca P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.473294 21636 tablet_replica.cc:333] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.474283 21636 raft_consensus.cc:2243] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.475822 21636 raft_consensus.cc:2272] T 11f984a583c84fbe9ef7b3c817b06c44 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.479110 21636 tablet_replica.cc:333] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.479964 21636 raft_consensus.cc:2243] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.481560 21636 raft_consensus.cc:2272] T e19799c88cba4c9194a9308860a76da7 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.485670 21636 tablet_replica.cc:333] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.486511 21636 raft_consensus.cc:2243] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.488046 21636 raft_consensus.cc:2272] T 0670a9005da848cd93427b2d2b4aa41d P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.490916 21636 tablet_replica.cc:333] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.491811 21636 raft_consensus.cc:2243] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.493378 21636 raft_consensus.cc:2272] T adf76c1fde374ccbbb4a0edda126b851 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.496657 21636 tablet_replica.cc:333] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.497455 21636 raft_consensus.cc:2243] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.500540 21636 raft_consensus.cc:2272] T 09deb1f2792d4748b024f33e42736814 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.504280 21636 tablet_replica.cc:333] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.505121 21636 raft_consensus.cc:2243] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.505877 21636 raft_consensus.cc:2272] T af39123de9434cc0b451f4b03d305310 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.508342 21636 tablet_replica.cc:333] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.508944 21636 raft_consensus.cc:2243] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.509667 21636 raft_consensus.cc:2272] T 38001bb9b85f4814aff1beccb3ae9acb P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.512874 21636 tablet_replica.cc:333] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.513677 21636 raft_consensus.cc:2243] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.515067 21636 raft_consensus.cc:2272] T 485fd0c9273e482692ddc7589cb09f02 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.517373 21636 tablet_replica.cc:333] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.518086 21636 raft_consensus.cc:2243] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.518814 21636 raft_consensus.cc:2272] T a6da5381f5554dc69fb15991d351fb20 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.522624 21636 tablet_replica.cc:333] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.523941 21636 raft_consensus.cc:2243] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.526358 21636 raft_consensus.cc:2272] T 2a537de05aee48178205ec0e7e0d436e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.531246 21636 tablet_replica.cc:333] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.532459 21636 raft_consensus.cc:2243] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.533998 21636 raft_consensus.cc:2272] T d6a04bfd7f184dd7a86718230ea0680e P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.538607 21636 tablet_replica.cc:333] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.539184 21636 raft_consensus.cc:2243] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.539904 21636 raft_consensus.cc:2272] T 0ad66b5c9842437d9320507b92b78fd5 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.542665 21636 tablet_replica.cc:333] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2: stopping tablet replica
I20260709 13:15:43.543151 21636 raft_consensus.cc:2243] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.544351 21636 raft_consensus.cc:2272] T 26c26954cc1042d884cd8813a9346a27 P 74c0a02901bc45c891e42d0d1bbbf6a2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.875568 21636 tablet_server.cc:196] TabletServer@127.21.33.4:0 shutdown complete.
I20260709 13:15:43.954499 21636 master.cc:562] Master@127.21.33.62:44143 shutting down...
W20260709 13:15:43.999892 22712 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:15:44.000270 22712 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:44.000555 22712 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:52.014958 22650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111) [suppressed 526 similar messages]
W20260709 13:15:52.018709 22650 catalog_manager.cc:4755] TS acfda9b350734def9b6c621f66a5f247 (127.21.33.1:45095): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2f666ea4542d4951919f36de0b43b972: Network error: Client connection negotiation failed: client connection to 127.21.33.1:45095: connect: Connection refused (error 111)
W20260709 13:16:13.904662 22651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
W20260709 13:16:13.908990 22651 catalog_manager.cc:4755] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): ChangeConfig:REMOVE_PEER RPC failed for tablet ba1db22a7176432c86e7264f87984ba3: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
W20260709 13:16:15.708446 22651 catalog_manager.cc:4755] TS 6545453e74344bb2957dd3729c5b3870 (127.21.33.3:35233): ChangeConfig:REMOVE_PEER RPC failed for tablet 0ad66b5c9842437d9320507b92b78fd5: Network error: Client connection negotiation failed: client connection to 127.21.33.3:35233: connect: Connection refused (error 111)
I20260709 13:16:17.183300 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:17.184710 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:17.185220 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6e189717484b49e8bb20c98667e6d294: stopping tablet replica
I20260709 13:16:17.354595 21636 master.cc:584] Master@127.21.33.62:44143 shutdown complete.
I20260709 13:16:17.412292 21636 test_util.cc:182] -----------------------------------------------
I20260709 13:16:17.412575 21636 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602629827198-21636-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (321724 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:16:17.417285 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:43497
I20260709 13:16:17.418767 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:17.426249 31729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:17.426308 31728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:17.426494 31731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:17.428316 21636 server_base.cc:1061] running on GCE node
I20260709 13:16:17.432934 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:17.433244 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:17.433450 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602977433421 us; error 0 us; skew 500 ppm
I20260709 13:16:17.434131 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:17.437044 21636 webserver.cc:533] Webserver started at http://127.21.33.62:35239/ using document root <none> and password file <none>
I20260709 13:16:17.437698 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:17.437919 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:17.438236 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:17.439774 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "5f865afa026547d5aa8394713f4c8ddf"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.444824 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:17.448877 31736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.449973 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:16:17.450317 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "5f865afa026547d5aa8394713f4c8ddf"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.450680 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:17.473912 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:17.475390 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:17.527817 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:43497
I20260709 13:16:17.527969 31787 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:43497 every 8 connection(s)
I20260709 13:16:17.533048 31788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:17.548137 31788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf: Bootstrap starting.
I20260709 13:16:17.554545 31788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:17.560036 31788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf: No bootstrap required, opened a new log
I20260709 13:16:17.562820 31788 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f865afa026547d5aa8394713f4c8ddf" member_type: VOTER }
I20260709 13:16:17.563364 31788 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:17.563606 31788 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f865afa026547d5aa8394713f4c8ddf, State: Initialized, Role: FOLLOWER
I20260709 13:16:17.564414 31788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f865afa026547d5aa8394713f4c8ddf" member_type: VOTER }
I20260709 13:16:17.564875 31788 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:17.565145 31788 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:17.565439 31788 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:17.572589 31788 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f865afa026547d5aa8394713f4c8ddf" member_type: VOTER }
I20260709 13:16:17.573338 31788 leader_election.cc:304] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f865afa026547d5aa8394713f4c8ddf; no voters: 
I20260709 13:16:17.574796 31788 leader_election.cc:290] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:17.575238 31791 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:17.576799 31791 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 1 LEADER]: Becoming Leader. State: Replica: 5f865afa026547d5aa8394713f4c8ddf, State: Running, Role: LEADER
I20260709 13:16:17.577673 31791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f865afa026547d5aa8394713f4c8ddf" member_type: VOTER }
I20260709 13:16:17.578752 31788 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:17.587213 31793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f865afa026547d5aa8394713f4c8ddf. Latest consensus state: current_term: 1 leader_uuid: "5f865afa026547d5aa8394713f4c8ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f865afa026547d5aa8394713f4c8ddf" member_type: VOTER } }
I20260709 13:16:17.587136 31792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f865afa026547d5aa8394713f4c8ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f865afa026547d5aa8394713f4c8ddf" member_type: VOTER } }
I20260709 13:16:17.588153 31793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:17.588238 31792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:17.590251 31801 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:17.595577 31801 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:17.597430 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:17.606407 31801 catalog_manager.cc:1383] Generated new cluster ID: dd9160e7fd394b5eaaf4b18e128fa025
I20260709 13:16:17.606761 31801 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:17.623261 31801 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:17.624725 31801 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:17.636999 31801 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf: Generated new TSK 0
I20260709 13:16:17.637773 31801 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:17.665037 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:17.673277 31809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:17.674093 31810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:17.675762 31812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:17.678309 21636 server_base.cc:1061] running on GCE node
I20260709 13:16:17.681600 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:17.681871 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:17.682076 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602977682052 us; error 0 us; skew 500 ppm
I20260709 13:16:17.682710 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:17.686074 21636 webserver.cc:533] Webserver started at http://127.21.33.1:33119/ using document root <none> and password file <none>
I20260709 13:16:17.686694 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:17.686897 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:17.687264 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:17.688735 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "103d6e4c312a47ff9b258c14ac2871e5"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.694824 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:16:17.699039 31817 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.700294 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:16:17.700809 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "103d6e4c312a47ff9b258c14ac2871e5"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.701218 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:17.719512 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:17.720952 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:17.722728 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:17.725600 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:17.725842 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.726130 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:17.726347 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.782192 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:35847
I20260709 13:16:17.782339 31879 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:35847 every 8 connection(s)
I20260709 13:16:17.788333 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:17.796836 31884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:17.798367 31885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:17.803246 21636 server_base.cc:1061] running on GCE node
W20260709 13:16:17.803402 31888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:17.806092 31880 heartbeater.cc:344] Connected to a master server at 127.21.33.62:43497
I20260709 13:16:17.807523 31880 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:17.807796 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:17.808066 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:17.808296 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602977808275 us; error 0 us; skew 500 ppm
I20260709 13:16:17.808822 31880 heartbeater.cc:507] Master 127.21.33.62:43497 requested a full tablet report, sending...
I20260709 13:16:17.808959 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:17.812836 31753 ts_manager.cc:194] Registered new tserver with Master: 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:17.812879 21636 webserver.cc:533] Webserver started at http://127.21.33.2:37215/ using document root <none> and password file <none>
I20260709 13:16:17.813709 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:17.813952 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:17.814239 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:17.815392 31753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34408
I20260709 13:16:17.816066 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "a588481f03f6474d9009d5bd5e08e887"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.821635 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:16:17.826058 31892 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.827028 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:16:17.827402 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "a588481f03f6474d9009d5bd5e08e887"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.827719 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:17.841457 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:17.843076 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:17.844985 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:17.847776 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:17.848049 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.848376 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:17.848528 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.909901 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:37703
I20260709 13:16:17.910048 31954 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:37703 every 8 connection(s)
I20260709 13:16:17.920017 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:17.929590 31958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:17.934053 31959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:17.938251 21636 server_base.cc:1061] running on GCE node
W20260709 13:16:17.937820 31961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:17.939056 31955 heartbeater.cc:344] Connected to a master server at 127.21.33.62:43497
I20260709 13:16:17.939661 31955 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:17.939986 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:17.940300 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:17.940531 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602977940508 us; error 0 us; skew 500 ppm
I20260709 13:16:17.940800 31955 heartbeater.cc:507] Master 127.21.33.62:43497 requested a full tablet report, sending...
I20260709 13:16:17.941422 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:17.944031 31753 ts_manager.cc:194] Registered new tserver with Master: a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:17.945119 21636 webserver.cc:533] Webserver started at http://127.21.33.3:37805/ using document root <none> and password file <none>
I20260709 13:16:17.945899 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:17.946156 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:17.946264 31753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34422
I20260709 13:16:17.946557 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:17.948429 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "ccf4710690c7473d814836da1babee51"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.953912 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:16:17.959045 31966 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.960147 21636 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:16:17.960466 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "ccf4710690c7473d814836da1babee51"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:17.960731 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:17.975582 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:17.977106 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:17.979527 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:17.982863 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:17.983114 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:17.983608 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:17.983858 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.047849 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:38635
I20260709 13:16:18.047974 32028 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:38635 every 8 connection(s)
I20260709 13:16:18.055209 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:18.066017 32032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.067836 32033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.069631 32035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.072499 21636 server_base.cc:1061] running on GCE node
I20260709 13:16:18.073238 32029 heartbeater.cc:344] Connected to a master server at 127.21.33.62:43497
I20260709 13:16:18.073719 32029 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:18.073943 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:18.074221 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:18.074435 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783602978074412 us; error 0 us; skew 500 ppm
I20260709 13:16:18.074774 32029 heartbeater.cc:507] Master 127.21.33.62:43497 requested a full tablet report, sending...
I20260709 13:16:18.075191 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:18.078636 31753 ts_manager.cc:194] Registered new tserver with Master: ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:18.079702 21636 webserver.cc:533] Webserver started at http://127.21.33.4:44241/ using document root <none> and password file <none>
I20260709 13:16:18.080734 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:18.081063 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:18.081344 31753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34428
I20260709 13:16:18.081470 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:18.084147 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-3-root/instance:
uuid: "32690b3987094d51bb4f214321a29067"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-p8rg"
I20260709 13:16:18.090652 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:16:18.095598 32040 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.096568 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:16:18.096985 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-3-root
uuid: "32690b3987094d51bb4f214321a29067"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-p8rg"
I20260709 13:16:18.097308 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:18.118701 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:18.120227 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:18.122161 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:18.125267 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:18.125630 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:16:18.125962 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:18.126164 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.187280 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.4:34551
I20260709 13:16:18.187649 32102 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.4:34551 every 8 connection(s)
I20260709 13:16:18.208344 32103 heartbeater.cc:344] Connected to a master server at 127.21.33.62:43497
I20260709 13:16:18.208861 32103 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:18.210132 32103 heartbeater.cc:507] Master 127.21.33.62:43497 requested a full tablet report, sending...
I20260709 13:16:18.213429 31753 ts_manager.cc:194] Registered new tserver with Master: 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:18.213699 21636 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.021681788s
I20260709 13:16:18.215227 31753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34432
I20260709 13:16:18.253366 31753 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34448:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:18.592724 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:18.593286 32068 tablet_service.cc:1511] Processing CreateTablet for tablet 221d9ca0b1f9450ead6ef9e225fe17ed (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:18.595407 32068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 221d9ca0b1f9450ead6ef9e225fe17ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.599740 32064 tablet_service.cc:1511] Processing CreateTablet for tablet be826572f20c45fe84142ea4f204ba26 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:18.601168 31844 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4c038e4f034907a5fcaf6ab632a887 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:18.602603 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be826572f20c45fe84142ea4f204ba26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.604034 31844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4c038e4f034907a5fcaf6ab632a887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.603946 32065 tablet_service.cc:1511] Processing CreateTablet for tablet bfc3d708d55c46bcb69cfb239324f375 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:18.601684 31845 tablet_service.cc:1511] Processing CreateTablet for tablet 221d9ca0b1f9450ead6ef9e225fe17ed (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:18.606977 32066 tablet_service.cc:1511] Processing CreateTablet for tablet 340107361dee4b3997a43799863f36e8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:18.608290 31845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 221d9ca0b1f9450ead6ef9e225fe17ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.610281 32067 tablet_service.cc:1511] Processing CreateTablet for tablet 8e16bbde8ad04886af986347646976b7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:18.615047 31842 tablet_service.cc:1511] Processing CreateTablet for tablet 58ea2a2bfc574918bd4f2bf7f2839e9c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:18.616910 31842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ea2a2bfc574918bd4f2bf7f2839e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.616797 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc3d708d55c46bcb69cfb239324f375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.619792 32067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e16bbde8ad04886af986347646976b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.620378 31840 tablet_service.cc:1511] Processing CreateTablet for tablet be826572f20c45fe84142ea4f204ba26 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:18.623251 31841 tablet_service.cc:1511] Processing CreateTablet for tablet bfc3d708d55c46bcb69cfb239324f375 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:18.626065 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340107361dee4b3997a43799863f36e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.625439 31843 tablet_service.cc:1511] Processing CreateTablet for tablet 8e16bbde8ad04886af986347646976b7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:18.627758 31840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be826572f20c45fe84142ea4f204ba26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.630216 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc3d708d55c46bcb69cfb239324f375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.633939 31843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e16bbde8ad04886af986347646976b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.639119 32063 tablet_service.cc:1511] Processing CreateTablet for tablet 4df0e0908b634f91a377721d9f0e3ef7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:18.641197 32063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df0e0908b634f91a377721d9f0e3ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.641871 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 79ebfc33b37b45f69df96a3f4bb8f422 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:18.643682 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ebfc33b37b45f69df96a3f4bb8f422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.665622 31920 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4c038e4f034907a5fcaf6ab632a887 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:18.667500 31920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4c038e4f034907a5fcaf6ab632a887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.667697 32123 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:18.671015 31919 tablet_service.cc:1511] Processing CreateTablet for tablet 8e16bbde8ad04886af986347646976b7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:18.672744 31919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e16bbde8ad04886af986347646976b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.675019 31918 tablet_service.cc:1511] Processing CreateTablet for tablet 340107361dee4b3997a43799863f36e8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:18.676585 31918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340107361dee4b3997a43799863f36e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.679165 31917 tablet_service.cc:1511] Processing CreateTablet for tablet 58ea2a2bfc574918bd4f2bf7f2839e9c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:18.681102 31917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ea2a2bfc574918bd4f2bf7f2839e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.682940 32064 tablet_service.cc:1511] Processing CreateTablet for tablet a2ce179ed2a94df48e1b5f00fec58804 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:18.683112 31916 tablet_service.cc:1511] Processing CreateTablet for tablet bfc3d708d55c46bcb69cfb239324f375 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:18.684525 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ce179ed2a94df48e1b5f00fec58804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.685804 32123 tablet_bootstrap.cc:654] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.686015 31915 tablet_service.cc:1511] Processing CreateTablet for tablet 4df0e0908b634f91a377721d9f0e3ef7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:18.689009 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 79ebfc33b37b45f69df96a3f4bb8f422 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:18.693796 31838 tablet_service.cc:1511] Processing CreateTablet for tablet a2ce179ed2a94df48e1b5f00fec58804 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:18.695531 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ce179ed2a94df48e1b5f00fec58804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.697046 31913 tablet_service.cc:1511] Processing CreateTablet for tablet a2ce179ed2a94df48e1b5f00fec58804 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:18.698765 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2ce179ed2a94df48e1b5f00fec58804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.702005 31916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc3d708d55c46bcb69cfb239324f375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.704936 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df0e0908b634f91a377721d9f0e3ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.704682 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 4b0b712922784fd8a86a1c486c9d372e (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:18.706202 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ebfc33b37b45f69df96a3f4bb8f422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.713606 31837 tablet_service.cc:1511] Processing CreateTablet for tablet 4b0b712922784fd8a86a1c486c9d372e (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:18.715996 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0b712922784fd8a86a1c486c9d372e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.725172 31994 tablet_service.cc:1511] Processing CreateTablet for tablet 221d9ca0b1f9450ead6ef9e225fe17ed (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:18.726886 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 221d9ca0b1f9450ead6ef9e225fe17ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.730997 31993 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4c038e4f034907a5fcaf6ab632a887 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:18.732697 31993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4c038e4f034907a5fcaf6ab632a887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.734982 31992 tablet_service.cc:1511] Processing CreateTablet for tablet 340107361dee4b3997a43799863f36e8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:18.736671 31992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 340107361dee4b3997a43799863f36e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.738832 31991 tablet_service.cc:1511] Processing CreateTablet for tablet 58ea2a2bfc574918bd4f2bf7f2839e9c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:18.762652 31988 tablet_service.cc:1511] Processing CreateTablet for tablet 79ebfc33b37b45f69df96a3f4bb8f422 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:18.766278 31987 tablet_service.cc:1511] Processing CreateTablet for tablet 4b0b712922784fd8a86a1c486c9d372e (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:18.762986 31912 tablet_service.cc:1511] Processing CreateTablet for tablet 973ae55c06694504ba369b35b010b993 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:18.773708 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973ae55c06694504ba369b35b010b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.748848 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0b712922784fd8a86a1c486c9d372e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.782867 31989 tablet_service.cc:1511] Processing CreateTablet for tablet 4df0e0908b634f91a377721d9f0e3ef7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:18.784127 31990 tablet_service.cc:1511] Processing CreateTablet for tablet be826572f20c45fe84142ea4f204ba26 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:18.790315 31991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ea2a2bfc574918bd4f2bf7f2839e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.792583 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be826572f20c45fe84142ea4f204ba26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.794350 31987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0b712922784fd8a86a1c486c9d372e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.798237 31988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ebfc33b37b45f69df96a3f4bb8f422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.807363 31989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df0e0908b634f91a377721d9f0e3ef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.819485 31880 heartbeater.cc:499] Master 127.21.33.62:43497 was elected leader, sending a full tablet report...
I20260709 13:16:18.838125 32125 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:18.840752 31986 tablet_service.cc:1511] Processing CreateTablet for tablet 973ae55c06694504ba369b35b010b993 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:18.843089 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973ae55c06694504ba369b35b010b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.843423 31839 tablet_service.cc:1511] Processing CreateTablet for tablet a930e148e8204d8db397cb765fbef4ef (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:18.844995 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a930e148e8204d8db397cb765fbef4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.848742 32066 tablet_service.cc:1511] Processing CreateTablet for tablet 973ae55c06694504ba369b35b010b993 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:18.859478 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973ae55c06694504ba369b35b010b993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.895731 32123 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:18.896435 32123 ts_tablet_manager.cc:1403] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.229s	user 0.019s	sys 0.027s
W20260709 13:16:18.905941 31737 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 59580.2us
I20260709 13:16:18.932117 31985 tablet_service.cc:1511] Processing CreateTablet for tablet a930e148e8204d8db397cb765fbef4ef (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:18.933952 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a930e148e8204d8db397cb765fbef4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.934561 31844 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfe802c323741a9b432e1eca10b2698 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:18.936038 31844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfe802c323741a9b432e1eca10b2698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.949903 31955 heartbeater.cc:499] Master 127.21.33.62:43497 was elected leader, sending a full tablet report...
I20260709 13:16:18.936162 32123 raft_consensus.cc:359] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:18.954991 32123 raft_consensus.cc:385] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.955452 32123 raft_consensus.cc:740] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.939383 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfe802c323741a9b432e1eca10b2698 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:18.956516 32123 consensus_queue.cc:260] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:18.957870 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfe802c323741a9b432e1eca10b2698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.961076 32125 tablet_bootstrap.cc:654] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.946869 31918 tablet_service.cc:1511] Processing CreateTablet for tablet a930e148e8204d8db397cb765fbef4ef (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:18.962436 31918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a930e148e8204d8db397cb765fbef4ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.989274 32066 tablet_service.cc:1511] Processing CreateTablet for tablet a6aaf6fc3e504bc79c75c64bf99d89e7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:18.991384 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6aaf6fc3e504bc79c75c64bf99d89e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.992034 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfe802c323741a9b432e1eca10b2698 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:18.995193 31912 tablet_service.cc:1511] Processing CreateTablet for tablet 039ecd5ff518405fb0fc382c46e7e7c4 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:18.994570 31837 tablet_service.cc:1511] Processing CreateTablet for tablet a6aaf6fc3e504bc79c75c64bf99d89e7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:19.003017 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6aaf6fc3e504bc79c75c64bf99d89e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.003953 31986 tablet_service.cc:1511] Processing CreateTablet for tablet 039ecd5ff518405fb0fc382c46e7e7c4 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:19.005555 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 039ecd5ff518405fb0fc382c46e7e7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.993464 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfe802c323741a9b432e1eca10b2698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.996138 31994 tablet_service.cc:1511] Processing CreateTablet for tablet a6aaf6fc3e504bc79c75c64bf99d89e7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:19.010087 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 039ecd5ff518405fb0fc382c46e7e7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.019264 31840 tablet_service.cc:1511] Processing CreateTablet for tablet 125eaf8f67bd4727b6d49528c95ca751 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:19.021014 31840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125eaf8f67bd4727b6d49528c95ca751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.014029 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 039ecd5ff518405fb0fc382c46e7e7c4 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:19.022786 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 039ecd5ff518405fb0fc382c46e7e7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.033749 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6aaf6fc3e504bc79c75c64bf99d89e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.035158 31915 tablet_service.cc:1511] Processing CreateTablet for tablet 125eaf8f67bd4727b6d49528c95ca751 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:19.039505 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125eaf8f67bd4727b6d49528c95ca751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.040891 32123 ts_tablet_manager.cc:1434] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.144s	user 0.013s	sys 0.002s
I20260709 13:16:19.049710 32126 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:19.049911 32103 heartbeater.cc:499] Master 127.21.33.62:43497 was elected leader, sending a full tablet report...
I20260709 13:16:19.058184 32123 tablet_bootstrap.cc:492] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:19.068441 32068 tablet_service.cc:1511] Processing CreateTablet for tablet 395cb97d328c4ccc9cb0b96fe5d83dd6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:19.070097 32068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395cb97d328c4ccc9cb0b96fe5d83dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.073742 32127 tablet_bootstrap.cc:492] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:19.062641 31991 tablet_service.cc:1511] Processing CreateTablet for tablet 125eaf8f67bd4727b6d49528c95ca751 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:19.068317 32125 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:19.081308 32125 ts_tablet_manager.cc:1403] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.252s	user 0.014s	sys 0.019s
I20260709 13:16:19.081441 31991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125eaf8f67bd4727b6d49528c95ca751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.083407 32126 tablet_bootstrap.cc:654] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.073288 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 395cb97d328c4ccc9cb0b96fe5d83dd6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:19.086102 32029 heartbeater.cc:499] Master 127.21.33.62:43497 was elected leader, sending a full tablet report...
I20260709 13:16:19.087060 31913 tablet_service.cc:1511] Processing CreateTablet for tablet 395cb97d328c4ccc9cb0b96fe5d83dd6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:19.089888 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395cb97d328c4ccc9cb0b96fe5d83dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.090739 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395cb97d328c4ccc9cb0b96fe5d83dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.084941 32125 raft_consensus.cc:359] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:19.121658 32125 raft_consensus.cc:385] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.124470 32123 tablet_bootstrap.cc:654] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.128363 31985 tablet_service.cc:1511] Processing CreateTablet for tablet 77e32837dc3e4e7d99a94d477a4d50d1 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:19.129930 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77e32837dc3e4e7d99a94d477a4d50d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.131413 32127 tablet_bootstrap.cc:654] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.122083 32125 raft_consensus.cc:740] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.145242 32125 consensus_queue.cc:260] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:19.157836 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 77e32837dc3e4e7d99a94d477a4d50d1 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:19.159403 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77e32837dc3e4e7d99a94d477a4d50d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.161787 32126 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:19.162659 32126 ts_tablet_manager.cc:1403] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.129s	user 0.019s	sys 0.000s
I20260709 13:16:19.184804 32127 tablet_bootstrap.cc:492] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:19.185631 32127 ts_tablet_manager.cc:1403] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.155s	user 0.020s	sys 0.039s
I20260709 13:16:19.187592 31840 tablet_service.cc:1511] Processing CreateTablet for tablet 77e32837dc3e4e7d99a94d477a4d50d1 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:19.189137 31840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77e32837dc3e4e7d99a94d477a4d50d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.189474 32126 raft_consensus.cc:359] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:19.193282 32126 raft_consensus.cc:385] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.193708 32126 raft_consensus.cc:740] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.190388 31985 tablet_service.cc:1511] Processing CreateTablet for tablet 62477156fe044dad9635981556678bef (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:19.192344 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 62477156fe044dad9635981556678bef (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:19.195130 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62477156fe044dad9635981556678bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.194749 32126 consensus_queue.cc:260] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:19.191516 32127 raft_consensus.cc:359] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.196226 32127 raft_consensus.cc:385] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.196555 32127 raft_consensus.cc:740] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.197451 31913 tablet_service.cc:1511] Processing CreateTablet for tablet 62477156fe044dad9635981556678bef (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:19.197600 32127 consensus_queue.cc:260] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:19.193781 32104 tablet.cc:2406] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:19.202034 31917 tablet_service.cc:1511] Processing CreateTablet for tablet 6247a65da5884caba020ed13be64352f (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:19.199653 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62477156fe044dad9635981556678bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.207752 31837 tablet_service.cc:1511] Processing CreateTablet for tablet 6247a65da5884caba020ed13be64352f (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:19.211338 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6247a65da5884caba020ed13be64352f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.214116 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62477156fe044dad9635981556678bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.218058 31917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6247a65da5884caba020ed13be64352f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.258496 31991 tablet_service.cc:1511] Processing CreateTablet for tablet 6247a65da5884caba020ed13be64352f (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:19.260214 32062 tablet_service.cc:1511] Processing CreateTablet for tablet d3127b63807542e6bb6a2a33677baabc (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:19.260569 31991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6247a65da5884caba020ed13be64352f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.260516 32126 ts_tablet_manager.cc:1434] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.097s	user 0.017s	sys 0.039s
I20260709 13:16:19.265021 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3127b63807542e6bb6a2a33677baabc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.275098 32123 tablet_bootstrap.cc:492] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:19.275907 32123 ts_tablet_manager.cc:1403] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.218s	user 0.018s	sys 0.007s
I20260709 13:16:19.277196 32127 ts_tablet_manager.cc:1434] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.091s	user 0.009s	sys 0.007s
I20260709 13:16:19.268968 31844 tablet_service.cc:1511] Processing CreateTablet for tablet d3127b63807542e6bb6a2a33677baabc (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:19.266288 32126 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:19.281016 31844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3127b63807542e6bb6a2a33677baabc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.281062 32125 ts_tablet_manager.cc:1434] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.199s	user 0.023s	sys 0.015s
I20260709 13:16:19.282274 31914 tablet_service.cc:1511] Processing CreateTablet for tablet d3127b63807542e6bb6a2a33677baabc (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:19.286971 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3127b63807542e6bb6a2a33677baabc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.297080 32126 tablet_bootstrap.cc:654] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.304262 32127 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:19.304528 32125 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:19.311656 31994 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad8727419f947cc957c6638ca5fd93a (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:19.329128 32068 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad8727419f947cc957c6638ca5fd93a (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:19.330552 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad8727419f947cc957c6638ca5fd93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.327976 32123 raft_consensus.cc:359] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.331095 32123 raft_consensus.cc:385] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.331431 32123 raft_consensus.cc:740] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.332592 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad8727419f947cc957c6638ca5fd93a (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:19.334093 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad8727419f947cc957c6638ca5fd93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.339470 32127 tablet_bootstrap.cc:654] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.341907 32123 consensus_queue.cc:260] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.344909 32123 ts_tablet_manager.cc:1434] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.003s
I20260709 13:16:19.346117 32123 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:19.330566 32068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad8727419f947cc957c6638ca5fd93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.351914 31913 tablet_service.cc:1511] Processing CreateTablet for tablet ab0ae780de5843cea623c39fd867cc57 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:19.353415 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab0ae780de5843cea623c39fd867cc57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.355683 31985 tablet_service.cc:1511] Processing CreateTablet for tablet ab0ae780de5843cea623c39fd867cc57 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:19.357189 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab0ae780de5843cea623c39fd867cc57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.361812 32125 tablet_bootstrap.cc:654] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.365185 32123 tablet_bootstrap.cc:654] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.370810 32127 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:19.371428 32127 ts_tablet_manager.cc:1403] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.005s
I20260709 13:16:19.374686 32065 tablet_service.cc:1511] Processing CreateTablet for tablet ab0ae780de5843cea623c39fd867cc57 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:19.376188 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab0ae780de5843cea623c39fd867cc57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.377698 32126 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:19.378360 32126 ts_tablet_manager.cc:1403] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.001s
I20260709 13:16:19.378720 31840 tablet_service.cc:1511] Processing CreateTablet for tablet dea199ae81974f50bda37760537988a5 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:19.380208 31840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea199ae81974f50bda37760537988a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.382687 32126 raft_consensus.cc:359] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.385201 31914 tablet_service.cc:1511] Processing CreateTablet for tablet dea199ae81974f50bda37760537988a5 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:19.387889 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea199ae81974f50bda37760537988a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.391434 32127 raft_consensus.cc:359] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.392329 32127 raft_consensus.cc:385] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.392635 32127 raft_consensus.cc:740] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.393658 32127 consensus_queue.cc:260] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.398393 32126 raft_consensus.cc:385] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.400415 32126 raft_consensus.cc:740] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.403539 32126 consensus_queue.cc:260] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.403635 31986 tablet_service.cc:1511] Processing CreateTablet for tablet dea199ae81974f50bda37760537988a5 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:19.407958 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea199ae81974f50bda37760537988a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.409225 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 75f88110f6d141dc97dbef5c1c4f8391 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:19.410800 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f88110f6d141dc97dbef5c1c4f8391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.414593 32127 ts_tablet_manager.cc:1434] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:16:19.415692 32126 ts_tablet_manager.cc:1434] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:16:19.416898 32123 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:19.417557 32123 ts_tablet_manager.cc:1403] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.000s
I20260709 13:16:19.420094 31917 tablet_service.cc:1511] Processing CreateTablet for tablet 75f88110f6d141dc97dbef5c1c4f8391 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:19.421651 31917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f88110f6d141dc97dbef5c1c4f8391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.421288 32123 raft_consensus.cc:359] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:19.422153 32123 raft_consensus.cc:385] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.422549 32123 raft_consensus.cc:740] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.422222 31990 tablet_service.cc:1511] Processing CreateTablet for tablet 2611ce5875fe4e55a870d70749bd1ec4 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:19.423712 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2611ce5875fe4e55a870d70749bd1ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.423400 32123 consensus_queue.cc:260] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:19.431106 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 2611ce5875fe4e55a870d70749bd1ec4 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:19.416925 32126 tablet_bootstrap.cc:492] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:19.435303 32123 ts_tablet_manager.cc:1434] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:16:19.427448 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 75f88110f6d141dc97dbef5c1c4f8391 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:19.447098 32128 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.449729 32128 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.452374 32123 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:19.448835 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f88110f6d141dc97dbef5c1c4f8391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.461795 32128 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:19.457612 32127 tablet_bootstrap.cc:492] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:19.465553 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2611ce5875fe4e55a870d70749bd1ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.498780 32135 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.499541 32135 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.500165 31912 tablet_service.cc:1511] Processing CreateTablet for tablet 372dd26a25ab4cf686e6c74fcc9b6bb6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:19.501691 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372dd26a25ab4cf686e6c74fcc9b6bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.511081 32125 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:19.499197 31844 tablet_service.cc:1511] Processing CreateTablet for tablet 2611ce5875fe4e55a870d70749bd1ec4 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:19.513988 32128 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.523018 32135 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:19.523417 31991 tablet_service.cc:1511] Processing CreateTablet for tablet 372dd26a25ab4cf686e6c74fcc9b6bb6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:19.529163 32125 ts_tablet_manager.cc:1403] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.225s	user 0.024s	sys 0.012s
I20260709 13:16:19.528853 32128 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.539824 31844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2611ce5875fe4e55a870d70749bd1ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.540454 31991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372dd26a25ab4cf686e6c74fcc9b6bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.548979 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 08f4aafcb3434b9badfaffebc83bd15c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:19.550927 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08f4aafcb3434b9badfaffebc83bd15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.553757 31985 tablet_service.cc:1511] Processing CreateTablet for tablet 08f4aafcb3434b9badfaffebc83bd15c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:19.555280 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08f4aafcb3434b9badfaffebc83bd15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.557346 32128 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:19.558125 32128 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.539932 32066 tablet_service.cc:1511] Processing CreateTablet for tablet 372dd26a25ab4cf686e6c74fcc9b6bb6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:19.558640 32128 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:19.560081 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372dd26a25ab4cf686e6c74fcc9b6bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.561100 32128 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:19.562520 32134 raft_consensus.cc:493] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.563086 32134 raft_consensus.cc:515] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:19.566093 32134 leader_election.cc:290] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:19.597263 32123 tablet_bootstrap.cc:654] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.599293 32135 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.597407 32125 raft_consensus.cc:359] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:19.616848 32125 raft_consensus.cc:385] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.617249 32125 raft_consensus.cc:740] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.618309 32125 consensus_queue.cc:260] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:19.626557 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:19.626767 31913 tablet_service.cc:1511] Processing CreateTablet for tablet 809ff13e62914dd7854fa320a1173da0 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:19.628808 32135 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.629858 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 809ff13e62914dd7854fa320a1173da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.659919 32133 raft_consensus.cc:493] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.660658 32133 raft_consensus.cc:515] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:19.661039 31837 tablet_service.cc:1511] Processing CreateTablet for tablet 08f4aafcb3434b9badfaffebc83bd15c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:19.662744 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08f4aafcb3434b9badfaffebc83bd15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.664045 32133 leader_election.cc:290] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:19.680588 31838 tablet_service.cc:1511] Processing CreateTablet for tablet 809ff13e62914dd7854fa320a1173da0 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:19.682610 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 809ff13e62914dd7854fa320a1173da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.681039 31915 tablet_service.cc:1511] Processing CreateTablet for tablet 658f92ce9d684ffbbec92446706f44c9 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:19.687372 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 658f92ce9d684ffbbec92446706f44c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.694221 32126 tablet_bootstrap.cc:654] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.695482 31841 tablet_service.cc:1511] Processing CreateTablet for tablet fb6e2830158d41d3bb09aeaad85a5f84 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:19.697463 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb6e2830158d41d3bb09aeaad85a5f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.697697 32127 tablet_bootstrap.cc:700] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: Time spent opening tablet: real 0.151s	user 0.001s	sys 0.003s
I20260709 13:16:19.698655 32127 tablet_bootstrap.cc:654] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.714462 32135 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:19.735896 32125 ts_tablet_manager.cc:1434] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.202s	user 0.004s	sys 0.007s
I20260709 13:16:19.754199 32125 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:19.787021 31986 tablet_service.cc:1511] Processing CreateTablet for tablet fb6e2830158d41d3bb09aeaad85a5f84 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:19.788771 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb6e2830158d41d3bb09aeaad85a5f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.791309 31994 tablet_service.cc:1511] Processing CreateTablet for tablet 658f92ce9d684ffbbec92446706f44c9 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:19.792728 32125 tablet_bootstrap.cc:654] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.793095 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 658f92ce9d684ffbbec92446706f44c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.798682 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 658f92ce9d684ffbbec92446706f44c9 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:19.800278 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 658f92ce9d684ffbbec92446706f44c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.799898 32062 tablet_service.cc:1511] Processing CreateTablet for tablet fb6e2830158d41d3bb09aeaad85a5f84 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:19.803643 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb6e2830158d41d3bb09aeaad85a5f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.806069 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:19.806701 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:19.807540 32003 raft_consensus.cc:2468] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:19.806123 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:19.810986 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:19.811833 32134 raft_consensus.cc:493] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.812410 32134 raft_consensus.cc:515] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:19.813619 31915 tablet_service.cc:1511] Processing CreateTablet for tablet c49fb2266a654cc9bdc77936ae8f38fe (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:19.814596 32134 leader_election.cc:290] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:19.815091 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49fb2266a654cc9bdc77936ae8f38fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.814911 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:19.815619 31969 leader_election.cc:343] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.817203 32123 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
W20260709 13:16:19.817270 31967 leader_election.cc:343] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.818102 32123 ts_tablet_manager.cc:1403] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.372s	user 0.027s	sys 0.006s
I20260709 13:16:19.815646 32078 raft_consensus.cc:2468] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:19.817950 31967 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:19.819242 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:19.820338 32135 raft_consensus.cc:2749] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.821389 31969 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:19.821655 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:19.822480 32135 raft_consensus.cc:2804] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.822886 32135 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.823246 32135 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:19.823873 31821 leader_election.cc:343] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:19.825053 31821 leader_election.cc:343] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.827057 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:19.827127 32044 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:19.828563 32148 raft_consensus.cc:2804] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:19.829007 32148 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.829550 32148 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:19.839704 31841 tablet_service.cc:1511] Processing CreateTablet for tablet c49fb2266a654cc9bdc77936ae8f38fe (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 13:16:19.841995 32044 leader_election.cc:343] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.842597 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49fb2266a654cc9bdc77936ae8f38fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.843364 32126 tablet_bootstrap.cc:492] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:19.843966 32126 ts_tablet_manager.cc:1403] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.427s	user 0.015s	sys 0.005s
I20260709 13:16:19.844429 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 809ff13e62914dd7854fa320a1173da0 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:19.840744 32148 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.856380 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:19.857659 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 809ff13e62914dd7854fa320a1173da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.858165 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:19.857414 32148 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:19.904713 32135 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.906241 32004 raft_consensus.cc:2393] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:19.907172 32123 raft_consensus.cc:359] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.911034 32133 raft_consensus.cc:493] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.910310 32066 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc5e72ae63b4c96bc88199a467670e2 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260709 13:16:19.911087 32041 leader_election.cc:343] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.912745 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:19.922638 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:19.911705 31985 tablet_service.cc:1511] Processing CreateTablet for tablet c49fb2266a654cc9bdc77936ae8f38fe (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:19.913427 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:19.914381 31844 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc5e72ae63b4c96bc88199a467670e2 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:19.921034 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:19.920976 32133 raft_consensus.cc:515] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.922788 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc5e72ae63b4c96bc88199a467670e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.928753 32133 leader_election.cc:290] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:16:19.929003 31819 leader_election.cc:343] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.927306 31844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc5e72ae63b4c96bc88199a467670e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.929675 31819 leader_election.cc:304] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:19.931013 32134 raft_consensus.cc:2749] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.932472 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49fb2266a654cc9bdc77936ae8f38fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.927935 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:19.915318 32123 raft_consensus.cc:385] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:16:19.935582 32042 leader_election.cc:343] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.925042 32127 tablet_bootstrap.cc:492] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:19.946384 32042 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
W20260709 13:16:19.943077 31819 leader_election.cc:343] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.948045 32148 raft_consensus.cc:2749] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.947727 31819 leader_election.cc:304] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:19.950322 32134 raft_consensus.cc:2749] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:19.952699 32123 raft_consensus.cc:740] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.953924 32123 consensus_queue.cc:260] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:19.944829 32126 raft_consensus.cc:359] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.946863 32127 ts_tablet_manager.cc:1403] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.503s	user 0.020s	sys 0.019s
I20260709 13:16:19.968606 32126 raft_consensus.cc:385] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.969120 32126 raft_consensus.cc:740] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.941788 32041 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:19.971885 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc5e72ae63b4c96bc88199a467670e2 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:19.979104 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc5e72ae63b4c96bc88199a467670e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.983832 32148 raft_consensus.cc:2749] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:19.983626 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 119407us
W20260709 13:16:19.999655 32041 leader_election.cc:343] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.999835 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:19.986249 32127 raft_consensus.cc:359] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:19.986445 31990 tablet_service.cc:1511] Processing CreateTablet for tablet 2b97d68a6e314f6792e4c8bfa99d3025 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:20.001744 32127 raft_consensus.cc:385] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.003506 32127 raft_consensus.cc:740] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
W20260709 13:16:19.983579 32042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 48201.2us
W20260709 13:16:19.993232 31819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 50205.9us
I20260709 13:16:19.994465 32135 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.007602 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:20.008805 32076 raft_consensus.cc:2393] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:19.983687 32126 consensus_queue.cc:260] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.002058 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e16bbde8ad04886af986347646976b7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:20.009992 32077 raft_consensus.cc:2468] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:20.002399 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:19.989012 32125 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
W20260709 13:16:20.002784 31968 leader_election.cc:343] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.013284 32125 ts_tablet_manager.cc:1403] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.259s	user 0.018s	sys 0.016s
I20260709 13:16:20.002954 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b97d68a6e314f6792e4c8bfa99d3025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.001331 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:20.018345 31895 leader_election.cc:304] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:20.004420 32127 consensus_queue.cc:260] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:20.019742 31895 leader_election.cc:343] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.020736 32133 raft_consensus.cc:2804] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.020854 32127 ts_tablet_manager.cc:1434] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.006s
I20260709 13:16:20.021312 32133 raft_consensus.cc:493] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.021806 32133 raft_consensus.cc:3060] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.017385 32125 raft_consensus.cc:359] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:20.021878 31968 leader_election.cc:343] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.022346 32125 raft_consensus.cc:385] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.022704 32125 raft_consensus.cc:740] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.024014 32127 tablet_bootstrap.cc:492] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:20.023912 32125 consensus_queue.cc:260] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.032017 32125 ts_tablet_manager.cc:1434] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:16:20.037344 31837 tablet_service.cc:1511] Processing CreateTablet for tablet 2b97d68a6e314f6792e4c8bfa99d3025 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:20.037550 31912 tablet_service.cc:1511] Processing CreateTablet for tablet 716630c6a4e249aeb96ef3ac276290c8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260709 13:16:20.038305 32042 leader_election.cc:343] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.038936 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b97d68a6e314f6792e4c8bfa99d3025. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.039459 32042 leader_election.cc:343] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.039330 32066 tablet_service.cc:1511] Processing CreateTablet for tablet 2b97d68a6e314f6792e4c8bfa99d3025 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:20.038960 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 716630c6a4e249aeb96ef3ac276290c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.039951 32042 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:20.040983 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b97d68a6e314f6792e4c8bfa99d3025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.045701 32123 ts_tablet_manager.cc:1434] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.227s	user 0.015s	sys 0.001s
I20260709 13:16:20.034570 32133 raft_consensus.cc:515] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.049365 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e16bbde8ad04886af986347646976b7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:20.050119 32078 raft_consensus.cc:3060] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.050647 32123 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:20.055896 31986 tablet_service.cc:1511] Processing CreateTablet for tablet 716630c6a4e249aeb96ef3ac276290c8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:20.057372 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 716630c6a4e249aeb96ef3ac276290c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.062635 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:20.064888 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e16bbde8ad04886af986347646976b7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:20.065860 31854 raft_consensus.cc:2468] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
W20260709 13:16:20.068060 31893 leader_election.cc:343] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.068585 31893 leader_election.cc:304] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:20.070039 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e16bbde8ad04886af986347646976b7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:20.071401 31854 raft_consensus.cc:3060] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.075708 32127 tablet_bootstrap.cc:654] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.075754 32126 ts_tablet_manager.cc:1434] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.231s	user 0.007s	sys 0.011s
I20260709 13:16:20.077379 32133 leader_election.cc:290] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:20.084573 31854 raft_consensus.cc:2468] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:20.085197 32126 tablet_bootstrap.cc:492] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:20.080773 31841 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd8ceaec471453d9530212df301bc15 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:20.086414 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd8ceaec471453d9530212df301bc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.086186 31893 leader_election.cc:304] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:20.087877 32163 raft_consensus.cc:2804] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.088174 32125 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:20.088459 32163 raft_consensus.cc:697] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:20.090066 32163 consensus_queue.cc:237] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.092495 32078 raft_consensus.cc:2468] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:20.082647 31915 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd8ceaec471453d9530212df301bc15 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:20.098789 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd8ceaec471453d9530212df301bc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.079713 32164 raft_consensus.cc:2749] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.098712 31994 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd8ceaec471453d9530212df301bc15 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:20.099717 32148 raft_consensus.cc:2749] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:20.099964 32042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 60548.9us
I20260709 13:16:20.108623 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd8ceaec471453d9530212df301bc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.109149 31969 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:20.111481 32135 raft_consensus.cc:2749] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.112483 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 716630c6a4e249aeb96ef3ac276290c8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:20.114360 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 716630c6a4e249aeb96ef3ac276290c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.125538 32126 tablet_bootstrap.cc:654] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.131191 32123 tablet_bootstrap.cc:654] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.133239 32125 tablet_bootstrap.cc:654] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.142714 32064 tablet_service.cc:1511] Processing CreateTablet for tablet b90a6b486bef42019735b76ff4da04a1 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:20.144301 31838 tablet_service.cc:1511] Processing CreateTablet for tablet b90a6b486bef42019735b76ff4da04a1 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:20.146199 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90a6b486bef42019735b76ff4da04a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.144937 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90a6b486bef42019735b76ff4da04a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.163015 32127 tablet_bootstrap.cc:492] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:20.164196 32127 ts_tablet_manager.cc:1403] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.141s	user 0.008s	sys 0.012s
I20260709 13:16:20.176265 31912 tablet_service.cc:1511] Processing CreateTablet for tablet b90a6b486bef42019735b76ff4da04a1 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:20.178005 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90a6b486bef42019735b76ff4da04a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.163100 32126 tablet_bootstrap.cc:492] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:20.191175 32126 ts_tablet_manager.cc:1403] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.010s
I20260709 13:16:20.193922 31990 tablet_service.cc:1511] Processing CreateTablet for tablet 14858ce2d22c4dba9d14295ca6c2b23c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:20.195499 32123 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:20.196411 32123 ts_tablet_manager.cc:1403] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.146s	user 0.016s	sys 0.008s
I20260709 13:16:20.195571 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14858ce2d22c4dba9d14295ca6c2b23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.197216 32127 raft_consensus.cc:359] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.198186 32127 raft_consensus.cc:385] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.202181 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 14858ce2d22c4dba9d14295ca6c2b23c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:20.206544 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14858ce2d22c4dba9d14295ca6c2b23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.208479 32123 raft_consensus.cc:359] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:20.204627 32127 raft_consensus.cc:740] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.215747 31841 tablet_service.cc:1511] Processing CreateTablet for tablet 14858ce2d22c4dba9d14295ca6c2b23c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:20.217465 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14858ce2d22c4dba9d14295ca6c2b23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.212380 32123 raft_consensus.cc:385] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.217839 32125 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:20.218120 32123 raft_consensus.cc:740] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.222414 31986 tablet_service.cc:1511] Processing CreateTablet for tablet 4c11d6a5d0564e90b9ceda978d9225cb (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:20.223451 31915 tablet_service.cc:1511] Processing CreateTablet for tablet 4c11d6a5d0564e90b9ceda978d9225cb (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:20.224112 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c11d6a5d0564e90b9ceda978d9225cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.218575 32125 ts_tablet_manager.cc:1403] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.185s	user 0.012s	sys 0.012s
I20260709 13:16:20.224689 32123 consensus_queue.cc:260] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:20.228123 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c11d6a5d0564e90b9ceda978d9225cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.215870 32127 consensus_queue.cc:260] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.233600 31751 catalog_manager.cc:5697] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.235503 32127 ts_tablet_manager.cc:1434] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.004s
I20260709 13:16:20.240367 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 4c11d6a5d0564e90b9ceda978d9225cb (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:20.250514 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c11d6a5d0564e90b9ceda978d9225cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.256582 31838 tablet_service.cc:1511] Processing CreateTablet for tablet 62e37d9df96544f08314921b5ba47624 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:20.243229 32126 raft_consensus.cc:359] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.258322 32126 raft_consensus.cc:385] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.258679 32126 raft_consensus.cc:740] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.259645 32126 consensus_queue.cc:260] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.260208 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62e37d9df96544f08314921b5ba47624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.263197 32126 ts_tablet_manager.cc:1434] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.002s
I20260709 13:16:20.264453 32126 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:20.267087 32123 ts_tablet_manager.cc:1434] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.005s
I20260709 13:16:20.268962 32123 tablet_bootstrap.cc:492] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:20.241437 32127 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:20.278932 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 62e37d9df96544f08314921b5ba47624 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:20.280611 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62e37d9df96544f08314921b5ba47624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.240367 32125 raft_consensus.cc:359] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.288000 32125 raft_consensus.cc:385] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.288432 32125 raft_consensus.cc:740] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.289636 32125 consensus_queue.cc:260] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.291088 31985 tablet_service.cc:1511] Processing CreateTablet for tablet 62e37d9df96544f08314921b5ba47624 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:20.296819 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62e37d9df96544f08314921b5ba47624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.303908 31841 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4e043220524dfa8c9106bfe7690bcd (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:20.306946 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4e043220524dfa8c9106bfe7690bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.317404 31913 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4e043220524dfa8c9106bfe7690bcd (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:20.317901 32066 tablet_service.cc:1511] Processing CreateTablet for tablet c85fd3d52a714f44a3e322f4488f1a5c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:20.319155 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4e043220524dfa8c9106bfe7690bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.319267 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c85fd3d52a714f44a3e322f4488f1a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.313539 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 1b4e043220524dfa8c9106bfe7690bcd (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:20.321761 32127 tablet_bootstrap.cc:654] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.322717 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b4e043220524dfa8c9106bfe7690bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.323120 31994 tablet_service.cc:1511] Processing CreateTablet for tablet c85fd3d52a714f44a3e322f4488f1a5c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:20.324501 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c85fd3d52a714f44a3e322f4488f1a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.326759 32163 raft_consensus.cc:493] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.327086 32126 tablet_bootstrap.cc:654] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.327322 31837 tablet_service.cc:1511] Processing CreateTablet for tablet c85fd3d52a714f44a3e322f4488f1a5c (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:20.327530 32163 raft_consensus.cc:515] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.329398 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c85fd3d52a714f44a3e322f4488f1a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.352465 32134 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.356482 32134 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.360589 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:20.365720 32148 raft_consensus.cc:493] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.366736 32148 raft_consensus.cc:515] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:16:20.370462 31895 leader_election.cc:343] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.372078 32125 ts_tablet_manager.cc:1434] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.147s	user 0.009s	sys 0.001s
I20260709 13:16:20.375813 31912 tablet_service.cc:1511] Processing CreateTablet for tablet 3474410429d7478d8ba2221c7400cb01 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:20.347353 32163 leader_election.cc:290] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.379118 32148 leader_election.cc:290] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:20.379686 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:20.380610 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:20.381376 31929 raft_consensus.cc:2468] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
W20260709 13:16:20.381572 32041 leader_election.cc:343] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.382948 32042 leader_election.cc:304] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:20.384284 32148 raft_consensus.cc:2804] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.384684 32148 raft_consensus.cc:493] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.385006 32148 raft_consensus.cc:3060] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.385774 32134 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:20.390362 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:20.391260 32004 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:20.393887 31821 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:20.395932 32125 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:20.397737 32163 raft_consensus.cc:493] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.398520 32163 raft_consensus.cc:515] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.399461 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3474410429d7478d8ba2221c7400cb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.401962 32134 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.402655 32134 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.403132 32134 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.408236 32125 tablet_bootstrap.cc:654] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.436654 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 3474410429d7478d8ba2221c7400cb01 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:20.438313 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3474410429d7478d8ba2221c7400cb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.438838 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:20.439393 32135 raft_consensus.cc:493] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.440274 32135 raft_consensus.cc:515] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.442095 32123 tablet_bootstrap.cc:654] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.422021 32163 leader_election.cc:290] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.458217 31917 tablet_service.cc:1511] Processing CreateTablet for tablet e9e460a405ae485fb15869970e178caf (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:20.460539 31917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e460a405ae485fb15869970e178caf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.462598 31895 leader_election.cc:343] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.445658 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:20.483003 32126 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:20.484194 32126 ts_tablet_manager.cc:1403] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.220s	user 0.033s	sys 0.037s
I20260709 13:16:20.484797 31990 tablet_service.cc:1511] Processing CreateTablet for tablet 3474410429d7478d8ba2221c7400cb01 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:20.487421 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3474410429d7478d8ba2221c7400cb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.490474 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:20.490085 31918 tablet_service.cc:1511] Processing CreateTablet for tablet 441576026a7f44f990e9aa61c86435d6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:20.479450 31929 raft_consensus.cc:2468] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:20.493173 31918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441576026a7f44f990e9aa61c86435d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.493649 31968 leader_election.cc:304] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:20.479717 32135 leader_election.cc:290] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.526449 32127 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:20.526876 31986 tablet_service.cc:1511] Processing CreateTablet for tablet e9e460a405ae485fb15869970e178caf (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:20.527509 32127 ts_tablet_manager.cc:1403] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.286s	user 0.007s	sys 0.015s
I20260709 13:16:20.528589 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e460a405ae485fb15869970e178caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.526484 32178 raft_consensus.cc:2804] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.529781 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 441576026a7f44f990e9aa61c86435d6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:20.530426 32178 raft_consensus.cc:493] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.531052 32178 raft_consensus.cc:3060] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.531880 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441576026a7f44f990e9aa61c86435d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.551769 32148 raft_consensus.cc:515] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:20.557091 31991 tablet_service.cc:1511] Processing CreateTablet for tablet d7340289c3434c3782e3e367d4de5258 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:20.560614 31991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7340289c3434c3782e3e367d4de5258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.520048 32135 raft_consensus.cc:493] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.562032 32135 raft_consensus.cc:515] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.578975 31838 tablet_service.cc:1511] Processing CreateTablet for tablet 441576026a7f44f990e9aa61c86435d6 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:20.581066 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441576026a7f44f990e9aa61c86435d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.584177 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
W20260709 13:16:20.586485 32041 leader_election.cc:343] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.588099 32135 leader_election.cc:290] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.581233 32163 consensus_queue.cc:1048] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:16:20.605329 31841 tablet_service.cc:1511] Processing CreateTablet for tablet e9e460a405ae485fb15869970e178caf (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:20.607088 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e460a405ae485fb15869970e178caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.622299 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:20.624802 31820 leader_election.cc:343] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.626154 32148 leader_election.cc:290] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:20.608513 32123 tablet_bootstrap.cc:492] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:20.631237 32123 ts_tablet_manager.cc:1403] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.363s	user 0.026s	sys 0.032s
W20260709 13:16:20.631367 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:20.631697 32134 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:20.633646 32127 raft_consensus.cc:359] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:20.634696 32127 raft_consensus.cc:385] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.610234 32126 raft_consensus.cc:359] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:20.616690 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:20.617657 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:20.620236 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:20.639219 32126 raft_consensus.cc:385] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.635082 32127 raft_consensus.cc:740] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.654903 32126 raft_consensus.cc:740] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.646370 31929 raft_consensus.cc:3060] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.635061 32123 raft_consensus.cc:359] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.645757 32003 raft_consensus.cc:2468] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:20.658208 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
W20260709 13:16:20.665884 31740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 50228.4us
I20260709 13:16:20.666149 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:20.664075 32123 raft_consensus.cc:385] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.666913 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:20.671528 32127 consensus_queue.cc:260] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:20.672012 31896 leader_election.cc:304] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:20.667860 32064 tablet_service.cc:1511] Processing CreateTablet for tablet d7340289c3434c3782e3e367d4de5258 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:20.674010 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7340289c3434c3782e3e367d4de5258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.674376 32127 ts_tablet_manager.cc:1434] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.146s	user 0.006s	sys 0.005s
I20260709 13:16:20.675570 32127 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:20.670421 32123 raft_consensus.cc:740] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
W20260709 13:16:20.677870 31820 leader_election.cc:343] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.658161 32126 consensus_queue.cc:260] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:20.678371 31928 raft_consensus.cc:2468] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:20.679157 32123 consensus_queue.cc:260] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.679912 32163 raft_consensus.cc:2804] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.680294 32078 raft_consensus.cc:2468] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
W20260709 13:16:20.664718 31969 leader_election.cc:343] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.682983 32163 raft_consensus.cc:493] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.683462 32163 raft_consensus.cc:3060] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.697101 32183 consensus_queue.cc:1048] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:20.699389 31968 leader_election.cc:304] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:20.698029 32126 ts_tablet_manager.cc:1434] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.213s	user 0.015s	sys 0.003s
I20260709 13:16:20.701870 32126 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:20.702800 32125 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:20.703500 31837 tablet_service.cc:1511] Processing CreateTablet for tablet d7340289c3434c3782e3e367d4de5258 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:20.711747 31917 tablet_service.cc:1511] Processing CreateTablet for tablet 91b9568c210d4ef387297f17cde15389 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:20.713145 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7340289c3434c3782e3e367d4de5258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.689450 32178 raft_consensus.cc:515] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.705354 32123 ts_tablet_manager.cc:1434] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 13:16:20.713256 31917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91b9568c210d4ef387297f17cde15389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.704490 32177 raft_consensus.cc:2804] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.711836 32125 ts_tablet_manager.cc:1403] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.316s	user 0.022s	sys 0.023s
I20260709 13:16:20.731640 32134 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:20.746706 32123 tablet_bootstrap.cc:492] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:20.734061 32004 raft_consensus.cc:2393] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:20.734612 32178 leader_election.cc:290] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.736090 31994 tablet_service.cc:1511] Processing CreateTablet for tablet 91b9568c210d4ef387297f17cde15389 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:20.736598 32177 raft_consensus.cc:493] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.737964 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:20.752444 31896 leader_election.cc:304] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:20.752638 32177 raft_consensus.cc:3060] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.753131 32003 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.753377 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91b9568c210d4ef387297f17cde15389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.755340 32183 raft_consensus.cc:3060] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.741240 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:20.741995 31844 tablet_service.cc:1511] Processing CreateTablet for tablet c1bcd4895944417f88a1d53c87eb4a22 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:20.736611 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:20.744136 32066 tablet_service.cc:1511] Processing CreateTablet for tablet 91b9568c210d4ef387297f17cde15389 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:20.771670 31844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bcd4895944417f88a1d53c87eb4a22. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.773196 31969 leader_election.cc:343] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.773622 32178 raft_consensus.cc:493] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.783166 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 7460aac4590c4910abca0a74a4863b5d (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:20.797341 32178 raft_consensus.cc:515] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:20.800781 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:20.801436 32178 leader_election.cc:290] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:20.802362 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:20.802667 32066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91b9568c210d4ef387297f17cde15389. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.802610 31968 leader_election.cc:343] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:20.803866 31967 leader_election.cc:343] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.804468 32127 tablet_bootstrap.cc:654] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.804330 31967 leader_election.cc:304] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:20.807647 32123 tablet_bootstrap.cc:654] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.790964 32125 raft_consensus.cc:359] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:20.809309 32125 raft_consensus.cc:385] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.758473 32163 raft_consensus.cc:515] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.809827 32125 raft_consensus.cc:740] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.811390 32125 consensus_queue.cc:260] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:20.779604 31912 tablet_service.cc:1511] Processing CreateTablet for tablet c1bcd4895944417f88a1d53c87eb4a22 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:20.813989 32163 leader_election.cc:290] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.785259 31990 tablet_service.cc:1511] Processing CreateTablet for tablet c1bcd4895944417f88a1d53c87eb4a22 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:20.817080 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bcd4895944417f88a1d53c87eb4a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.817371 31838 tablet_service.cc:1511] Processing CreateTablet for tablet 7460aac4590c4910abca0a74a4863b5d (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:20.817782 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:20.819024 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7460aac4590c4910abca0a74a4863b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.816679 31915 tablet_service.cc:1511] Processing CreateTablet for tablet 7460aac4590c4910abca0a74a4863b5d (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:20.822357 31915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7460aac4590c4910abca0a74a4863b5d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.730355 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 55254.6us
I20260709 13:16:20.825668 31929 raft_consensus.cc:2468] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:20.826258 32178 raft_consensus.cc:2749] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.840636 32182 raft_consensus.cc:493] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.841370 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:20.841367 32182 raft_consensus.cc:515] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:20.842434 32077 raft_consensus.cc:3060] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.844779 32182 leader_election.cc:290] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:20.846066 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:20.846700 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:20.846839 31854 raft_consensus.cc:2468] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
W20260709 13:16:20.848901 31896 leader_election.cc:343] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.849639 31893 leader_election.cc:304] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: ccf4710690c7473d814836da1babee51
I20260709 13:16:20.850958 32182 raft_consensus.cc:2804] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.851410 32182 raft_consensus.cc:493] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.851758 32182 raft_consensus.cc:3060] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.853660 32042 leader_election.cc:304] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:20.854163 31986 tablet_service.cc:1511] Processing CreateTablet for tablet 604a6aa17b764d61b1403b11157ddbe7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:20.855068 32148 raft_consensus.cc:2804] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.855671 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 604a6aa17b764d61b1403b11157ddbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.862918 31912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1bcd4895944417f88a1d53c87eb4a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.857542 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7460aac4590c4910abca0a74a4863b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.878777 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 604a6aa17b764d61b1403b11157ddbe7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:20.891372 31839 tablet_service.cc:1511] Processing CreateTablet for tablet 604a6aa17b764d61b1403b11157ddbe7 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:20.898254 31839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 604a6aa17b764d61b1403b11157ddbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.906951 32177 raft_consensus.cc:515] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.908422 32004 raft_consensus.cc:2393] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:20.911664 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:20.912737 31929 raft_consensus.cc:2393] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:20.908651 31918 tablet_service.cc:1511] Processing CreateTablet for tablet 56a207058f154c27b443ceacc88454fe (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:20.915513 32126 tablet_bootstrap.cc:654] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.916594 31918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a207058f154c27b443ceacc88454fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.878026 32125 ts_tablet_manager.cc:1434] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.133s	user 0.010s	sys 0.000s
I20260709 13:16:20.921037 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 604a6aa17b764d61b1403b11157ddbe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.931090 32125 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:20.912011 32077 raft_consensus.cc:2468] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:20.945286 31994 tablet_service.cc:1511] Processing CreateTablet for tablet 56a207058f154c27b443ceacc88454fe (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:20.947309 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a207058f154c27b443ceacc88454fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.947883 32177 leader_election.cc:290] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:20.948582 31895 leader_election.cc:304] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: ccf4710690c7473d814836da1babee51
I20260709 13:16:20.932814 32148 raft_consensus.cc:697] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:20.959860 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 56a207058f154c27b443ceacc88454fe (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:20.961903 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "340107361dee4b3997a43799863f36e8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:20.950233 32183 raft_consensus.cc:2749] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.971500 32077 raft_consensus.cc:2393] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:20.972463 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a207058f154c27b443ceacc88454fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.975679 31969 leader_election.cc:304] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:20.977061 32177 raft_consensus.cc:2749] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.978001 32183 raft_consensus.cc:2804] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.977761 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 87e90fd5526b4a2fbfac6d06dc90f086 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:20.978551 32183 raft_consensus.cc:697] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:20.979357 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e90fd5526b4a2fbfac6d06dc90f086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.983036 32134 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.983680 32134 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:20.991940 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:20.993068 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:20.979508 32183 consensus_queue.cc:237] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:20.993921 32077 raft_consensus.cc:2468] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
W20260709 13:16:20.993983 31819 leader_election.cc:343] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.995046 32134 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:20.996799 31820 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:20.998561 32134 raft_consensus.cc:2804] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.999058 32134 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.999410 32134 raft_consensus.cc:3060] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.000149 32148 consensus_queue.cc:237] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:21.009747 32123 tablet_bootstrap.cc:492] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:21.001327 31928 raft_consensus.cc:2468] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:21.043908 31991 tablet_service.cc:1511] Processing CreateTablet for tablet 87e90fd5526b4a2fbfac6d06dc90f086 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:21.046373 31991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e90fd5526b4a2fbfac6d06dc90f086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.052515 32123 ts_tablet_manager.cc:1403] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.306s	user 0.017s	sys 0.024s
I20260709 13:16:21.059357 32068 tablet_service.cc:1511] Processing CreateTablet for tablet 142b461239534bae915d26581eb87d21 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:21.060942 32068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142b461239534bae915d26581eb87d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.062170 31913 tablet_service.cc:1511] Processing CreateTablet for tablet 142b461239534bae915d26581eb87d21 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:21.063283 32127 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:21.063890 32127 ts_tablet_manager.cc:1403] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.389s	user 0.022s	sys 0.035s
I20260709 13:16:21.009778 32003 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:21.063349 31841 tablet_service.cc:1511] Processing CreateTablet for tablet 142b461239534bae915d26581eb87d21 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:21.073881 32127 raft_consensus.cc:359] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.056784 32123 raft_consensus.cc:359] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:21.082429 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142b461239534bae915d26581eb87d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.080101 31913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142b461239534bae915d26581eb87d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.007736 31838 tablet_service.cc:1511] Processing CreateTablet for tablet 87e90fd5526b4a2fbfac6d06dc90f086 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:21.093148 31821 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:21.082970 32127 raft_consensus.cc:385] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.084530 32123 raft_consensus.cc:385] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.104197 32181 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.108624 31837 tablet_service.cc:1511] Processing CreateTablet for tablet 87842118d5214e359e7983d9ede2213b (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:21.109833 31838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e90fd5526b4a2fbfac6d06dc90f086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.103281 31985 tablet_service.cc:1511] Processing CreateTablet for tablet 87842118d5214e359e7983d9ede2213b (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:21.111657 31988 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5b26df57d6488798ed73660ded04a5 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:16:21.087695 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 118566us
I20260709 13:16:21.100122 32127 raft_consensus.cc:740] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.128177 32182 raft_consensus.cc:515] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.100652 32123 raft_consensus.cc:740] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.130808 32065 tablet_service.cc:1511] Processing CreateTablet for tablet 87842118d5214e359e7983d9ede2213b (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:21.134021 32065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87842118d5214e359e7983d9ede2213b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.125782 32125 tablet_bootstrap.cc:654] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.131237 31968 leader_election.cc:304] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:21.138065 32177 raft_consensus.cc:2804] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.139282 32177 raft_consensus.cc:697] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:21.140386 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87842118d5214e359e7983d9ede2213b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.139935 31840 tablet_service.cc:1511] Processing CreateTablet for tablet 48d352eb04d043328db56fab2171304e (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:21.140187 32177 consensus_queue.cc:237] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:21.140003 32126 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:21.143669 32126 ts_tablet_manager.cc:1403] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.442s	user 0.020s	sys 0.008s
I20260709 13:16:21.144063 32127 consensus_queue.cc:260] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.149662 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87842118d5214e359e7983d9ede2213b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.150012 31917 tablet_service.cc:1511] Processing CreateTablet for tablet 48d352eb04d043328db56fab2171304e (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:21.151983 31917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48d352eb04d043328db56fab2171304e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.158824 31990 tablet_service.cc:1511] Processing CreateTablet for tablet 48d352eb04d043328db56fab2171304e (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:21.160641 31990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48d352eb04d043328db56fab2171304e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.167986 31843 tablet_service.cc:1511] Processing CreateTablet for tablet 34769db9ad7a46a8b075b865d2a2b6ea (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:21.170024 31843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34769db9ad7a46a8b075b865d2a2b6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.170775 32123 consensus_queue.cc:260] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:21.174094 31920 tablet_service.cc:1511] Processing CreateTablet for tablet 34769db9ad7a46a8b075b865d2a2b6ea (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:21.175201 32123 ts_tablet_manager.cc:1434] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.122s	user 0.008s	sys 0.001s
I20260709 13:16:21.175655 31920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34769db9ad7a46a8b075b865d2a2b6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.176576 32123 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:21.179683 32126 raft_consensus.cc:359] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.180603 32126 raft_consensus.cc:385] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.180938 32126 raft_consensus.cc:740] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.181901 32126 consensus_queue.cc:260] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.184696 31988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5b26df57d6488798ed73660ded04a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.185075 32126 ts_tablet_manager.cc:1434] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:16:21.187496 32127 ts_tablet_manager.cc:1434] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.123s	user 0.012s	sys 0.000s
I20260709 13:16:21.189407 32127 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:21.195053 32064 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5b26df57d6488798ed73660ded04a5 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:21.196913 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5b26df57d6488798ed73660ded04a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.194208 31994 tablet_service.cc:1511] Processing CreateTablet for tablet ea7811f2bbfe41aea0eba9a542d7dee8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:21.203279 31994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7811f2bbfe41aea0eba9a542d7dee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.219151 32062 tablet_service.cc:1511] Processing CreateTablet for tablet 34769db9ad7a46a8b075b865d2a2b6ea (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:21.220630 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34769db9ad7a46a8b075b865d2a2b6ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:21.169916 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 221620us
I20260709 13:16:21.226715 31914 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5b26df57d6488798ed73660ded04a5 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:21.228892 31914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5b26df57d6488798ed73660ded04a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.223932 31840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48d352eb04d043328db56fab2171304e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.232450 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:21.233263 31855 raft_consensus.cc:3060] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.200347 32181 raft_consensus.cc:697] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:21.238538 32181 consensus_queue.cc:237] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.200512 32182 leader_election.cc:290] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:21.253379 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:21.261751 32134 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:21.269429 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:21.269969 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
W20260709 13:16:21.270176 31896 leader_election.cc:343] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.270841 32078 raft_consensus.cc:3060] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.291482 31753 catalog_manager.cc:5697] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.282227 32134 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:21.290946 32126 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:21.316143 32125 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:21.317027 32125 ts_tablet_manager.cc:1403] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.386s	user 0.018s	sys 0.012s
I20260709 13:16:21.329058 32148 raft_consensus.cc:493] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:21.346333 31819 leader_election.cc:343] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.351866 32064 tablet_service.cc:1511] Processing CreateTablet for tablet ea7811f2bbfe41aea0eba9a542d7dee8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:21.354804 31837 tablet_service.cc:1511] Processing CreateTablet for tablet ea7811f2bbfe41aea0eba9a542d7dee8 (DEFAULT_TABLE table=test-workload [id=b20b3615d00841c189a84679de41a665]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:21.357468 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7811f2bbfe41aea0eba9a542d7dee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.364675 32123 tablet_bootstrap.cc:654] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.366569 31750 catalog_manager.cc:5697] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.334198 32148 raft_consensus.cc:515] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:21.373024 31752 catalog_manager.cc:5697] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.337641 32125 raft_consensus.cc:359] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.355922 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7811f2bbfe41aea0eba9a542d7dee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.383828 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:21.355785 32127 tablet_bootstrap.cc:654] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.398049 31928 raft_consensus.cc:2393] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:21.399677 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "973ae55c06694504ba369b35b010b993" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:21.385733 32125 raft_consensus.cc:385] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.401820 32125 raft_consensus.cc:740] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.401741 32044 leader_election.cc:304] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:21.403178 32188 raft_consensus.cc:3060] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.402945 32125 consensus_queue.cc:260] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.402599 31751 catalog_manager.cc:5697] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:21.397938 32148 leader_election.cc:290] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:21.405686 32125 ts_tablet_manager.cc:1434] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.005s
I20260709 13:16:21.352622 31855 raft_consensus.cc:2468] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:21.409772 31893 leader_election.cc:304] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: ccf4710690c7473d814836da1babee51
I20260709 13:16:21.411401 32182 raft_consensus.cc:2804] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.411948 32182 raft_consensus.cc:697] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:21.413358 32183 raft_consensus.cc:493] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.413972 32183 raft_consensus.cc:515] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.414835 32126 tablet_bootstrap.cc:654] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.386080 32078 raft_consensus.cc:2468] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:21.417946 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:21.413018 32182 consensus_queue.cc:237] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.420001 32003 raft_consensus.cc:2468] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:21.420890 31820 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:21.421352 31896 leader_election.cc:304] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:21.422423 32194 raft_consensus.cc:2804] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.422847 32194 raft_consensus.cc:493] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.423161 32194 raft_consensus.cc:3060] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.423076 32134 raft_consensus.cc:2804] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.424975 32134 raft_consensus.cc:697] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:21.419565 32183 leader_election.cc:290] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:21.429378 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:21.427032 32134 consensus_queue.cc:237] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:21.432700 32188 raft_consensus.cc:2749] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:21.450137 31893 leader_election.cc:343] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.460423 32125 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:21.479991 32183 consensus_queue.cc:1048] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.485812 32127 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:21.486508 32127 ts_tablet_manager.cc:1403] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.297s	user 0.012s	sys 0.067s
I20260709 13:16:21.486923 32194 raft_consensus.cc:515] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.488021 32123 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:21.488673 32123 ts_tablet_manager.cc:1403] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.312s	user 0.013s	sys 0.004s
I20260709 13:16:21.492566 32127 raft_consensus.cc:359] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.495615 32123 raft_consensus.cc:359] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.496028 32127 raft_consensus.cc:385] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.496660 32123 raft_consensus.cc:385] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.496738 32127 raft_consensus.cc:740] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.497407 32123 raft_consensus.cc:740] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.500288 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:21.500461 32123 consensus_queue.cc:260] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:21.502285 31893 leader_election.cc:343] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.504812 32123 ts_tablet_manager.cc:1434] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.005s
I20260709 13:16:21.506182 32123 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:21.508631 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:21.509399 32003 raft_consensus.cc:3060] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.498056 32127 consensus_queue.cc:260] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.514582 32127 ts_tablet_manager.cc:1434] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.003s
I20260709 13:16:21.515904 32127 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:21.521649 32194 leader_election.cc:290] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:21.529261 32125 tablet_bootstrap.cc:654] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.531139 32123 tablet_bootstrap.cc:654] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.532702 32177 raft_consensus.cc:493] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.532866 32188 consensus_queue.cc:1048] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.533756 32177 raft_consensus.cc:515] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.539322 32177 leader_election.cc:290] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:21.542651 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0b712922784fd8a86a1c486c9d372e" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:21.544889 31967 leader_election.cc:343] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.551973 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0b712922784fd8a86a1c486c9d372e" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:21.535863 32003 raft_consensus.cc:2468] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:21.554106 31896 leader_election.cc:304] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
W20260709 13:16:21.555155 31969 leader_election.cc:343] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.555998 31969 leader_election.cc:304] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:21.557605 32193 raft_consensus.cc:2804] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.558255 32193 raft_consensus.cc:697] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:21.558677 32126 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:21.559285 32126 ts_tablet_manager.cc:1403] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.311s	user 0.009s	sys 0.011s
I20260709 13:16:21.559195 32193 consensus_queue.cc:237] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.561589 32178 raft_consensus.cc:2749] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.550462 31753 catalog_manager.cc:5697] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.567718 32188 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.568092 31752 catalog_manager.cc:5697] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.572683 32188 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.583184 32177 consensus_queue.cc:1048] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:16:21.597069 32188 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:21.597347 32134 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.597946 32134 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.599443 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:21.600591 31855 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:21.603147 32041 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:21.603535 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:21.605311 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:21.606066 32077 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:21.605682 32126 raft_consensus.cc:359] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:21.606777 32126 raft_consensus.cc:385] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.605327 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:21.607563 32126 raft_consensus.cc:740] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
W20260709 13:16:21.607620 32044 leader_election.cc:343] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.609761 32127 tablet_bootstrap.cc:654] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:21.609647 31821 leader_election.cc:343] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.611614 31820 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: ccf4710690c7473d814836da1babee51
I20260709 13:16:21.612653 32181 raft_consensus.cc:2804] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.613042 32181 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.613382 32181 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.606176 32134 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:21.606856 32188 raft_consensus.cc:2804] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.621910 32182 consensus_queue.cc:1048] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.622577 32188 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.625962 32188 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.608707 32126 consensus_queue.cc:260] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:21.667649 32134 consensus_queue.cc:1048] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.670843 32177 consensus_queue.cc:1048] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:21.681744 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:21.692687 32126 ts_tablet_manager.cc:1434] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.133s	user 0.009s	sys 0.005s
I20260709 13:16:21.695240 32126 tablet_bootstrap.cc:492] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:21.705219 32125 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:21.714573 32125 ts_tablet_manager.cc:1403] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.286s	user 0.024s	sys 0.042s
I20260709 13:16:21.709414 32178 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.714663 32077 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.715699 32178 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:21.718998 32178 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:21.708369 32127 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:21.719956 32127 ts_tablet_manager.cc:1403] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.204s	user 0.043s	sys 0.036s
I20260709 13:16:21.719892 32182 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.720691 32182 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:21.723564 32127 raft_consensus.cc:359] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.724422 32127 raft_consensus.cc:385] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.724859 32127 raft_consensus.cc:740] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.725004 32182 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:21.725981 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:21.725924 32127 consensus_queue.cc:260] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.728935 32127 ts_tablet_manager.cc:1434] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:21.730198 32127 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:21.730049 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:21.731335 32076 raft_consensus.cc:2468] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:21.734071 31969 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:21.735729 32178 raft_consensus.cc:2804] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:21.736308 32178 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.736771 32178 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:21.738144 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:21.739789 32123 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:21.740384 32123 ts_tablet_manager.cc:1403] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.234s	user 0.013s	sys 0.039s
I20260709 13:16:21.742703 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:21.746232 32076 raft_consensus.cc:2393] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:21.733974 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:21.748438 32125 raft_consensus.cc:359] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:21.749321 31968 leader_election.cc:343] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.752113 32123 raft_consensus.cc:359] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.753362 32123 raft_consensus.cc:385] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.753711 32123 raft_consensus.cc:740] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.754864 32123 consensus_queue.cc:260] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.750540 32126 tablet_bootstrap.cc:654] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.750428 31895 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:21.752990 32125 raft_consensus.cc:385] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.756268 32125 raft_consensus.cc:740] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.757112 32123 ts_tablet_manager.cc:1434] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:16:21.760908 32127 tablet_bootstrap.cc:654] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.764441 32123 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:21.757259 32197 consensus_queue.cc:1048] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:16:21.763796 32182 raft_consensus.cc:3060] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.761184 32188 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.771334 32125 consensus_queue.cc:260] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.779117 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:21.787741 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
W20260709 13:16:21.792532 32044 leader_election.cc:343] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.780488 32188 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:21.796039 32181 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.805070 31855 raft_consensus.cc:2393] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:21.801654 31750 catalog_manager.cc:5697] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.795401 32178 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:21.822275 32178 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:21.823395 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:21.824115 32178 raft_consensus.cc:493] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.827278 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:21.828136 32076 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:21.829149 31968 leader_election.cc:343] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.818485 32041 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:21.832938 32222 raft_consensus.cc:2749] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.841281 32193 consensus_queue.cc:1048] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.846366 32125 ts_tablet_manager.cc:1434] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.131s	user 0.010s	sys 0.004s
I20260709 13:16:21.847741 32125 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:21.860316 32123 tablet_bootstrap.cc:654] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.860903 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:21.863194 32075 raft_consensus.cc:2393] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:21.825076 32178 raft_consensus.cc:515] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.868024 32178 leader_election.cc:290] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:21.868098 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:21.869092 32178 raft_consensus.cc:493] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.869320 31854 raft_consensus.cc:2468] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:21.869560 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:21.869711 32178 raft_consensus.cc:515] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.871258 31967 leader_election.cc:304] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:21.872483 32178 leader_election.cc:290] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:16:21.872555 31968 leader_election.cc:343] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.875756 32230 raft_consensus.cc:2804] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.876309 32230 raft_consensus.cc:493] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.876709 32230 raft_consensus.cc:3060] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.892081 32230 raft_consensus.cc:515] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:21.895200 32230 leader_election.cc:290] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:21.896850 31853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:21.897769 31853 raft_consensus.cc:3060] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.901793 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:21.904389 32127 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:21.905077 32127 ts_tablet_manager.cc:1403] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.175s	user 0.015s	sys 0.005s
I20260709 13:16:21.906152 32182 raft_consensus.cc:2749] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.862166 32181 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:21.919374 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
W20260709 13:16:21.921339 31968 leader_election.cc:343] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.915491 32076 raft_consensus.cc:2468] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:21.937342 32126 tablet_bootstrap.cc:492] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:21.938390 32126 ts_tablet_manager.cc:1403] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.244s	user 0.030s	sys 0.040s
I20260709 13:16:21.933378 32125 tablet_bootstrap.cc:654] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.915987 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:21.937423 31969 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:21.911070 31853 raft_consensus.cc:2468] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:21.943980 32003 raft_consensus.cc:3060] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.948534 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:21.948911 31967 leader_election.cc:304] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:21.951573 31968 leader_election.cc:304] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:21.955070 32181 consensus_queue.cc:1048] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.958175 32178 raft_consensus.cc:2804] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:16:21.962188 31821 leader_election.cc:343] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.966730 31821 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:21.971638 32127 raft_consensus.cc:359] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.972893 32127 raft_consensus.cc:385] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.973245 32127 raft_consensus.cc:740] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.976176 32181 raft_consensus.cc:2749] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.981730 32127 consensus_queue.cc:260] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:21.961992 32233 raft_consensus.cc:2804] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:21.994874 32230 raft_consensus.cc:2749] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.998178 32126 raft_consensus.cc:359] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.001076 32126 raft_consensus.cc:385] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.001719 32126 raft_consensus.cc:740] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.002588 32126 consensus_queue.cc:260] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.004367 32123 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:22.005759 32123 ts_tablet_manager.cc:1403] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.243s	user 0.014s	sys 0.010s
I20260709 13:16:22.010201 32193 consensus_queue.cc:1048] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.019770 32126 ts_tablet_manager.cc:1434] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.081s	user 0.011s	sys 0.001s
I20260709 13:16:22.021454 32126 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.030166 32123 raft_consensus.cc:359] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.031154 32123 raft_consensus.cc:385] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.031586 32123 raft_consensus.cc:740] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.048029 32123 consensus_queue.cc:260] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.049866 32125 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:22.050828 32125 ts_tablet_manager.cc:1403] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.203s	user 0.016s	sys 0.014s
I20260709 13:16:22.051337 32123 ts_tablet_manager.cc:1434] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.045s	user 0.010s	sys 0.000s
I20260709 13:16:22.053251 32123 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:22.052386 32181 consensus_queue.cc:1048] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.019075 32178 raft_consensus.cc:697] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:22.025086 32233 raft_consensus.cc:697] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:22.062003 32126 tablet_bootstrap.cc:654] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.058202 32182 consensus_queue.cc:1048] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:22.068696 32222 raft_consensus.cc:493] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.069442 32222 raft_consensus.cc:515] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.078434 32222 leader_election.cc:290] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.079763 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0b712922784fd8a86a1c486c9d372e" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:22.083671 32002 raft_consensus.cc:2468] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:22.082679 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0b712922784fd8a86a1c486c9d372e" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:22.086504 32044 leader_election.cc:304] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:22.089064 32222 raft_consensus.cc:2804] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.089879 32233 consensus_queue.cc:237] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:22.091284 32041 leader_election.cc:343] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.076413 32178 consensus_queue.cc:237] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.101464 32123 tablet_bootstrap.cc:654] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.090782 32222 raft_consensus.cc:493] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.115906 32127 ts_tablet_manager.cc:1434] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.210s	user 0.014s	sys 0.014s
I20260709 13:16:22.115293 32125 raft_consensus.cc:359] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.117359 32125 raft_consensus.cc:385] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.117812 32125 raft_consensus.cc:740] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.119050 32127 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:22.118950 32125 consensus_queue.cc:260] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.132810 32125 ts_tablet_manager.cc:1434] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.003s
I20260709 13:16:22.138671 32125 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:22.123085 31855 raft_consensus.cc:3060] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.120716 32222 raft_consensus.cc:3060] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.178126 31853 raft_consensus.cc:3060] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.179255 32126 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.179920 32126 ts_tablet_manager.cc:1403] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.159s	user 0.023s	sys 0.039s
I20260709 13:16:22.186339 32126 raft_consensus.cc:359] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.187175 32126 raft_consensus.cc:385] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.187479 32126 raft_consensus.cc:740] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.188199 32126 consensus_queue.cc:260] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.190557 32126 ts_tablet_manager.cc:1434] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:16:22.191864 32126 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.198765 32127 tablet_bootstrap.cc:654] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.201834 32126 tablet_bootstrap.cc:654] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.207818 32192 consensus_queue.cc:1048] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.222780 32222 raft_consensus.cc:515] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.241092 32125 tablet_bootstrap.cc:654] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.248814 32123 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:22.250675 32123 ts_tablet_manager.cc:1403] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.198s	user 0.030s	sys 0.043s
I20260709 13:16:22.256961 32126 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.257844 32126 ts_tablet_manager.cc:1403] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.066s	user 0.020s	sys 0.009s
I20260709 13:16:22.261653 32127 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:22.262431 32127 ts_tablet_manager.cc:1403] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.144s	user 0.018s	sys 0.013s
I20260709 13:16:22.265291 32125 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:22.265919 32125 ts_tablet_manager.cc:1403] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.128s	user 0.013s	sys 0.004s
I20260709 13:16:22.265022 31750 catalog_manager.cc:5697] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:22.266197 32127 raft_consensus.cc:359] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.266953 32127 raft_consensus.cc:385] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.267251 32127 raft_consensus.cc:740] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.268038 32127 consensus_queue.cc:260] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.267778 31750 catalog_manager.cc:5697] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 2, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.268456 32125 raft_consensus.cc:359] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.269158 32125 raft_consensus.cc:385] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.269551 32125 raft_consensus.cc:740] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.271479 32127 ts_tablet_manager.cc:1434] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:22.273089 32127 tablet_bootstrap.cc:492] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:22.262764 32222 leader_election.cc:290] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.274930 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0b712922784fd8a86a1c486c9d372e" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:22.275718 32004 raft_consensus.cc:3060] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.276942 32123 raft_consensus.cc:359] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.277878 32123 raft_consensus.cc:385] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.278138 32123 raft_consensus.cc:740] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.280879 32127 tablet_bootstrap.cc:654] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.281567 32126 raft_consensus.cc:359] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.282440 32126 raft_consensus.cc:385] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.282740 32126 raft_consensus.cc:740] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.283578 32126 consensus_queue.cc:260] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.285976 32123 consensus_queue.cc:260] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.293771 32123 ts_tablet_manager.cc:1434] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.001s
I20260709 13:16:22.295205 32134 raft_consensus.cc:493] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 32690b3987094d51bb4f214321a29067)
I20260709 13:16:22.295197 32123 tablet_bootstrap.cc:492] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:22.296030 32134 raft_consensus.cc:515] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.301483 32126 ts_tablet_manager.cc:1434] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:16:22.302707 32252 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.303609 32252 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.306344 32252 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:22.307437 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:22.307898 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:22.308679 32077 raft_consensus.cc:2368] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 for earlier term 1. Current term is 2.
I20260709 13:16:22.308859 32004 raft_consensus.cc:2468] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:22.307468 32126 tablet_bootstrap.cc:492] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.310528 32044 leader_election.cc:304] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:22.311777 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b0b712922784fd8a86a1c486c9d372e" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:22.312625 32222 raft_consensus.cc:2804] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.313318 32222 raft_consensus.cc:697] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:22.314620 32222 consensus_queue.cc:237] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.310566 31895 leader_election.cc:400] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551) with higher term. Message: Invalid argument: T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 for earlier term 1. Current term is 2.
I20260709 13:16:22.317764 31895 leader_election.cc:403] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:22.318883 32193 raft_consensus.cc:493] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.319471 32193 raft_consensus.cc:515] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.323766 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:22.325835 31893 leader_election.cc:343] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.328168 32193 leader_election.cc:290] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:22.328867 32193 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:16:22.338142 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:22.339974 32134 leader_election.cc:290] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:22.340973 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
W20260709 13:16:22.343536 31896 leader_election.cc:343] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.344091 31896 leader_election.cc:304] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:22.341326 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:22.348192 32178 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.351337 32178 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.350353 31820 leader_election.cc:304] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:22.356588 32181 raft_consensus.cc:2749] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:22.357599 32178 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.356772 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:22.317298 32125 consensus_queue.cc:260] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.358973 31855 raft_consensus.cc:3060] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.359803 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:22.360749 31852 raft_consensus.cc:2393] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:22.362612 32252 raft_consensus.cc:2749] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.365320 32126 tablet_bootstrap.cc:654] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.358700 32077 raft_consensus.cc:2393] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:22.372524 31969 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:22.373145 32125 ts_tablet_manager.cc:1434] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.002s
I20260709 13:16:22.374290 32233 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.375948 32193 raft_consensus.cc:2749] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551) with higher term. Message: Invalid argument: T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 for earlier term 1. Current term is 2.
I20260709 13:16:22.384601 32123 tablet_bootstrap.cc:654] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.374352 32125 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:22.396051 32127 tablet_bootstrap.cc:492] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:22.399981 32233 raft_consensus.cc:2749] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.401039 32127 ts_tablet_manager.cc:1403] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.128s	user 0.025s	sys 0.013s
I20260709 13:16:22.403261 32126 tablet_bootstrap.cc:492] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.403982 32126 ts_tablet_manager.cc:1403] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.097s	user 0.033s	sys 0.002s
I20260709 13:16:22.406024 31855 raft_consensus.cc:2468] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:22.408192 32126 raft_consensus.cc:359] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.409299 32126 raft_consensus.cc:385] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.409665 32126 raft_consensus.cc:740] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.410633 32126 consensus_queue.cc:260] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.416198 32222 raft_consensus.cc:493] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.416900 32222 raft_consensus.cc:515] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.420703 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:22.421460 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:22.422482 32042 leader_election.cc:343] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.423444 32222 leader_election.cc:290] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:16:22.423393 32041 leader_election.cc:343] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.417289 32127 raft_consensus.cc:359] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.424257 32041 leader_election.cc:304] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:22.424757 32127 raft_consensus.cc:385] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.425263 32127 raft_consensus.cc:740] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.425474 32222 raft_consensus.cc:2749] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.426743 32127 consensus_queue.cc:260] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.430263 32127 ts_tablet_manager.cc:1434] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:16:22.431555 32127 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:22.434232 32126 ts_tablet_manager.cc:1434] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:16:22.436192 32125 tablet_bootstrap.cc:654] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.436379 32126 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.442098 32127 tablet_bootstrap.cc:654] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.444621 32126 tablet_bootstrap.cc:654] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.434413 32123 tablet_bootstrap.cc:492] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:22.451411 32123 ts_tablet_manager.cc:1403] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.000s
I20260709 13:16:22.455405 32127 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:22.456329 32127 ts_tablet_manager.cc:1403] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.002s
I20260709 13:16:22.455691 31752 catalog_manager.cc:5697] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.455367 32123 raft_consensus.cc:359] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.457440 32123 raft_consensus.cc:385] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.457813 32123 raft_consensus.cc:740] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.459096 32123 consensus_queue.cc:260] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.463972 32123 ts_tablet_manager.cc:1434] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:16:22.465248 32123 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:22.465426 32126 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.466269 32126 ts_tablet_manager.cc:1403] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.007s
I20260709 13:16:22.461244 32127 raft_consensus.cc:359] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.470784 32127 raft_consensus.cc:385] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.471109 32127 raft_consensus.cc:740] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.471894 32127 consensus_queue.cc:260] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.476178 32127 ts_tablet_manager.cc:1434] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:16:22.477634 32127 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:22.479106 32126 raft_consensus.cc:359] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.479842 32126 raft_consensus.cc:385] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.480136 32126 raft_consensus.cc:740] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.481344 32126 consensus_queue.cc:260] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.484277 32126 ts_tablet_manager.cc:1434] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:16:22.488638 32127 tablet_bootstrap.cc:654] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.495971 32126 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.499949 32125 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:22.500653 32125 ts_tablet_manager.cc:1403] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.127s	user 0.015s	sys 0.010s
I20260709 13:16:22.502812 31929 raft_consensus.cc:1275] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:22.505927 32233 consensus_queue.cc:1048] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.510404 32125 raft_consensus.cc:359] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.511310 32125 raft_consensus.cc:385] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.513650 32123 tablet_bootstrap.cc:654] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.511606 32125 raft_consensus.cc:740] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.518375 32126 tablet_bootstrap.cc:654] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.519448 32125 consensus_queue.cc:260] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.528623 32125 ts_tablet_manager.cc:1434] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.001s
I20260709 13:16:22.529708 32125 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:22.528082 32233 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.533555 32233 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.537536 32233 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:22.539160 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:22.540079 31928 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:22.540109 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:22.541290 32126 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.541674 31968 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:22.541991 32126 ts_tablet_manager.cc:1403] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.008s
I20260709 13:16:22.541201 31855 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:22.542346 32178 consensus_queue.cc:1048] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.545023 32077 raft_consensus.cc:1275] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:22.542657 32242 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.546259 32242 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.546721 32242 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.552629 32236 consensus_queue.cc:1048] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:22.555859 31968 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:22.560475 32125 tablet_bootstrap.cc:654] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.563361 32126 raft_consensus.cc:359] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.564895 32126 raft_consensus.cc:385] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.565344 32126 raft_consensus.cc:740] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.566277 32126 consensus_queue.cc:260] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.568790 32126 ts_tablet_manager.cc:1434] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:16:22.578076 32193 raft_consensus.cc:493] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.579373 32126 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.578878 32193 raft_consensus.cc:515] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.583146 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "039ecd5ff518405fb0fc382c46e7e7c4" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:22.583382 32193 leader_election.cc:290] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:22.583989 32076 raft_consensus.cc:2468] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:22.584088 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "039ecd5ff518405fb0fc382c46e7e7c4" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:22.585065 32004 raft_consensus.cc:2468] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:22.586028 31895 leader_election.cc:304] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:22.587584 32193 raft_consensus.cc:2804] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.587960 32193 raft_consensus.cc:493] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.588336 32193 raft_consensus.cc:3060] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.604445 32193 raft_consensus.cc:515] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.606284 32242 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.608343 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "039ecd5ff518405fb0fc382c46e7e7c4" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:22.609246 32004 raft_consensus.cc:3060] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.609532 32242 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:22.609833 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:22.610281 32127 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:22.610548 31852 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.611442 32127 ts_tablet_manager.cc:1403] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.134s	user 0.019s	sys 0.008s
I20260709 13:16:22.614547 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:22.615321 31929 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.615576 32127 raft_consensus.cc:359] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.616632 32127 raft_consensus.cc:385] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.617023 32127 raft_consensus.cc:740] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.619302 32127 consensus_queue.cc:260] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.620600 32126 tablet_bootstrap.cc:654] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.625324 32193 leader_election.cc:290] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:22.630787 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "039ecd5ff518405fb0fc382c46e7e7c4" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:22.631569 32077 raft_consensus.cc:3060] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.644021 32004 raft_consensus.cc:2468] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:22.645946 32077 raft_consensus.cc:2468] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:22.646266 31896 leader_election.cc:304] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:22.647557 32182 raft_consensus.cc:2804] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.648113 32182 raft_consensus.cc:697] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:22.649062 32182 consensus_queue.cc:237] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:22.656091 31852 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:22.659993 32125 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:22.660065 32193 raft_consensus.cc:493] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.661073 32193 raft_consensus.cc:515] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.662364 31967 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:22.664889 32127 ts_tablet_manager.cc:1434] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 13:16:22.666019 32127 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:22.667366 32193 leader_election.cc:290] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.668220 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:22.668979 32076 raft_consensus.cc:2468] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:22.660637 32125 ts_tablet_manager.cc:1403] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.131s	user 0.012s	sys 0.008s
I20260709 13:16:22.671411 32123 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:22.671546 31895 leader_election.cc:304] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:22.672005 32123 ts_tablet_manager.cc:1403] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.207s	user 0.023s	sys 0.006s
I20260709 13:16:22.673271 32252 raft_consensus.cc:2804] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.673714 32252 raft_consensus.cc:493] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.674135 32252 raft_consensus.cc:3060] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.673820 32125 raft_consensus.cc:359] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.674650 32125 raft_consensus.cc:385] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.674935 32125 raft_consensus.cc:740] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.674923 32123 raft_consensus.cc:359] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.675791 32123 raft_consensus.cc:385] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.676278 32123 raft_consensus.cc:740] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.675926 32125 consensus_queue.cc:260] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.673969 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:22.677297 31852 raft_consensus.cc:2468] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:22.678514 32125 ts_tablet_manager.cc:1434] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:22.678979 32178 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.679597 32178 raft_consensus.cc:697] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:22.680074 32125 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:22.680518 32178 consensus_queue.cc:237] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.681175 31929 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:22.683751 32123 consensus_queue.cc:260] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.687779 32125 tablet_bootstrap.cc:654] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:22.695317 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:22.700819 32123 ts_tablet_manager.cc:1434] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 13:16:22.706244 32126 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.709451 32126 ts_tablet_manager.cc:1403] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.002s
I20260709 13:16:22.710168 32123 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:22.711905 32127 tablet_bootstrap.cc:654] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.716782 32126 raft_consensus.cc:359] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.718034 32126 raft_consensus.cc:385] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.718437 32126 raft_consensus.cc:740] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.719585 32126 consensus_queue.cc:260] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.725633 32126 ts_tablet_manager.cc:1434] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 13:16:22.726907 32126 tablet_bootstrap.cc:492] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:22.739233 32252 raft_consensus.cc:515] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.746592 32188 raft_consensus.cc:493] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.756618 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:22.754555 32188 raft_consensus.cc:515] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.758627 32127 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:22.759639 32127 ts_tablet_manager.cc:1403] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.094s	user 0.016s	sys 0.004s
I20260709 13:16:22.762241 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6aaf6fc3e504bc79c75c64bf99d89e7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:22.763473 32004 raft_consensus.cc:2468] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:22.765900 32044 leader_election.cc:304] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:22.767674 32222 raft_consensus.cc:2804] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.768234 32222 raft_consensus.cc:493] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.768952 32222 raft_consensus.cc:3060] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.769731 32125 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:22.770390 32125 ts_tablet_manager.cc:1403] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.091s	user 0.021s	sys 0.030s
I20260709 13:16:22.757416 32077 raft_consensus.cc:3060] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.773197 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:22.773438 32123 tablet_bootstrap.cc:654] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.786640 31753 catalog_manager.cc:5697] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.774950 32125 raft_consensus.cc:359] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.771677 32252 leader_election.cc:290] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.782835 32178 raft_consensus.cc:493] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.794955 32178 raft_consensus.cc:515] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.783668 32127 raft_consensus.cc:359] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.801750 32127 raft_consensus.cc:385] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.805658 32127 raft_consensus.cc:740] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.807214 32127 consensus_queue.cc:260] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:22.784624 32126 tablet_bootstrap.cc:654] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.791923 32125 raft_consensus.cc:385] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.808924 32125 raft_consensus.cc:740] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.800522 31852 raft_consensus.cc:3060] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.810380 32125 consensus_queue.cc:260] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.818248 32125 ts_tablet_manager.cc:1434] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.000s
I20260709 13:16:22.820314 32125 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:22.786586 32188 leader_election.cc:290] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.823267 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6aaf6fc3e504bc79c75c64bf99d89e7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:22.823812 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:22.824546 31855 raft_consensus.cc:2468] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
W20260709 13:16:22.825822 31967 leader_election.cc:343] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.837527 32188 consensus_queue.cc:1048] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.827572 32178 leader_election.cc:290] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.840426 32222 raft_consensus.cc:515] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.844515 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:22.843987 32127 ts_tablet_manager.cc:1434] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.001s
I20260709 13:16:22.846864 32127 tablet_bootstrap.cc:492] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:22.854738 32077 raft_consensus.cc:2468] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:22.863366 32222 leader_election.cc:290] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:22.864651 32127 tablet_bootstrap.cc:654] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.865248 32123 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:22.865895 32123 ts_tablet_manager.cc:1403] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.156s	user 0.026s	sys 0.019s
I20260709 13:16:22.866974 32126 tablet_bootstrap.cc:492] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:22.867511 32126 ts_tablet_manager.cc:1403] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.141s	user 0.015s	sys 0.013s
I20260709 13:16:22.869678 32123 raft_consensus.cc:359] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:22.917721 32123 raft_consensus.cc:385] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.918381 32123 raft_consensus.cc:740] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.920750 32123 consensus_queue.cc:260] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:22.926846 31969 leader_election.cc:343] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.928220 31969 leader_election.cc:304] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:22.929134 32123 ts_tablet_manager.cc:1434] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.063s	user 0.013s	sys 0.003s
I20260709 13:16:22.940591 32188 raft_consensus.cc:493] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.944208 32188 raft_consensus.cc:515] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:22.949301 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6aaf6fc3e504bc79c75c64bf99d89e7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:22.929263 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6aaf6fc3e504bc79c75c64bf99d89e7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:22.953626 32003 raft_consensus.cc:3060] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.961743 32178 raft_consensus.cc:2749] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.967469 31855 raft_consensus.cc:3060] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:22.974813 31969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 48399.4us
I20260709 13:16:22.979326 32275 consensus_queue.cc:1048] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.991004 32252 raft_consensus.cc:493] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.995797 32127 tablet_bootstrap.cc:492] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:22.997413 32127 ts_tablet_manager.cc:1403] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.151s	user 0.016s	sys 0.001s
I20260709 13:16:23.005749 32188 leader_election.cc:290] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:22.999778 32252 raft_consensus.cc:515] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.014046 32125 tablet_bootstrap.cc:654] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.017778 32182 raft_consensus.cc:493] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.012614 32126 raft_consensus.cc:359] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.019110 32126 raft_consensus.cc:385] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.018869 32182 raft_consensus.cc:515] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.019439 32126 raft_consensus.cc:740] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.020541 32126 consensus_queue.cc:260] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.030674 32123 tablet_bootstrap.cc:492] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:23.051733 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:23.053190 32126 ts_tablet_manager.cc:1434] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.185s	user 0.012s	sys 0.006s
I20260709 13:16:23.063511 32182 leader_election.cc:290] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:23.065138 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:23.065357 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea2a2bfc574918bd4f2bf7f2839e9c" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:23.068601 31893 leader_election.cc:343] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.069557 31896 leader_election.cc:304] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:23.071256 32182 raft_consensus.cc:2749] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.069844 32127 raft_consensus.cc:359] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.072885 32127 raft_consensus.cc:385] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:16:23.073939 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 73763.3us
I20260709 13:16:23.072460 32252 leader_election.cc:290] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:23.081393 31929 raft_consensus.cc:3060] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:23.056340 31821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 83659.6us
I20260709 13:16:23.095841 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:23.097842 32041 leader_election.cc:343] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.099030 31895 leader_election.cc:304] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:23.073324 32127 raft_consensus.cc:740] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.096186 32123 tablet_bootstrap.cc:654] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.116896 32252 raft_consensus.cc:2804] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.117914 32252 raft_consensus.cc:697] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:23.123679 31855 raft_consensus.cc:2468] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:23.125937 32003 raft_consensus.cc:2468] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:23.128849 32041 leader_election.cc:304] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:23.129591 32280 raft_consensus.cc:493] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.131392 31852 raft_consensus.cc:2468] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:23.131770 32280 raft_consensus.cc:515] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.129791 32127 consensus_queue.cc:260] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.119956 32252 consensus_queue.cc:237] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:16:23.108728 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 54589.3us
I20260709 13:16:23.139283 32126 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:23.145859 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:23.146714 32004 raft_consensus.cc:2468] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:23.147789 32280 leader_election.cc:290] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:23.148207 32044 leader_election.cc:304] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:23.148610 32280 raft_consensus.cc:2804] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.149415 32275 raft_consensus.cc:2804] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.149350 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:23.149734 32280 raft_consensus.cc:697] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:23.150034 32275 raft_consensus.cc:493] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.150522 32275 raft_consensus.cc:3060] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.151304 32280 consensus_queue.cc:237] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:23.152086 32041 leader_election.cc:343] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.157843 32233 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.160175 32127 ts_tablet_manager.cc:1434] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.146s	user 0.009s	sys 0.003s
I20260709 13:16:23.169601 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:23.170709 32076 raft_consensus.cc:2468] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:23.178535 32125 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:23.176332 31753 catalog_manager.cc:5697] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.194655 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:23.195761 31930 raft_consensus.cc:2468] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
W20260709 13:16:23.186475 31820 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 63300.5us
I20260709 13:16:23.196892 32127 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:23.198233 32125 ts_tablet_manager.cc:1403] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.378s	user 0.029s	sys 0.036s
I20260709 13:16:23.208992 32233 raft_consensus.cc:493] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.209461 32123 tablet_bootstrap.cc:492] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:23.210083 32123 ts_tablet_manager.cc:1403] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.180s	user 0.015s	sys 0.017s
I20260709 13:16:23.209774 32233 raft_consensus.cc:515] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.210566 32042 leader_election.cc:304] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:23.212567 32282 raft_consensus.cc:2804] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.213420 32282 raft_consensus.cc:493] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.213886 32282 raft_consensus.cc:3060] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.214480 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62477156fe044dad9635981556678bef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:23.215368 31927 raft_consensus.cc:2468] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:23.219740 32178 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.222190 32240 raft_consensus.cc:493] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.223140 32240 raft_consensus.cc:515] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.232570 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:23.232827 32240 leader_election.cc:290] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:16:23.234660 31967 leader_election.cc:343] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.234457 32283 consensus_queue.cc:1048] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.236219 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62477156fe044dad9635981556678bef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:23.234539 32233 leader_election.cc:290] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:23.240661 31968 leader_election.cc:304] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
W20260709 13:16:23.241421 31969 leader_election.cc:343] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.243827 32240 raft_consensus.cc:2804] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.244804 32240 raft_consensus.cc:493] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.245236 32240 raft_consensus.cc:3060] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.222065 31895 leader_election.cc:304] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:23.251572 32275 raft_consensus.cc:515] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.266323 32275 leader_election.cc:290] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:23.263006 32123 raft_consensus.cc:359] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.269680 32123 raft_consensus.cc:385] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.249927 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:23.270548 32123 raft_consensus.cc:740] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.273087 32127 tablet_bootstrap.cc:654] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.268745 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:23.283047 32003 raft_consensus.cc:3060] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.277947 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
W20260709 13:16:23.288250 32041 leader_election.cc:343] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.282238 32126 tablet_bootstrap.cc:654] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:23.271667 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 49833.3us
I20260709 13:16:23.291468 31927 raft_consensus.cc:2468] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:23.303525 32125 raft_consensus.cc:359] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.304798 32125 raft_consensus.cc:385] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.307346 31968 leader_election.cc:304] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:23.308564 32233 raft_consensus.cc:2804] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.309206 32233 raft_consensus.cc:493] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.305382 32125 raft_consensus.cc:740] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.271816 32252 raft_consensus.cc:2804] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.309589 32233 raft_consensus.cc:3060] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.319751 32125 consensus_queue.cc:260] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.319269 32252 raft_consensus.cc:493] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.320264 32193 consensus_queue.cc:1048] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:16:23.334026 32125 ts_tablet_manager.cc:1434] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.135s	user 0.015s	sys 0.000s
I20260709 13:16:23.341439 32125 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:23.327586 32252 raft_consensus.cc:3060] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.340173 32123 consensus_queue.cc:260] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.382455 32123 ts_tablet_manager.cc:1434] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.172s	user 0.008s	sys 0.005s
I20260709 13:16:23.396221 32126 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:23.397710 32126 ts_tablet_manager.cc:1403] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.267s	user 0.028s	sys 0.029s
I20260709 13:16:23.401157 32123 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:23.409942 32127 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:23.410750 32127 ts_tablet_manager.cc:1403] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.230s	user 0.020s	sys 0.005s
I20260709 13:16:23.427426 32125 tablet_bootstrap.cc:654] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.427445 32126 raft_consensus.cc:359] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.433755 32126 raft_consensus.cc:385] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.434487 32126 raft_consensus.cc:740] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.439266 32123 tablet_bootstrap.cc:654] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.440367 32003 raft_consensus.cc:2468] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:23.441874 32044 leader_election.cc:304] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:23.442291 32252 raft_consensus.cc:515] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.443295 32275 raft_consensus.cc:2804] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.443828 32275 raft_consensus.cc:697] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:23.444761 32275 consensus_queue.cc:237] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.446053 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:23.446812 31852 raft_consensus.cc:3060] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.447208 32240 raft_consensus.cc:515] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.454470 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:23.456017 32282 raft_consensus.cc:515] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.458609 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62477156fe044dad9635981556678bef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:23.459513 32252 leader_election.cc:290] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:23.459599 31928 raft_consensus.cc:3060] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.467439 32077 raft_consensus.cc:2393] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:23.471109 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62477156fe044dad9635981556678bef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:23.472872 32075 raft_consensus.cc:3060] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.477963 32126 consensus_queue.cc:260] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.481974 32126 ts_tablet_manager.cc:1434] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 13:16:23.483855 32126 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:23.485128 31852 raft_consensus.cc:2468] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:23.487036 31893 leader_election.cc:304] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:23.489454 32252 raft_consensus.cc:2804] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.490584 32252 raft_consensus.cc:697] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:23.464751 32127 raft_consensus.cc:359] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.492340 32127 raft_consensus.cc:385] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.492882 32127 raft_consensus.cc:740] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.496378 32233 raft_consensus.cc:515] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.491941 32252 consensus_queue.cc:237] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.497581 32240 leader_election.cc:290] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:23.497907 32127 consensus_queue.cc:260] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.505782 32282 leader_election.cc:290] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:23.504555 32127 ts_tablet_manager.cc:1434] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.093s	user 0.011s	sys 0.001s
I20260709 13:16:23.510007 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:23.506788 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:23.512365 31928 raft_consensus.cc:2468] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:23.513033 31852 raft_consensus.cc:2393] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:23.514458 31968 leader_election.cc:304] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:23.515482 32233 leader_election.cc:290] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:23.516896 32297 raft_consensus.cc:2804] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.519397 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:23.519479 32041 leader_election.cc:304] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:23.521804 32282 raft_consensus.cc:2749] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:16:23.522605 31967 leader_election.cc:343] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.523604 32125 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:23.524334 32125 ts_tablet_manager.cc:1403] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.188s	user 0.019s	sys 0.005s
I20260709 13:16:23.528029 32125 raft_consensus.cc:359] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.532209 32075 raft_consensus.cc:2468] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:23.529259 32125 raft_consensus.cc:385] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.535562 32125 raft_consensus.cc:740] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.536456 32125 consensus_queue.cc:260] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.538558 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:23.539530 31928 raft_consensus.cc:3060] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.535960 31751 catalog_manager.cc:5697] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.529737 32297 raft_consensus.cc:697] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:23.556834 32126 tablet_bootstrap.cc:654] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.558007 32181 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.558542 31928 raft_consensus.cc:2468] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:23.558736 32181 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.559434 32125 ts_tablet_manager.cc:1434] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:16:23.560374 31968 leader_election.cc:304] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:23.564869 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:23.565773 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:23.566843 32004 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:23.566996 32233 raft_consensus.cc:2804] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.570312 31821 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:23.565869 32076 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:23.573297 32181 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:23.575254 32125 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:23.577363 32285 raft_consensus.cc:2804] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:23.573189 32233 raft_consensus.cc:697] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:23.579780 32285 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.580291 32285 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.581084 32123 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:23.581760 32123 ts_tablet_manager.cc:1403] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.188s	user 0.013s	sys 0.008s
I20260709 13:16:23.586809 32233 consensus_queue.cc:237] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.591185 32127 tablet_bootstrap.cc:492] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:23.558017 32297 consensus_queue.cc:237] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.590471 32123 raft_consensus.cc:359] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.595801 32123 raft_consensus.cc:385] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.596704 32123 raft_consensus.cc:740] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.597985 32123 consensus_queue.cc:260] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.601410 32123 ts_tablet_manager.cc:1434] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.002s
I20260709 13:16:23.604817 32283 consensus_queue.cc:1048] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.607975 32123 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:23.603994 31753 catalog_manager.cc:5697] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.611299 32275 consensus_queue.cc:1048] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.614638 31753 catalog_manager.cc:5697] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.606056 32127 tablet_bootstrap.cc:654] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.623054 32123 tablet_bootstrap.cc:654] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.631883 32125 tablet_bootstrap.cc:654] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.611965 32285 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.636979 32127 tablet_bootstrap.cc:492] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:23.637720 32127 ts_tablet_manager.cc:1403] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.003s
I20260709 13:16:23.647440 32285 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:23.648442 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:23.649042 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:23.649302 32003 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.649890 32075 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:23.652915 32123 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:23.653698 32123 ts_tablet_manager.cc:1403] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.003s
I20260709 13:16:23.669678 32188 consensus_queue.cc:1048] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:16:23.685014 32285 raft_consensus.cc:493] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.685972 32285 raft_consensus.cc:515] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.662955 32127 raft_consensus.cc:359] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.687966 32127 raft_consensus.cc:385] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.688803 32127 raft_consensus.cc:740] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.689800 32127 consensus_queue.cc:260] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:23.657480 32123 raft_consensus.cc:359] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.691052 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:23.672426 32075 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:23.680518 32003 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:23.691130 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:23.694347 31928 raft_consensus.cc:2393] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:23.694531 31821 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:23.694864 32127 ts_tablet_manager.cc:1434] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:16:23.695520 32181 raft_consensus.cc:2804] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:23.696031 31819 leader_election.cc:304] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:23.696610 32127 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:23.703426 32285 leader_election.cc:290] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:23.706558 32285 raft_consensus.cc:3060] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.707286 32127 tablet_bootstrap.cc:654] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.652913 32126 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:23.710911 32126 ts_tablet_manager.cc:1403] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.228s	user 0.014s	sys 0.010s
I20260709 13:16:23.716506 32182 consensus_queue.cc:1048] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:23.721729 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:23.691589 32123 raft_consensus.cc:385] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.724634 32123 raft_consensus.cc:740] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.681914 31750 catalog_manager.cc:5697] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.724954 32125 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:23.734828 32125 ts_tablet_manager.cc:1403] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.173s	user 0.010s	sys 0.010s
I20260709 13:16:23.736070 32126 raft_consensus.cc:359] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.738370 32126 raft_consensus.cc:385] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.738759 32126 raft_consensus.cc:740] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.726130 32123 consensus_queue.cc:260] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.739701 32126 consensus_queue.cc:260] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.740772 32181 raft_consensus.cc:697] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:23.746917 32181 consensus_queue.cc:237] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.753654 32126 ts_tablet_manager.cc:1434] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.000s
I20260709 13:16:23.754633 32285 raft_consensus.cc:2749] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.755499 32126 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:23.767460 32123 ts_tablet_manager.cc:1434] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.113s	user 0.010s	sys 0.000s
I20260709 13:16:23.774863 32123 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:23.779311 32125 raft_consensus.cc:359] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.780663 32125 raft_consensus.cc:385] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.781095 32125 raft_consensus.cc:740] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.780432 31751 catalog_manager.cc:5697] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.782729 32125 consensus_queue.cc:260] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.785460 32125 ts_tablet_manager.cc:1434] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:16:23.787248 32125 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:23.793133 32127 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:23.794116 32127 ts_tablet_manager.cc:1403] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.024s
I20260709 13:16:23.794176 32181 raft_consensus.cc:493] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.795077 32181 raft_consensus.cc:515] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.798501 32181 leader_election.cc:290] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:23.799219 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:23.800189 32003 raft_consensus.cc:2393] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:23.801520 32125 tablet_bootstrap.cc:654] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.808732 32126 tablet_bootstrap.cc:654] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.810173 32127 raft_consensus.cc:359] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.811506 32127 raft_consensus.cc:385] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.811817 32182 raft_consensus.cc:493] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.812402 32182 raft_consensus.cc:515] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.814368 32123 tablet_bootstrap.cc:654] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.815598 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77e32837dc3e4e7d99a94d477a4d50d1" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:23.819178 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247a65da5884caba020ed13be64352f" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:23.820077 32003 raft_consensus.cc:2468] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:23.817468 31820 leader_election.cc:304] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:23.821444 31896 leader_election.cc:304] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:23.822460 32318 raft_consensus.cc:3060] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.822882 32182 leader_election.cc:290] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:23.811839 32127 raft_consensus.cc:740] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.822511 32193 raft_consensus.cc:2804] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.826648 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247a65da5884caba020ed13be64352f" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:23.826397 32127 consensus_queue.cc:260] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:23.826253 32193 raft_consensus.cc:493] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:23.829137 31893 leader_election.cc:343] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.829644 32193 raft_consensus.cc:3060] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.831816 32127 ts_tablet_manager.cc:1434] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:16:23.842432 32127 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:23.834273 31752 catalog_manager.cc:5697] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 2, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.853693 32125 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:23.855428 32125 ts_tablet_manager.cc:1403] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.069s	user 0.017s	sys 0.003s
I20260709 13:16:23.857956 32318 raft_consensus.cc:2749] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.858742 32125 raft_consensus.cc:359] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.853688 32193 raft_consensus.cc:515] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.861188 32125 raft_consensus.cc:385] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.863667 32125 raft_consensus.cc:740] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.867542 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247a65da5884caba020ed13be64352f" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:23.867237 32125 consensus_queue.cc:260] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.868008 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247a65da5884caba020ed13be64352f" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:23.868438 31852 raft_consensus.cc:3060] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.866108 32193 leader_election.cc:290] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:23.869122 32003 raft_consensus.cc:3060] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.870147 32126 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:23.877548 32123 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:23.880271 31852 raft_consensus.cc:2468] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:23.880863 32123 ts_tablet_manager.cc:1403] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.106s	user 0.019s	sys 0.005s
I20260709 13:16:23.882102 32127 tablet_bootstrap.cc:654] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.884560 32123 raft_consensus.cc:359] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.885685 31893 leader_election.cc:304] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:23.886179 32123 raft_consensus.cc:385] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.886492 32123 raft_consensus.cc:740] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.886986 32193 raft_consensus.cc:2804] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.887290 32123 consensus_queue.cc:260] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.888219 32193 raft_consensus.cc:697] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:23.889335 32123 ts_tablet_manager.cc:1434] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.002s
I20260709 13:16:23.889642 32193 consensus_queue.cc:237] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.880873 32126 ts_tablet_manager.cc:1403] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.126s	user 0.017s	sys 0.019s
I20260709 13:16:23.898700 32123 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:23.898509 32126 raft_consensus.cc:359] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.899335 32126 raft_consensus.cc:385] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.899626 32126 raft_consensus.cc:740] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.900480 32126 consensus_queue.cc:260] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.903470 32126 ts_tablet_manager.cc:1434] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:23.880365 32003 raft_consensus.cc:2468] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:23.905956 32126 tablet_bootstrap.cc:492] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:23.907019 32193 raft_consensus.cc:493] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.908165 32123 tablet_bootstrap.cc:654] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.907754 32193 raft_consensus.cc:515] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.913133 32127 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:23.928817 32127 ts_tablet_manager.cc:1403] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.087s	user 0.018s	sys 0.008s
I20260709 13:16:23.915722 32126 tablet_bootstrap.cc:654] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.932417 32193 leader_election.cc:290] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:23.927769 32125 ts_tablet_manager.cc:1434] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.072s	user 0.020s	sys 0.002s
I20260709 13:16:23.935042 32125 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:23.936481 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:23.925484 31750 catalog_manager.cc:5697] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.936944 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:23.938197 31893 leader_election.cc:343] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.950434 32125 tablet_bootstrap.cc:654] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.952119 32188 consensus_queue.cc:1048] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.954620 31749 catalog_manager.cc:5697] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.956691 32075 raft_consensus.cc:2468] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:23.958351 31895 leader_election.cc:304] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:23.959758 32193 raft_consensus.cc:2804] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.960196 32193 raft_consensus.cc:493] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.960639 32193 raft_consensus.cc:3060] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.966598 32127 raft_consensus.cc:359] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.973704 32127 raft_consensus.cc:385] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.974630 32127 raft_consensus.cc:740] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.975687 32127 consensus_queue.cc:260] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:23.978211 32127 ts_tablet_manager.cc:1434] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 13:16:23.979631 32127 tablet_bootstrap.cc:492] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:23.973407 32193 raft_consensus.cc:515] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:23.994462 32123 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:23.995239 32297 consensus_queue.cc:1048] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.998435 32193 leader_election.cc:290] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:23.995260 32123 ts_tablet_manager.cc:1403] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.097s	user 0.018s	sys 0.004s
I20260709 13:16:24.002231 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:24.007854 32127 tablet_bootstrap.cc:654] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.008697 32077 raft_consensus.cc:3060] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.007078 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:24.010599 32125 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:24.011270 32125 ts_tablet_manager.cc:1403] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.005s
W20260709 13:16:24.012480 31893 leader_election.cc:343] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.017962 32123 raft_consensus.cc:359] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.018930 32123 raft_consensus.cc:385] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.019225 32123 raft_consensus.cc:740] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.020099 32123 consensus_queue.cc:260] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.022298 32188 raft_consensus.cc:493] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.023039 32188 raft_consensus.cc:515] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.025729 32297 consensus_queue.cc:1048] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:24.027048 32123 ts_tablet_manager.cc:1434] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:16:24.028389 32233 consensus_queue.cc:1048] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.034085 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2611ce5875fe4e55a870d70749bd1ec4" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:24.035032 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2611ce5875fe4e55a870d70749bd1ec4" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:24.034010 32188 leader_election.cc:290] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:16:24.035912 32044 leader_election.cc:343] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.025595 32125 raft_consensus.cc:359] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:24.036759 32041 leader_election.cc:343] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.037418 32041 leader_election.cc:304] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:24.036908 32125 raft_consensus.cc:385] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.038540 32188 raft_consensus.cc:2749] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.038825 32125 raft_consensus.cc:740] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.039958 32125 consensus_queue.cc:260] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.041198 32233 raft_consensus.cc:493] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.041874 32233 raft_consensus.cc:515] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:24.044869 32233 leader_election.cc:290] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:24.048739 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea199ae81974f50bda37760537988a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:24.050756 31967 leader_election.cc:343] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.051288 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea199ae81974f50bda37760537988a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:24.052101 31929 raft_consensus.cc:2468] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:24.052928 32188 consensus_queue.cc:1048] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.054421 31968 leader_election.cc:304] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:24.054972 32125 ts_tablet_manager.cc:1434] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:16:24.056052 32233 raft_consensus.cc:2804] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.056340 32125 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:24.056505 32233 raft_consensus.cc:493] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.057118 32233 raft_consensus.cc:3060] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.038439 32123 tablet_bootstrap.cc:492] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:24.101787 32182 consensus_queue.cc:1048] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:24.117780 32181 raft_consensus.cc:493] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.119553 32181 raft_consensus.cc:515] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.124320 32181 leader_election.cc:290] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:24.128119 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:24.129038 32078 raft_consensus.cc:2468] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:24.135604 32125 tablet_bootstrap.cc:654] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.163811 32127 tablet_bootstrap.cc:492] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:24.164513 32127 ts_tablet_manager.cc:1403] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.185s	user 0.026s	sys 0.031s
I20260709 13:16:24.171928 32182 consensus_queue.cc:1048] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:24.176141 32127 raft_consensus.cc:359] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.177052 32127 raft_consensus.cc:385] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.179976 31820 leader_election.cc:304] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:24.181310 32181 raft_consensus.cc:2804] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.181736 32181 raft_consensus.cc:493] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.182078 32181 raft_consensus.cc:3060] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.177340 32127 raft_consensus.cc:740] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.185278 32123 tablet_bootstrap.cc:654] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.186028 32077 raft_consensus.cc:2468] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:24.188892 32126 tablet_bootstrap.cc:492] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:24.190194 32126 ts_tablet_manager.cc:1403] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.285s	user 0.013s	sys 0.022s
I20260709 13:16:24.180115 32074 raft_consensus.cc:1275] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:24.205524 32127 consensus_queue.cc:260] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.205996 32318 consensus_queue.cc:1048] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.214377 32127 ts_tablet_manager.cc:1434] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.005s
I20260709 13:16:24.216034 32127 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:24.252177 31895 leader_election.cc:304] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:24.256399 32193 raft_consensus.cc:2804] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.245260 32126 raft_consensus.cc:359] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.259104 32126 raft_consensus.cc:385] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.260452 32126 raft_consensus.cc:740] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.262104 32193 raft_consensus.cc:697] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:24.265527 32193 consensus_queue.cc:237] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:24.272881 32282 raft_consensus.cc:493] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.274313 32282 raft_consensus.cc:515] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:24.275163 32340 raft_consensus.cc:493] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.275784 32340 raft_consensus.cc:515] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:24.235487 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 106035us
I20260709 13:16:24.244911 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:24.281724 32282 leader_election.cc:290] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:24.314930 32126 consensus_queue.cc:260] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.301968 32003 raft_consensus.cc:2468] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:24.328128 32340 leader_election.cc:290] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:24.328831 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:24.330313 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab0ae780de5843cea623c39fd867cc57" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:24.335796 32125 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:24.339816 32125 ts_tablet_manager.cc:1403] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.284s	user 0.029s	sys 0.012s
I20260709 13:16:24.290122 32233 raft_consensus.cc:515] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:24.344417 31969 leader_election.cc:343] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.322687 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
W20260709 13:16:24.352029 32042 leader_election.cc:343] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.336884 32181 raft_consensus.cc:515] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.388005 32126 ts_tablet_manager.cc:1434] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.197s	user 0.018s	sys 0.000s
I20260709 13:16:24.389074 32125 raft_consensus.cc:359] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:24.391055 32125 raft_consensus.cc:385] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.391443 32125 raft_consensus.cc:740] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.391055 32123 tablet_bootstrap.cc:492] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:24.394217 32123 ts_tablet_manager.cc:1403] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.356s	user 0.020s	sys 0.021s
I20260709 13:16:24.392220 32125 consensus_queue.cc:260] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:24.371507 32303 consensus_queue.cc:1048] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:16:24.384179 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab0ae780de5843cea623c39fd867cc57" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
W20260709 13:16:24.399581 32041 leader_election.cc:343] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.403899 32181 leader_election.cc:290] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:24.402958 32041 leader_election.cc:304] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:24.412228 31928 raft_consensus.cc:2468] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:24.419752 31968 leader_election.cc:304] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:24.421972 32340 raft_consensus.cc:2804] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.422451 32340 raft_consensus.cc:493] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.422786 32340 raft_consensus.cc:3060] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.427314 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea199ae81974f50bda37760537988a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:24.434134 32126 tablet_bootstrap.cc:492] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:24.398808 32182 consensus_queue.cc:1048] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:16:24.428310 31928 raft_consensus.cc:3060] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.411170 32004 raft_consensus.cc:1275] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:24.416049 32282 raft_consensus.cc:2749] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.421355 32233 leader_election.cc:290] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:24.454077 32123 raft_consensus.cc:359] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.454998 32123 raft_consensus.cc:385] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.455526 32123 raft_consensus.cc:740] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
W20260709 13:16:24.444757 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 46509.7us
I20260709 13:16:24.466547 32123 consensus_queue.cc:260] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.468750 32125 ts_tablet_manager.cc:1434] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.129s	user 0.029s	sys 0.031s
I20260709 13:16:24.480173 32127 tablet_bootstrap.cc:654] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.477291 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:24.485571 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea199ae81974f50bda37760537988a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:24.501271 32125 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:24.497484 32003 raft_consensus.cc:3060] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.517484 32123 ts_tablet_manager.cc:1434] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.003s
I20260709 13:16:24.513432 32318 consensus_queue.cc:1048] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:24.536638 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:24.538285 32077 raft_consensus.cc:3060] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:24.548919 31967 leader_election.cc:343] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.569579 32126 tablet_bootstrap.cc:654] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.582536 32077 raft_consensus.cc:2468] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:24.585918 32123 tablet_bootstrap.cc:492] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:24.590312 31750 catalog_manager.cc:5697] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.593758 32303 raft_consensus.cc:493] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.594419 32303 raft_consensus.cc:515] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.602473 32303 leader_election.cc:290] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:24.631562 31820 leader_election.cc:304] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:24.635829 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:24.656167 32346 raft_consensus.cc:2804] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.657672 32346 raft_consensus.cc:697] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:24.664475 32346 consensus_queue.cc:237] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.644408 32340 raft_consensus.cc:515] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.659060 31928 raft_consensus.cc:2468] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
W20260709 13:16:24.671247 31969 leader_election.cc:343] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.663436 32003 raft_consensus.cc:2468] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:24.662290 32182 consensus_queue.cc:1048] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:16:24.685247 32125 tablet_bootstrap.cc:654] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.687606 32275 raft_consensus.cc:493] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.689623 32275 raft_consensus.cc:515] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.692081 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab0ae780de5843cea623c39fd867cc57" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:24.681986 32340 leader_election.cc:290] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:24.695983 32075 raft_consensus.cc:3060] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.731226 32275 leader_election.cc:290] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:24.731622 32125 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:24.732411 32125 ts_tablet_manager.cc:1403] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.242s	user 0.022s	sys 0.005s
I20260709 13:16:24.733573 32123 tablet_bootstrap.cc:654] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.735137 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab0ae780de5843cea623c39fd867cc57" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:24.735955 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:24.736115 31927 raft_consensus.cc:2468] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:24.737663 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab0ae780de5843cea623c39fd867cc57" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:24.738302 31927 raft_consensus.cc:3060] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.739807 32125 raft_consensus.cc:359] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:24.740641 32125 raft_consensus.cc:385] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.741585 32042 leader_election.cc:304] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:24.740945 32125 raft_consensus.cc:740] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.748198 32125 consensus_queue.cc:260] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:24.749568 31968 leader_election.cc:304] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:24.751526 32125 ts_tablet_manager.cc:1434] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.002s
I20260709 13:16:24.753688 32125 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:24.759919 32126 tablet_bootstrap.cc:492] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:24.762208 32126 ts_tablet_manager.cc:1403] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.361s	user 0.018s	sys 0.028s
I20260709 13:16:24.759516 32340 raft_consensus.cc:2804] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.763935 32340 raft_consensus.cc:697] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:24.765084 32340 consensus_queue.cc:237] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:24.776245 31855 raft_consensus.cc:3060] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.786216 31927 raft_consensus.cc:2468] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:24.797399 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab0ae780de5843cea623c39fd867cc57" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:24.798454 32002 raft_consensus.cc:2393] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:24.798786 32127 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:24.799337 32127 ts_tablet_manager.cc:1403] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.584s	user 0.023s	sys 0.011s
I20260709 13:16:24.806749 32127 raft_consensus.cc:359] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.807703 32127 raft_consensus.cc:385] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.808097 32127 raft_consensus.cc:740] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.809262 32127 consensus_queue.cc:260] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.822865 32346 raft_consensus.cc:493] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.823552 32346 raft_consensus.cc:515] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:24.828732 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:24.834522 32182 consensus_queue.cc:1048] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:24.811863 31968 leader_election.cc:343] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.841450 32182 consensus_queue.cc:1048] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:24.843020 32123 tablet_bootstrap.cc:492] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:24.843371 32126 raft_consensus.cc:359] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.851164 32123 ts_tablet_manager.cc:1403] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.266s	user 0.023s	sys 0.004s
I20260709 13:16:24.851310 32126 raft_consensus.cc:385] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.851660 32126 raft_consensus.cc:740] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.850171 31968 leader_election.cc:304] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:24.852627 32126 consensus_queue.cc:260] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.837674 32346 leader_election.cc:290] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:24.854632 32123 raft_consensus.cc:359] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:24.855955 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 44555.7us
I20260709 13:16:24.857703 31968 leader_election.cc:304] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:24.861800 32303 raft_consensus.cc:2804] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.862445 32303 raft_consensus.cc:697] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:24.863971 32127 ts_tablet_manager.cc:1434] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 13:16:24.863735 32303 consensus_queue.cc:237] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.865388 32127 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:24.868228 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:24.871851 31820 leader_election.cc:304] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:24.856158 32126 ts_tablet_manager.cc:1434] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.003s
I20260709 13:16:24.875105 32126 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:24.858304 32075 raft_consensus.cc:2468] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:24.856190 32123 raft_consensus.cc:385] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.859028 32125 tablet_bootstrap.cc:654] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.893921 32123 raft_consensus.cc:740] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.894867 32123 consensus_queue.cc:260] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.856317 32340 raft_consensus.cc:2749] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.897642 32123 ts_tablet_manager.cc:1434] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260709 13:16:24.899016 32123 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:24.900036 32127 tablet_bootstrap.cc:654] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.913757 32282 raft_consensus.cc:2764] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:16:24.917122 32346 raft_consensus.cc:2749] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:24.924319 32123 tablet_bootstrap.cc:654] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.950587 32126 tablet_bootstrap.cc:654] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.968848 32125 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:24.969528 32125 ts_tablet_manager.cc:1403] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.217s	user 0.035s	sys 0.016s
I20260709 13:16:24.970394 31752 catalog_manager.cc:5697] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.974041 32125 raft_consensus.cc:359] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.974886 32125 raft_consensus.cc:385] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.975183 32125 raft_consensus.cc:740] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.975899 32125 consensus_queue.cc:260] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:24.978286 32125 ts_tablet_manager.cc:1434] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:16:24.979502 32125 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:24.986678 32125 tablet_bootstrap.cc:654] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.988140 32370 raft_consensus.cc:493] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.988834 32370 raft_consensus.cc:515] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:24.993930 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:24.996140 32346 raft_consensus.cc:493] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:24.996930 32042 leader_election.cc:343] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.997190 32346 raft_consensus.cc:515] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:24.999245 32181 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.999979 32181 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.001968 32346 leader_election.cc:290] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:25.002413 32181 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:25.002074 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea199ae81974f50bda37760537988a5" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:25.003461 31929 raft_consensus.cc:2393] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
I20260709 13:16:25.003765 32123 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:25.005774 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dea199ae81974f50bda37760537988a5" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.005950 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.009609 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:25.009691 31821 leader_election.cc:343] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.012192 31821 leader_election.cc:304] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:25.013908 32123 ts_tablet_manager.cc:1403] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.115s	user 0.011s	sys 0.009s
I20260709 13:16:25.015866 32370 leader_election.cc:290] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:25.018396 31820 leader_election.cc:343] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.019143 31820 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:25.021314 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.022375 32001 raft_consensus.cc:2468] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:25.013298 32126 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:25.024271 32126 ts_tablet_manager.cc:1403] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.150s	user 0.004s	sys 0.012s
I20260709 13:16:25.024045 32044 leader_election.cc:304] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:25.027048 32370 raft_consensus.cc:2804] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.027776 32370 raft_consensus.cc:493] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.028692 32370 raft_consensus.cc:3060] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.021409 32346 raft_consensus.cc:3060] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:25.040124 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:25.030581 32181 raft_consensus.cc:2749] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.031040 31749 catalog_manager.cc:5697] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:25.028980 32126 raft_consensus.cc:359] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.043694 32126 raft_consensus.cc:385] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.044024 32126 raft_consensus.cc:740] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.042627 32123 raft_consensus.cc:359] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.045045 32126 consensus_queue.cc:260] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.046459 32127 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:25.047158 32127 ts_tablet_manager.cc:1403] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.182s	user 0.032s	sys 0.024s
I20260709 13:16:25.049088 32126 ts_tablet_manager.cc:1434] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:16:25.050741 32126 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:25.046162 31749 catalog_manager.cc:5697] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.046373 32123 raft_consensus.cc:385] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.064790 32123 raft_consensus.cc:740] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.066036 32123 consensus_queue.cc:260] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.066198 32127 raft_consensus.cc:359] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.068755 32123 ts_tablet_manager.cc:1434] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 13:16:25.067154 32127 raft_consensus.cc:385] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.069943 32127 raft_consensus.cc:740] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.071305 32127 consensus_queue.cc:260] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.074620 32127 ts_tablet_manager.cc:1434] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:16:25.076984 32123 tablet_bootstrap.cc:492] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:25.088835 32123 tablet_bootstrap.cc:654] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.104103 32126 tablet_bootstrap.cc:654] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.108645 32346 raft_consensus.cc:2749] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.116927 32370 raft_consensus.cc:515] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.109697 32127 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:25.119587 32370 leader_election.cc:290] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:25.123195 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:25.123910 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:25.133701 32125 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:25.137143 32125 ts_tablet_manager.cc:1403] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.158s	user 0.049s	sys 0.050s
W20260709 13:16:25.137527 32042 leader_election.cc:343] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.133694 32126 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:25.138561 32126 ts_tablet_manager.cc:1403] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.088s	user 0.009s	sys 0.009s
I20260709 13:16:25.140439 32125 raft_consensus.cc:359] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.141248 32125 raft_consensus.cc:385] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.141884 32001 raft_consensus.cc:3060] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.141970 32126 raft_consensus.cc:359] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.142629 32127 tablet_bootstrap.cc:654] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.142899 32125 raft_consensus.cc:740] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.146653 32125 consensus_queue.cc:260] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.150766 32125 ts_tablet_manager.cc:1434] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:16:25.160883 32125 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:25.155974 32127 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:25.156121 32379 consensus_queue.cc:1048] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.163625 32127 ts_tablet_manager.cc:1403] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.007s
I20260709 13:16:25.165369 32355 raft_consensus.cc:493] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.166033 32355 raft_consensus.cc:515] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.167629 32001 raft_consensus.cc:2468] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:25.142958 32126 raft_consensus.cc:385] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.169663 32044 leader_election.cc:304] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:25.170315 32126 raft_consensus.cc:740] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.171263 32370 raft_consensus.cc:2804] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.171895 32370 raft_consensus.cc:697] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:25.172083 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:25.172901 32370 consensus_queue.cc:237] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.173849 32355 leader_election.cc:290] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:25.174463 32123 tablet_bootstrap.cc:492] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
W20260709 13:16:25.174626 31895 leader_election.cc:343] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.175813 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:25.177975 31896 leader_election.cc:343] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.178544 31896 leader_election.cc:304] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:25.180094 32355 raft_consensus.cc:2749] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.181414 32126 consensus_queue.cc:260] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.175038 32123 ts_tablet_manager.cc:1403] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.010s
I20260709 13:16:25.183532 32127 raft_consensus.cc:359] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.184829 32127 raft_consensus.cc:385] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.185388 32127 raft_consensus.cc:740] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.186509 32127 consensus_queue.cc:260] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.189741 32127 ts_tablet_manager.cc:1434] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:16:25.194173 32126 ts_tablet_manager.cc:1434] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.002s
I20260709 13:16:25.195708 32126 tablet_bootstrap.cc:492] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:25.199994 32123 raft_consensus.cc:359] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.180451 32125 tablet_bootstrap.cc:654] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.200881 32127 tablet_bootstrap.cc:492] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:25.200938 32123 raft_consensus.cc:385] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.204311 32123 raft_consensus.cc:740] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.207882 32127 tablet_bootstrap.cc:654] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.213848 32123 consensus_queue.cc:260] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.216485 32123 ts_tablet_manager.cc:1434] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.002s
I20260709 13:16:25.225154 32340 consensus_queue.cc:1048] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.227202 32126 tablet_bootstrap.cc:654] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.240502 32379 consensus_queue.cc:1048] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.246102 32125 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:25.246631 32123 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:25.246793 32125 ts_tablet_manager.cc:1403] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.003s
I20260709 13:16:25.245069 31749 catalog_manager.cc:5697] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.256086 32127 tablet_bootstrap.cc:492] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:25.256801 32127 ts_tablet_manager.cc:1403] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.004s
I20260709 13:16:25.261997 32370 raft_consensus.cc:493] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.262673 32370 raft_consensus.cc:515] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.263824 32123 tablet_bootstrap.cc:654] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.267505 32125 raft_consensus.cc:359] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.268440 32125 raft_consensus.cc:385] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.268718 32125 raft_consensus.cc:740] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.269483 32125 consensus_queue.cc:260] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.270933 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:25.272004 32125 ts_tablet_manager.cc:1434] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:16:25.272857 32370 leader_election.cc:290] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:25.273056 32127 raft_consensus.cc:359] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.275189 32127 raft_consensus.cc:385] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.277864 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.278759 32126 tablet_bootstrap.cc:492] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:25.279307 32126 ts_tablet_manager.cc:1403] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.004s
I20260709 13:16:25.272280 31929 raft_consensus.cc:2468] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:25.282577 32252 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.283320 32252 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.287344 32042 leader_election.cc:304] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:25.287329 32126 raft_consensus.cc:359] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.288514 32282 raft_consensus.cc:2804] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.288875 32126 raft_consensus.cc:385] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.289041 32282 raft_consensus.cc:493] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.289745 32282 raft_consensus.cc:3060] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.289269 32126 raft_consensus.cc:740] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.291226 32126 consensus_queue.cc:260] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.275640 32127 raft_consensus.cc:740] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.294296 32127 consensus_queue.cc:260] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.295722 32126 ts_tablet_manager.cc:1434] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:16:25.295855 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:25.296452 32004 raft_consensus.cc:2468] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:25.297191 32126 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
W20260709 13:16:25.298485 31895 leader_election.cc:343] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.304942 32126 tablet_bootstrap.cc:654] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.306818 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:25.308909 31893 leader_election.cc:343] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.309445 31893 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:25.310487 32355 raft_consensus.cc:2749] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.312280 32252 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.314227 32125 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:25.321736 32127 ts_tablet_manager.cc:1434] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.005s
I20260709 13:16:25.323408 32127 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:25.326745 32297 raft_consensus.cc:493] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.324424 32123 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:25.327615 32123 ts_tablet_manager.cc:1403] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.011s
I20260709 13:16:25.327411 32297 raft_consensus.cc:515] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.331547 32123 raft_consensus.cc:359] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.332415 32123 raft_consensus.cc:385] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.332738 32123 raft_consensus.cc:740] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.333549 32123 consensus_queue.cc:260] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.334146 32297 leader_election.cc:290] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.336037 32123 ts_tablet_manager.cc:1434] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:25.338337 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2611ce5875fe4e55a870d70749bd1ec4" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:25.339402 31855 raft_consensus.cc:2468] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:25.341655 31967 leader_election.cc:304] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:25.343017 32348 raft_consensus.cc:2804] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.343566 32348 raft_consensus.cc:493] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.343976 32348 raft_consensus.cc:3060] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.346977 32340 consensus_queue.cc:1048] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.342111 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2611ce5875fe4e55a870d70749bd1ec4" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:25.349355 32076 raft_consensus.cc:2468] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:25.337689 32123 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:25.365947 32297 consensus_queue.cc:1048] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.390982 32125 tablet_bootstrap.cc:654] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.400923 32275 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.401643 32275 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.404045 32126 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:25.405040 32126 ts_tablet_manager.cc:1403] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.108s	user 0.031s	sys 0.073s
I20260709 13:16:25.407362 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.408641 32001 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:25.409953 32044 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:25.410986 32370 raft_consensus.cc:2804] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.411418 32370 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.411791 32370 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.412638 32297 raft_consensus.cc:493] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.413470 32297 raft_consensus.cc:515] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.416579 32297 leader_election.cc:290] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:25.417881 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:25.417918 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:25.420630 31967 leader_election.cc:343] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.432402 32123 tablet_bootstrap.cc:654] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.433874 32348 raft_consensus.cc:515] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.435084 32340 consensus_queue.cc:1048] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.436205 32348 leader_election.cc:290] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.436671 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:25.437493 31855 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:25.438994 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2611ce5875fe4e55a870d70749bd1ec4" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:25.439687 31852 raft_consensus.cc:3060] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.442981 32370 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.443414 32126 raft_consensus.cc:359] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.444284 32126 raft_consensus.cc:385] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.444667 32126 raft_consensus.cc:740] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.447506 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:25.447328 32370 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.448230 31851 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.448442 32282 raft_consensus.cc:515] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.445391 32126 consensus_queue.cc:260] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.454070 32355 raft_consensus.cc:493] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.454593 32355 raft_consensus.cc:515] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.459259 32126 ts_tablet_manager.cc:1434] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.054s	user 0.010s	sys 0.000s
I20260709 13:16:25.465049 32127 tablet_bootstrap.cc:654] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.465967 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2611ce5875fe4e55a870d70749bd1ec4" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:25.467219 32076 raft_consensus.cc:3060] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:25.471298 31968 leader_election.cc:343] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.471949 31968 leader_election.cc:304] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:25.472812 32340 raft_consensus.cc:2749] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.473930 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:25.474932 32001 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.476495 32252 raft_consensus.cc:493] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.477337 32252 raft_consensus.cc:515] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.441288 32275 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.497120 32126 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:25.514863 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.516307 32004 raft_consensus.cc:2393] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:25.492336 32252 leader_election.cc:290] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.502558 32355 leader_election.cc:290] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:25.494076 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:25.538164 32282 leader_election.cc:290] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:25.530581 32075 raft_consensus.cc:2468] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:25.552943 31852 raft_consensus.cc:2468] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:25.558017 31851 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:25.558276 31967 leader_election.cc:304] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:25.560350 32340 raft_consensus.cc:2804] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.561324 32340 raft_consensus.cc:697] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:25.564484 32126 tablet_bootstrap.cc:654] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.566991 32340 consensus_queue.cc:237] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.581928 32125 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:25.583027 32125 ts_tablet_manager.cc:1403] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.269s	user 0.038s	sys 0.056s
I20260709 13:16:25.576879 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:25.588690 32076 raft_consensus.cc:2468] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:25.589449 32001 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:25.590642 31852 raft_consensus.cc:2468] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:25.592682 32041 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:25.593281 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372dd26a25ab4cf686e6c74fcc9b6bb6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:25.599227 31893 leader_election.cc:304] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:25.600090 32282 raft_consensus.cc:2804] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.600713 32282 raft_consensus.cc:697] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:25.600638 32396 raft_consensus.cc:2804] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.601707 32396 raft_consensus.cc:493] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.602183 32396 raft_consensus.cc:3060] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.602162 32282 consensus_queue.cc:237] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:25.621942 32395 raft_consensus.cc:493] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.623191 32395 raft_consensus.cc:515] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.632277 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:25.633100 31927 raft_consensus.cc:3060] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.664410 32126 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:25.665040 32126 ts_tablet_manager.cc:1403] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.177s	user 0.047s	sys 0.038s
I20260709 13:16:25.670306 32395 leader_election.cc:290] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:25.671811 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:25.672596 32004 raft_consensus.cc:2468] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:25.673600 32125 raft_consensus.cc:359] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.674998 32125 raft_consensus.cc:385] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.675153 31896 leader_election.cc:304] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:25.675437 32125 raft_consensus.cc:740] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.677671 32355 raft_consensus.cc:2804] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.678144 32355 raft_consensus.cc:493] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.678479 32355 raft_consensus.cc:3060] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.682777 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:25.684744 31893 leader_election.cc:343] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:25.661046 32397 log.cc:927] Time spent T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:16:25.661136 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 64907.9us
I20260709 13:16:25.676877 32125 consensus_queue.cc:260] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.693181 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:25.695008 32004 raft_consensus.cc:3060] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.696595 32125 ts_tablet_manager.cc:1434] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.113s	user 0.012s	sys 0.000s
I20260709 13:16:25.697069 32370 consensus_queue.cc:1048] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.699986 32396 raft_consensus.cc:515] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.716619 31927 raft_consensus.cc:2468] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:25.684942 32126 raft_consensus.cc:359] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.718883 32126 raft_consensus.cc:385] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.719237 32126 raft_consensus.cc:740] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.720749 32396 leader_election.cc:290] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:25.739477 32126 consensus_queue.cc:260] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.742029 32126 ts_tablet_manager.cc:1434] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.060s	user 0.011s	sys 0.010s
I20260709 13:16:25.758520 32125 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:25.760066 32126 tablet_bootstrap.cc:492] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:25.762892 32004 raft_consensus.cc:2468] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:25.751762 32370 raft_consensus.cc:493] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.776455 31751 catalog_manager.cc:5697] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:25.779354 32127 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:25.779959 32127 ts_tablet_manager.cc:1403] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.457s	user 0.018s	sys 0.005s
I20260709 13:16:25.789661 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:25.767685 32370 raft_consensus.cc:515] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.795765 31852 raft_consensus.cc:3060] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.823316 32044 leader_election.cc:304] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:25.832129 32275 raft_consensus.cc:2804] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.850417 32275 raft_consensus.cc:697] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:25.851486 32275 consensus_queue.cc:237] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:25.822333 31895 leader_election.cc:304] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:25.861519 32395 raft_consensus.cc:3060] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.868881 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:25.870047 32077 raft_consensus.cc:3060] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.875979 32127 raft_consensus.cc:359] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.877163 32127 raft_consensus.cc:385] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.879575 32370 leader_election.cc:290] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:25.881467 32125 tablet_bootstrap.cc:654] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.881898 32127 raft_consensus.cc:740] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.883726 32127 consensus_queue.cc:260] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.886502 32077 raft_consensus.cc:2468] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:25.887096 32127 ts_tablet_manager.cc:1434] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.107s	user 0.009s	sys 0.001s
I20260709 13:16:25.897754 32127 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:25.886790 32355 raft_consensus.cc:515] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:25.898157 31749 catalog_manager.cc:5697] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:25.894884 32126 tablet_bootstrap.cc:654] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.887668 31852 raft_consensus.cc:2468] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:25.934217 32181 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.937224 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 71162us
I20260709 13:16:25.948282 32181 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:25.964434 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:25.964061 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:25.976068 32127 tablet_bootstrap.cc:654] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.976562 32123 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:25.977046 32123 ts_tablet_manager.cc:1403] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.640s	user 0.035s	sys 0.035s
W20260709 13:16:25.988256 32041 leader_election.cc:343] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.997438 32181 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:25.999367 31895 leader_election.cc:304] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
W20260709 13:16:25.959009 31896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 81271.7us
I20260709 13:16:25.999444 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:26.002631 31927 raft_consensus.cc:2468] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:26.007062 32404 raft_consensus.cc:2804] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.007865 32404 raft_consensus.cc:697] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:26.008605 31819 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:26.011862 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:26.012843 32002 raft_consensus.cc:3060] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.011902 32318 raft_consensus.cc:2804] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.014619 32318 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.015040 32318 raft_consensus.cc:3060] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:26.022161 32042 leader_election.cc:343] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.022672 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:26.022790 32042 leader_election.cc:304] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:26.017844 32395 raft_consensus.cc:2749] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.028491 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:26.005803 32355 leader_election.cc:290] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:26.033854 32282 raft_consensus.cc:2749] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:26.042702 31893 leader_election.cc:343] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.008886 32404 consensus_queue.cc:237] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.046533 32297 consensus_queue.cc:1048] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.050226 32123 raft_consensus.cc:359] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.051138 32123 raft_consensus.cc:385] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.051419 32123 raft_consensus.cc:740] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.052317 32125 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:26.052990 32125 ts_tablet_manager.cc:1403] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.301s	user 0.014s	sys 0.004s
I20260709 13:16:26.052129 32123 consensus_queue.cc:260] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.071827 32282 consensus_queue.cc:1048] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:16:26.078740 31820 leader_election.cc:343] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.094756 32002 raft_consensus.cc:2468] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:26.097798 31896 leader_election.cc:304] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:26.100183 32355 raft_consensus.cc:2804] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.100988 32355 raft_consensus.cc:697] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:26.108024 32282 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.108876 32282 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.109346 32355 consensus_queue.cc:237] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.113325 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:26.096954 32125 raft_consensus.cc:359] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.116686 32125 raft_consensus.cc:385] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.117161 32125 raft_consensus.cc:740] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.119963 32125 consensus_queue.cc:260] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.128301 31753 catalog_manager.cc:5697] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:26.116876 32282 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:26.132477 32123 ts_tablet_manager.cc:1434] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.155s	user 0.013s	sys 0.015s
I20260709 13:16:26.118510 32318 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.143726 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:26.144485 31927 raft_consensus.cc:2468] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:26.147655 32419 consensus_queue.cc:1048] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.149861 31851 raft_consensus.cc:2393] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:26.151206 32125 ts_tablet_manager.cc:1434] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.098s	user 0.008s	sys 0.004s
I20260709 13:16:26.152427 32125 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:26.153726 32127 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:26.154654 32123 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:26.159301 32126 tablet_bootstrap.cc:492] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:26.160288 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:26.161084 31927 raft_consensus.cc:3060] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.159973 32126 ts_tablet_manager.cc:1403] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.408s	user 0.019s	sys 0.031s
I20260709 13:16:26.160243 32318 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:26.172008 32042 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:26.173893 32282 raft_consensus.cc:2804] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.174382 32282 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.174712 32282 raft_consensus.cc:3060] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.179623 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:26.181704 32123 tablet_bootstrap.cc:654] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.185762 32127 ts_tablet_manager.cc:1403] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.275s	user 0.009s	sys 0.027s
I20260709 13:16:26.193907 32297 consensus_queue.cc:1048] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.205446 32408 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.206384 32408 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.206254 32127 raft_consensus.cc:359] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.211294 32127 raft_consensus.cc:385] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.211704 32127 raft_consensus.cc:740] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.213447 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:26.215160 32127 consensus_queue.cc:260] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:26.215999 31893 leader_election.cc:343] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.218376 32408 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:26.219790 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:26.216327 32125 tablet_bootstrap.cc:654] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:26.224123 31896 leader_election.cc:343] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.224306 32370 consensus_queue.cc:1048] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.224910 31896 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:26.231880 32355 raft_consensus.cc:2749] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.222925 32126 raft_consensus.cc:359] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.234251 32126 raft_consensus.cc:385] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.234589 32126 raft_consensus.cc:740] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.232090 31752 catalog_manager.cc:5697] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:26.236469 32127 ts_tablet_manager.cc:1434] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:16:26.237746 32127 tablet_bootstrap.cc:492] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:26.247622 32370 consensus_queue.cc:1048] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.249220 32125 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:26.249862 32125 ts_tablet_manager.cc:1403] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.005s
I20260709 13:16:26.251461 32282 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.253057 32074 raft_consensus.cc:2393] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate 32690b3987094d51bb4f214321a29067 in this term.
I20260709 13:16:26.264634 32126 consensus_queue.cc:260] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.268139 32126 ts_tablet_manager.cc:1434] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.106s	user 0.006s	sys 0.001s
I20260709 13:16:26.270586 32282 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:26.271402 32123 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:26.273069 32123 ts_tablet_manager.cc:1403] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.119s	user 0.026s	sys 0.005s
I20260709 13:16:26.276949 32123 raft_consensus.cc:359] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.277813 32123 raft_consensus.cc:385] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.278316 32123 raft_consensus.cc:740] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.279554 32127 tablet_bootstrap.cc:654] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.282961 32123 consensus_queue.cc:260] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.277865 32297 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.197260 31927 raft_consensus.cc:2468] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:26.297051 32181 raft_consensus.cc:493] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.326462 32181 raft_consensus.cc:515] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.299088 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:26.329588 31854 raft_consensus.cc:2393] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:26.329739 32181 leader_election.cc:290] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:26.304713 32297 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:26.305714 32424 log.cc:927] Time spent T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: Append to log took a long time: real 0.073s	user 0.003s	sys 0.000s
W20260709 13:16:26.306284 32423 log.cc:927] Time spent T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:16:26.305784 31751 catalog_manager.cc:5697] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.309993 31928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:26.340306 31928 raft_consensus.cc:2393] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:26.341204 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:26.341735 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:26.342795 32002 raft_consensus.cc:2393] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:26.343851 31819 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:26.293815 32125 raft_consensus.cc:359] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.344717 32125 raft_consensus.cc:385] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.345057 32125 raft_consensus.cc:740] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.346015 32125 consensus_queue.cc:260] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.347533 32126 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:26.347316 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:26.347550 32318 raft_consensus.cc:2804] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.348420 32318 raft_consensus.cc:697] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:26.348662 32125 ts_tablet_manager.cc:1434] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.000s
I20260709 13:16:26.349375 32318 consensus_queue.cc:237] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.332813 32123 ts_tablet_manager.cc:1434] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:16:26.338663 32297 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:26.351706 31821 leader_election.cc:304] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:26.365163 32125 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:26.362502 31927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:26.359426 32041 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
W20260709 13:16:26.363149 31967 leader_election.cc:343] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.369132 32181 raft_consensus.cc:3060] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.394974 32275 raft_consensus.cc:2749] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.401154 32126 tablet_bootstrap.cc:654] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.412235 32123 tablet_bootstrap.cc:492] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:26.418061 32181 raft_consensus.cc:2749] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:26.431313 31968 leader_election.cc:343] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.451355 31968 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:26.453272 32400 raft_consensus.cc:2749] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.454885 32127 tablet_bootstrap.cc:492] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:26.455583 32127 ts_tablet_manager.cc:1403] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.218s	user 0.018s	sys 0.002s
I20260709 13:16:26.460474 32404 consensus_queue.cc:1048] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.465806 32125 tablet_bootstrap.cc:654] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.458350 32123 tablet_bootstrap.cc:654] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.465013 32127 raft_consensus.cc:359] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.467257 32127 raft_consensus.cc:385] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.467552 32127 raft_consensus.cc:740] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.475612 32415 consensus_queue.cc:1048] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:26.481649 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:26.471184 32127 consensus_queue.cc:260] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.494412 32127 ts_tablet_manager.cc:1434] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 13:16:26.495682 32127 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:26.496766 32126 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:26.498515 32404 raft_consensus.cc:493] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.499348 32404 raft_consensus.cc:515] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.500633 32125 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:26.501264 32125 ts_tablet_manager.cc:1403] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.001s
I20260709 13:16:26.504925 32127 tablet_bootstrap.cc:654] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.523211 32355 consensus_queue.cc:1048] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.511842 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716630c6a4e249aeb96ef3ac276290c8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:26.514798 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716630c6a4e249aeb96ef3ac276290c8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:26.515439 32126 ts_tablet_manager.cc:1403] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.010s
I20260709 13:16:26.515973 32404 leader_election.cc:290] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:26.527110 31749 catalog_manager.cc:5697] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.543147 32123 tablet_bootstrap.cc:492] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
W20260709 13:16:26.543828 31895 leader_election.cc:343] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.549243 32126 raft_consensus.cc:359] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.550279 32126 raft_consensus.cc:385] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.550624 32126 raft_consensus.cc:740] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.524909 32004 raft_consensus.cc:2468] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:26.551396 32126 consensus_queue.cc:260] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.554809 32126 ts_tablet_manager.cc:1434] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:16:26.556226 32126 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:26.543860 32123 ts_tablet_manager.cc:1403] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.176s	user 0.028s	sys 0.011s
I20260709 13:16:26.536487 32125 raft_consensus.cc:359] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.562127 32125 raft_consensus.cc:385] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.562469 32125 raft_consensus.cc:740] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.568238 31896 leader_election.cc:304] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:26.568132 32125 consensus_queue.cc:260] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.572605 32125 ts_tablet_manager.cc:1434] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.002s
I20260709 13:16:26.572558 32404 raft_consensus.cc:2804] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.574714 32404 raft_consensus.cc:493] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.575183 32404 raft_consensus.cc:3060] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.576694 32123 raft_consensus.cc:359] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.580209 32125 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:26.586441 32123 raft_consensus.cc:385] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.587023 32123 raft_consensus.cc:740] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.588125 32123 consensus_queue.cc:260] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.580752 32126 tablet_bootstrap.cc:654] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.596191 32123 ts_tablet_manager.cc:1434] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 13:16:26.599334 32125 tablet_bootstrap.cc:654] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.614430 32123 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:26.617146 32127 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:26.617251 32126 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:26.618064 32127 ts_tablet_manager.cc:1403] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.123s	user 0.005s	sys 0.018s
I20260709 13:16:26.618162 32126 ts_tablet_manager.cc:1403] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.014s
I20260709 13:16:26.626389 32127 raft_consensus.cc:359] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.627844 32127 raft_consensus.cc:385] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.628211 32127 raft_consensus.cc:740] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.628695 32415 raft_consensus.cc:493] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.629648 32415 raft_consensus.cc:515] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.633988 32127 consensus_queue.cc:260] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.638054 32127 ts_tablet_manager.cc:1434] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:16:26.643226 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b97d68a6e314f6792e4c8bfa99d3025" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:26.644049 32002 raft_consensus.cc:2468] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:26.646090 32044 leader_election.cc:304] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:26.646376 32415 leader_election.cc:290] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:26.647056 32318 raft_consensus.cc:493] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.651664 32126 raft_consensus.cc:359] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.653004 32181 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.653090 32126 raft_consensus.cc:385] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.653836 32126 raft_consensus.cc:740] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.653716 32181 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.654655 32126 consensus_queue.cc:260] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.657339 32181 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:26.650213 32318 raft_consensus.cc:515] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.657778 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:26.660022 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:26.661893 31929 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:26.663022 32318 leader_election.cc:290] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:26.647224 32275 raft_consensus.cc:2804] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.663918 32275 raft_consensus.cc:493] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.664223 32126 ts_tablet_manager.cc:1434] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:16:26.664369 32275 raft_consensus.cc:3060] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.665642 32126 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:26.665812 31819 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:26.667246 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:26.668110 31929 raft_consensus.cc:2468] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:26.669854 31819 leader_election.cc:304] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:26.661360 32002 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:26.671731 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:26.672551 32076 raft_consensus.cc:2468] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:26.678467 32396 consensus_queue.cc:1048] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.673830 31854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b97d68a6e314f6792e4c8bfa99d3025" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:26.679601 32355 consensus_queue.cc:1048] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.673295 32181 raft_consensus.cc:2804] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.682377 32181 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.682777 32181 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.683300 32127 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:26.691617 32318 raft_consensus.cc:2804] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.693930 32126 tablet_bootstrap.cc:654] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.695364 32123 tablet_bootstrap.cc:654] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.696321 32125 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:26.697068 32125 ts_tablet_manager.cc:1403] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.007s
I20260709 13:16:26.698254 32404 raft_consensus.cc:515] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.703169 32275 raft_consensus.cc:515] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.708316 32125 raft_consensus.cc:359] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.711074 32125 raft_consensus.cc:385] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.711984 32125 raft_consensus.cc:740] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.692195 32318 raft_consensus.cc:493] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.716681 32125 consensus_queue.cc:260] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.709606 32181 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.735467 32181 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:26.736398 32041 leader_election.cc:343] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.737536 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:26.738344 31929 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.746781 32127 tablet_bootstrap.cc:654] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.721225 32318 raft_consensus.cc:3060] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.747399 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716630c6a4e249aeb96ef3ac276290c8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:26.747080 32396 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.767094 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b97d68a6e314f6792e4c8bfa99d3025" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:26.767977 32003 raft_consensus.cc:3060] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.767779 32002 raft_consensus.cc:3060] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.766944 32396 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.773185 32000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:26.774149 32000 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.779422 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:26.780650 31999 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:26.784585 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:26.788359 31893 leader_election.cc:343] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.780112 32275 leader_election.cc:290] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:26.774044 32396 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:26.800505 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b97d68a6e314f6792e4c8bfa99d3025" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:26.801224 31851 raft_consensus.cc:3060] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.807941 32379 consensus_queue.cc:1048] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.799719 32123 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:26.813205 32123 ts_tablet_manager.cc:1403] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.199s	user 0.022s	sys 0.003s
I20260709 13:16:26.827193 32125 ts_tablet_manager.cc:1434] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.130s	user 0.020s	sys 0.026s
I20260709 13:16:26.845441 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "716630c6a4e249aeb96ef3ac276290c8" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:26.846390 32076 raft_consensus.cc:3060] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.860610 32003 raft_consensus.cc:2468] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:26.861631 31896 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:26.867993 32125 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:26.871039 32379 consensus_queue.cc:1048] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.874776 32044 leader_election.cc:304] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:26.818634 32404 leader_election.cc:290] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:26.880116 32126 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:26.880759 32126 ts_tablet_manager.cc:1403] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.215s	user 0.014s	sys 0.006s
I20260709 13:16:26.883790 32415 raft_consensus.cc:2804] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.884635 32415 raft_consensus.cc:697] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:26.882221 32396 raft_consensus.cc:2804] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:16:26.884334 31896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 62816.4us
I20260709 13:16:26.885969 32396 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.886437 32396 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.891865 32415 consensus_queue.cc:237] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.895912 32002 raft_consensus.cc:2468] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:26.897483 32076 raft_consensus.cc:2468] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:26.899749 31895 leader_election.cc:304] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:26.903398 32355 raft_consensus.cc:2804] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.903942 32355 raft_consensus.cc:697] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:26.907295 32127 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:26.907896 32127 ts_tablet_manager.cc:1403] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.231s	user 0.045s	sys 0.027s
I20260709 13:16:26.914052 32126 raft_consensus.cc:359] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.915194 32126 raft_consensus.cc:385] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.915604 32126 raft_consensus.cc:740] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.917382 31851 raft_consensus.cc:2468] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:26.917459 32318 raft_consensus.cc:515] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.904856 32355 consensus_queue.cc:237] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:26.921325 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:26.922233 31930 raft_consensus.cc:3060] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.912312 32000 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:26.924999 31821 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:26.925987 32448 raft_consensus.cc:2804] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.926549 32448 raft_consensus.cc:697] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:26.927017 32126 consensus_queue.cc:260] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.928825 32125 tablet_bootstrap.cc:654] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.928565 32318 leader_election.cc:290] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:26.944183 32126 ts_tablet_manager.cc:1434] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:16:26.946756 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:26.947844 32076 raft_consensus.cc:3060] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.956784 32126 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:26.963057 32396 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.963791 32448 consensus_queue.cc:237] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:26.966993 32396 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:26.967806 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
W20260709 13:16:26.969269 31893 leader_election.cc:343] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.970992 32127 raft_consensus.cc:359] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.971522 32000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:26.972069 32127 raft_consensus.cc:385] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.972268 32000 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.972412 32127 raft_consensus.cc:740] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.976390 31929 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:26.987433 32123 raft_consensus.cc:359] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.990337 32123 raft_consensus.cc:385] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.973398 32127 consensus_queue.cc:260] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:26.990888 32123 raft_consensus.cc:740] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.996191 32123 consensus_queue.cc:260] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.997598 32395 raft_consensus.cc:493] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.998286 32395 raft_consensus.cc:515] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:26.999238 32127 ts_tablet_manager.cc:1434] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.091s	user 0.010s	sys 0.001s
I20260709 13:16:27.000850 32395 leader_election.cc:290] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.005052 32123 ts_tablet_manager.cc:1434] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.191s	user 0.007s	sys 0.002s
I20260709 13:16:27.007915 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90a6b486bef42019735b76ff4da04a1" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:27.008236 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90a6b486bef42019735b76ff4da04a1" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:27.008997 32077 raft_consensus.cc:2468] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
W20260709 13:16:27.009889 31893 leader_election.cc:343] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.013360 31895 leader_election.cc:304] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:27.016827 32000 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:27.016952 32126 tablet_bootstrap.cc:654] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.028451 32127 tablet_bootstrap.cc:492] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:27.015729 32396 raft_consensus.cc:2804] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.026516 31930 raft_consensus.cc:2468] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:27.042629 32396 raft_consensus.cc:493] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.060173 31752 catalog_manager.cc:5697] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.077059 32123 tablet_bootstrap.cc:492] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:27.072184 32396 raft_consensus.cc:3060] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.093169 32076 raft_consensus.cc:2468] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:27.063285 31896 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
W20260709 13:16:27.098420 31896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 48416us
I20260709 13:16:27.099246 31819 leader_election.cc:304] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:27.104113 32181 raft_consensus.cc:2804] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.104713 32181 raft_consensus.cc:697] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:27.105659 32181 consensus_queue.cc:237] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.106665 32404 raft_consensus.cc:2804] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.107292 32404 raft_consensus.cc:697] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:27.108245 32404 consensus_queue.cc:237] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:27.145660 31821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 68484.8us
I20260709 13:16:27.173697 32125 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:27.174587 32125 ts_tablet_manager.cc:1403] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.307s	user 0.046s	sys 0.032s
I20260709 13:16:27.178395 32125 raft_consensus.cc:359] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.194171 32125 raft_consensus.cc:385] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.194471 32125 raft_consensus.cc:740] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.177485 32127 tablet_bootstrap.cc:654] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.200783 32125 consensus_queue.cc:260] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.202757 32125 ts_tablet_manager.cc:1434] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:16:27.210795 32125 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:27.236092 32125 tablet_bootstrap.cc:654] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.242347 31753 catalog_manager.cc:5697] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.246872 32396 raft_consensus.cc:515] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.250542 32123 tablet_bootstrap.cc:654] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.249655 32396 leader_election.cc:290] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.252585 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90a6b486bef42019735b76ff4da04a1" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:27.253795 32077 raft_consensus.cc:3060] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.257802 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90a6b486bef42019735b76ff4da04a1" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
W20260709 13:16:27.259717 31893 leader_election.cc:343] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.264616 32126 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:27.265303 32126 ts_tablet_manager.cc:1403] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.320s	user 0.015s	sys 0.004s
I20260709 13:16:27.271715 32077 raft_consensus.cc:2468] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:27.268572 32126 raft_consensus.cc:359] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.275717 32126 raft_consensus.cc:385] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.276116 32126 raft_consensus.cc:740] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.276997 32126 consensus_queue.cc:260] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.279844 32126 ts_tablet_manager.cc:1434] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:16:27.272584 32127 tablet_bootstrap.cc:492] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:27.275882 31895 leader_election.cc:304] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:27.282228 32127 ts_tablet_manager.cc:1403] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.275s	user 0.023s	sys 0.010s
I20260709 13:16:27.282714 32125 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:27.283324 32125 ts_tablet_manager.cc:1403] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.002s
I20260709 13:16:27.284294 32126 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:27.286646 32404 raft_consensus.cc:2804] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.286733 32125 raft_consensus.cc:359] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.287578 32404 raft_consensus.cc:697] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:27.289000 32404 consensus_queue.cc:237] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.287741 32125 raft_consensus.cc:385] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.291246 32125 raft_consensus.cc:740] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.292429 32127 raft_consensus.cc:359] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.292730 32123 tablet_bootstrap.cc:492] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:27.293298 32125 consensus_queue.cc:260] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.293905 32123 ts_tablet_manager.cc:1403] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.236s	user 0.015s	sys 0.007s
I20260709 13:16:27.295738 32126 tablet_bootstrap.cc:654] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.293233 32127 raft_consensus.cc:385] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.299479 32125 ts_tablet_manager.cc:1434] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:16:27.299623 32127 raft_consensus.cc:740] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.300536 32123 raft_consensus.cc:359] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.300874 32125 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:27.301321 32123 raft_consensus.cc:385] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.308063 32123 raft_consensus.cc:740] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.304453 32127 consensus_queue.cc:260] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.311020 32127 ts_tablet_manager.cc:1434] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:16:27.310293 31749 catalog_manager.cc:5697] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.316709 32125 tablet_bootstrap.cc:654] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.316845 32127 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:27.319131 32404 consensus_queue.cc:1048] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.321746 32123 consensus_queue.cc:260] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.326759 32123 ts_tablet_manager.cc:1434] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.004s
I20260709 13:16:27.328050 32123 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:27.338248 32123 tablet_bootstrap.cc:654] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.342044 32127 tablet_bootstrap.cc:654] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.347806 32125 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:27.348356 32126 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:27.348460 32125 ts_tablet_manager.cc:1403] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.003s
I20260709 13:16:27.349171 32126 ts_tablet_manager.cc:1403] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.065s	user 0.019s	sys 0.013s
I20260709 13:16:27.349411 32458 raft_consensus.cc:493] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.350142 32458 raft_consensus.cc:515] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.352584 32125 raft_consensus.cc:359] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.353653 32125 raft_consensus.cc:385] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.353973 32125 raft_consensus.cc:740] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.355502 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:27.356585 32125 consensus_queue.cc:260] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.355306 32126 raft_consensus.cc:359] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.358476 32126 raft_consensus.cc:385] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.358841 32126 raft_consensus.cc:740] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.360754 32126 consensus_queue.cc:260] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:27.363554 31968 leader_election.cc:343] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.361652 32125 ts_tablet_manager.cc:1434] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:16:27.365712 32125 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:27.307911 31753 catalog_manager.cc:5697] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.371236 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:27.370427 32458 leader_election.cc:290] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:27.373138 32077 raft_consensus.cc:2468] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:27.374836 31969 leader_election.cc:304] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:27.376235 32127 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:27.376801 32127 ts_tablet_manager.cc:1403] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.007s
I20260709 13:16:27.378005 32126 ts_tablet_manager.cc:1434] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 13:16:27.379125 32123 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:27.379400 32126 tablet_bootstrap.cc:492] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:27.379716 32123 ts_tablet_manager.cc:1403] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 13:16:27.380508 32127 raft_consensus.cc:359] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.387079 32127 raft_consensus.cc:385] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.387763 32125 tablet_bootstrap.cc:654] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.386732 32123 raft_consensus.cc:359] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.388183 32123 raft_consensus.cc:385] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.388489 32123 raft_consensus.cc:740] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.389932 32123 consensus_queue.cc:260] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.394218 31752 catalog_manager.cc:5697] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.395539 32123 ts_tablet_manager.cc:1434] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:16:27.397233 32126 tablet_bootstrap.cc:654] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.381961 32458 raft_consensus.cc:2804] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.398067 32458 raft_consensus.cc:493] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.398411 32458 raft_consensus.cc:3060] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.405479 32396 raft_consensus.cc:493] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.387425 32127 raft_consensus.cc:740] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.408941 32457 consensus_queue.cc:1048] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.409819 32396 raft_consensus.cc:515] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.411962 32127 consensus_queue.cc:260] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.414299 32127 ts_tablet_manager.cc:1434] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.005s
I20260709 13:16:27.415346 32127 tablet_bootstrap.cc:492] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:27.423691 32123 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:27.424620 32395 consensus_queue.cc:1048] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.428984 32458 raft_consensus.cc:515] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.435835 32282 consensus_queue.cc:1048] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.436167 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:27.436877 32077 raft_consensus.cc:3060] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.439492 31751 catalog_manager.cc:5697] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:27.441067 32297 raft_consensus.cc:493] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.441634 32297 raft_consensus.cc:515] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.446774 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14858ce2d22c4dba9d14295ca6c2b23c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:27.447288 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14858ce2d22c4dba9d14295ca6c2b23c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:27.447589 32074 raft_consensus.cc:2468] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:27.448014 31851 raft_consensus.cc:2468] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:27.448958 32297 leader_election.cc:290] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.449379 31969 leader_election.cc:304] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:27.451761 32297 raft_consensus.cc:2804] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.452368 32297 raft_consensus.cc:493] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.452895 32297 raft_consensus.cc:3060] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.470701 32458 leader_election.cc:290] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:27.474012 32127 tablet_bootstrap.cc:654] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.492935 32123 tablet_bootstrap.cc:654] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.498077 32125 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:27.498760 32125 ts_tablet_manager.cc:1403] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.133s	user 0.034s	sys 0.012s
I20260709 13:16:27.499644 32181 consensus_queue.cc:1048] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.502601 32125 raft_consensus.cc:359] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.503515 32125 raft_consensus.cc:385] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.503836 32125 raft_consensus.cc:740] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.504585 32125 consensus_queue.cc:260] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.506819 32125 ts_tablet_manager.cc:1434] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:16:27.507613 32370 consensus_queue.cc:1048] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.508057 32125 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:27.513082 32408 raft_consensus.cc:493] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.513691 32408 raft_consensus.cc:515] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.514636 32077 raft_consensus.cc:2468] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:27.520551 31969 leader_election.cc:304] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:27.522114 32400 raft_consensus.cc:2804] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.522655 32400 raft_consensus.cc:697] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:27.523512 32400 consensus_queue.cc:237] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.528014 32303 raft_consensus.cc:493] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.528637 32303 raft_consensus.cc:515] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.529248 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:27.516217 32125 tablet_bootstrap.cc:654] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.516628 32408 leader_election.cc:290] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.532871 32303 leader_election.cc:290] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.535300 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c85fd3d52a714f44a3e322f4488f1a5c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:27.535765 31852 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.533408 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4e043220524dfa8c9106bfe7690bcd" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:27.537098 31969 leader_election.cc:343] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.544368 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c85fd3d52a714f44a3e322f4488f1a5c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:27.546845 31967 leader_election.cc:343] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.547488 31967 leader_election.cc:304] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:27.500097 32126 tablet_bootstrap.cc:492] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:27.549386 32123 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:27.579952 32126 ts_tablet_manager.cc:1403] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.201s	user 0.019s	sys 0.027s
W20260709 13:16:27.561681 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 123573us
I20260709 13:16:27.580152 32123 ts_tablet_manager.cc:1403] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.157s	user 0.017s	sys 0.000s
I20260709 13:16:27.557328 32458 raft_consensus.cc:2749] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:27.570559 31893 leader_election.cc:343] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:27.564340 31968 leader_election.cc:343] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.562902 32396 leader_election.cc:290] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:27.609174 32396 consensus_queue.cc:1048] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.611119 32126 raft_consensus.cc:359] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.611970 32126 raft_consensus.cc:385] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.612252 32126 raft_consensus.cc:740] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.612341 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4e043220524dfa8c9106bfe7690bcd" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:27.555171 32181 consensus_queue.cc:1048] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 13:16:27.612947 32126 consensus_queue.cc:260] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.613891 31753 catalog_manager.cc:5697] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.554584 32297 raft_consensus.cc:515] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.646286 32126 ts_tablet_manager.cc:1434] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:16:27.649757 32379 raft_consensus.cc:493] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.650615 32379 raft_consensus.cc:515] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.653054 32379 leader_election.cc:290] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:27.654760 32127 tablet_bootstrap.cc:492] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:27.655774 32127 ts_tablet_manager.cc:1403] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.241s	user 0.010s	sys 0.012s
W20260709 13:16:27.662098 31820 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 64826.8us
I20260709 13:16:27.641894 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:27.665957 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90a6b486bef42019735b76ff4da04a1" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:27.667994 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:27.681833 31851 raft_consensus.cc:2468] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:27.683424 32078 raft_consensus.cc:2393] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:27.687341 32181 consensus_queue.cc:1048] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.690191 32125 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:27.691080 32125 ts_tablet_manager.cc:1403] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.183s	user 0.023s	sys 0.010s
I20260709 13:16:27.691577 32126 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:27.696019 32125 raft_consensus.cc:359] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.698576 32127 raft_consensus.cc:359] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.699420 32127 raft_consensus.cc:385] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.699764 32127 raft_consensus.cc:740] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.700556 32127 consensus_queue.cc:260] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.703850 32123 raft_consensus.cc:359] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.708613 32123 raft_consensus.cc:385] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.709033 32123 raft_consensus.cc:740] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.676070 32379 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.710789 32379 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.714323 32127 ts_tablet_manager.cc:1434] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.001s
I20260709 13:16:27.735855 32297 leader_election.cc:290] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.737849 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14858ce2d22c4dba9d14295ca6c2b23c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:27.738118 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14858ce2d22c4dba9d14295ca6c2b23c" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:27.738648 32074 raft_consensus.cc:3060] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.738705 31851 raft_consensus.cc:3060] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.742859 32379 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:27.744720 32126 tablet_bootstrap.cc:654] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.744956 32123 consensus_queue.cc:260] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.697314 32125 raft_consensus.cc:385] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.767930 32125 raft_consensus.cc:740] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
W20260709 13:16:27.768932 31896 leader_election.cc:343] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.773913 32127 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:27.771987 32076 raft_consensus.cc:2468] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:27.808091 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:27.809118 32003 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:27.818725 31821 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:27.820108 32379 raft_consensus.cc:2804] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:27.820528 32379 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.820866 32379 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:27.818730 31895 leader_election.cc:304] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:27.828409 32404 raft_consensus.cc:2804] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.828840 32404 raft_consensus.cc:493] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.829233 32404 raft_consensus.cc:3060] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:27.842845 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:16:27.844657 31819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 206588us
I20260709 13:16:27.862841 32487 consensus_queue.cc:1048] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.863802 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90a6b486bef42019735b76ff4da04a1" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:27.869663 32123 ts_tablet_manager.cc:1434] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.289s	user 0.016s	sys 0.006s
I20260709 13:16:27.853097 32396 consensus_queue.cc:1048] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:16:27.877333 32123 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:27.871467 31851 raft_consensus.cc:2468] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:27.889714 31967 leader_election.cc:304] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:27.891573 32303 raft_consensus.cc:2804] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.893453 32303 raft_consensus.cc:697] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:27.894434 32303 consensus_queue.cc:237] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:27.894928 32125 consensus_queue.cc:260] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.903868 32126 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:27.904551 32126 ts_tablet_manager.cc:1403] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.213s	user 0.011s	sys 0.006s
I20260709 13:16:27.907102 32125 ts_tablet_manager.cc:1434] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.216s	user 0.012s	sys 0.002s
I20260709 13:16:27.889673 32127 tablet_bootstrap.cc:654] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.919962 32126 raft_consensus.cc:359] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.921172 32126 raft_consensus.cc:385] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.921569 32126 raft_consensus.cc:740] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.934391 32125 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:27.937971 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:27.947558 32074 raft_consensus.cc:2468] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:27.953521 32484 raft_consensus.cc:493] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.954162 32484 raft_consensus.cc:515] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:27.962929 32126 consensus_queue.cc:260] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:27.967011 31853 raft_consensus.cc:3060] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.968918 32126 ts_tablet_manager.cc:1434] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.000s
I20260709 13:16:27.970252 32126 tablet_bootstrap.cc:492] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:27.981397 31893 leader_election.cc:304] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: ccf4710690c7473d814836da1babee51
I20260709 13:16:27.983847 32355 raft_consensus.cc:2804] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.984385 32355 raft_consensus.cc:493] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.984755 32355 raft_consensus.cc:3060] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.942689 32404 raft_consensus.cc:515] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:27.987743 32404 leader_election.cc:290] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:27.995153 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4e043220524dfa8c9106bfe7690bcd" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:27.995965 32078 raft_consensus.cc:3060] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.979803 32379 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.005905 31852 raft_consensus.cc:1240] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Rejecting Update request from peer a588481f03f6474d9009d5bd5e08e887 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:16:28.006410 32379 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:28.020785 32125 tablet_bootstrap.cc:654] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.043776 32078 raft_consensus.cc:2468] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:28.047470 31895 leader_election.cc:304] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:28.048830 31850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b4e043220524dfa8c9106bfe7690bcd" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:28.049669 31850 raft_consensus.cc:3060] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.039183 32000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:28.056772 32484 leader_election.cc:290] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:28.063627 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:28.068313 31850 raft_consensus.cc:2468] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:28.069466 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:28.072180 32297 consensus_queue.cc:1048] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.074069 32395 raft_consensus.cc:2804] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.074666 32395 raft_consensus.cc:697] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:28.075590 32395 consensus_queue.cc:237] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.080405 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:28.106866 31751 catalog_manager.cc:5697] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.126020 32458 raft_consensus.cc:493] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.126749 32458 raft_consensus.cc:515] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.126933 32355 raft_consensus.cc:515] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.130429 32458 leader_election.cc:290] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:28.132387 32123 tablet_bootstrap.cc:654] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.133723 32126 tablet_bootstrap.cc:654] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.139911 32355 leader_election.cc:290] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:28.154752 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3474410429d7478d8ba2221c7400cb01" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:28.156363 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:28.092036 31819 leader_election.cc:304] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
W20260709 13:16:28.158097 31969 leader_election.cc:343] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:28.158524 31819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 66991.2us
I20260709 13:16:28.159723 32379 raft_consensus.cc:2749] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:28.162241 31819 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:28.164520 32379 raft_consensus.cc:2749] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:28.173760 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 128773us
I20260709 13:16:28.177136 32125 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:28.177958 32125 ts_tablet_manager.cc:1403] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.244s	user 0.019s	sys 0.006s
I20260709 13:16:28.196707 31924 raft_consensus.cc:3060] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.202423 32396 consensus_queue.cc:1048] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.209124 32484 consensus_queue.cc:1059] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.210569 31896 leader_election.cc:304] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:28.211021 31853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:28.209520 32125 raft_consensus.cc:359] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:28.224931 32125 raft_consensus.cc:385] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.225296 32125 raft_consensus.cc:740] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
W20260709 13:16:28.186553 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 127780us
I20260709 13:16:28.217432 31853 raft_consensus.cc:3060] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.234442 32125 consensus_queue.cc:260] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:28.245805 32127 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:28.246536 32127 ts_tablet_manager.cc:1403] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.479s	user 0.009s	sys 0.026s
I20260709 13:16:28.269917 32126 tablet_bootstrap.cc:492] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:28.270645 32126 ts_tablet_manager.cc:1403] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.301s	user 0.017s	sys 0.013s
I20260709 13:16:28.271385 32408 raft_consensus.cc:3055] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:28.272195 32408 raft_consensus.cc:740] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:28.274941 32126 raft_consensus.cc:359] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.275780 32126 raft_consensus.cc:385] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.276089 32126 raft_consensus.cc:740] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.276733 32126 consensus_queue.cc:260] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.273414 32408 consensus_queue.cc:260] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:28.306552 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 51222.6us
I20260709 13:16:28.275548 32127 raft_consensus.cc:359] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.322568 32127 raft_consensus.cc:385] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.327366 32127 raft_consensus.cc:740] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.328715 32125 ts_tablet_manager.cc:1434] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.150s	user 0.008s	sys 0.002s
I20260709 13:16:28.329432 32123 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:28.330344 32123 ts_tablet_manager.cc:1403] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.454s	user 0.025s	sys 0.020s
W20260709 13:16:28.314231 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 60379.4us
I20260709 13:16:28.336537 32126 ts_tablet_manager.cc:1434] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:16:28.338786 32127 consensus_queue.cc:260] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.345433 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3474410429d7478d8ba2221c7400cb01" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:28.346331 31929 raft_consensus.cc:2468] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:28.346812 32125 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:28.353133 31853 raft_consensus.cc:2468] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:28.355913 31893 leader_election.cc:304] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:28.357244 32514 raft_consensus.cc:2804] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.357812 32514 raft_consensus.cc:697] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:28.358736 32514 consensus_queue.cc:237] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.362566 32126 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:28.361284 32123 raft_consensus.cc:359] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:28.364319 32123 raft_consensus.cc:385] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.364585 32123 raft_consensus.cc:740] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.365233 32123 consensus_queue.cc:260] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:28.373014 32355 raft_consensus.cc:2749] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:28.374511 32506 raft_consensus.cc:493] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.375088 32506 raft_consensus.cc:515] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:28.380295 31896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 169817us
I20260709 13:16:28.380789 32506 leader_election.cc:290] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:28.381901 31848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c85fd3d52a714f44a3e322f4488f1a5c" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:28.382722 31848 raft_consensus.cc:2468] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
W20260709 13:16:28.381234 31896 leader_election.cc:343] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.383955 32041 leader_election.cc:304] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:28.388932 32506 raft_consensus.cc:2804] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.389390 32506 raft_consensus.cc:493] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.399175 32127 ts_tablet_manager.cc:1434] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.152s	user 0.008s	sys 0.004s
I20260709 13:16:28.401991 32126 tablet_bootstrap.cc:654] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.415414 32408 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.395114 32506 raft_consensus.cc:3060] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.428399 32123 ts_tablet_manager.cc:1434] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.098s	user 0.008s	sys 0.000s
I20260709 13:16:28.429375 32127 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:28.439564 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c85fd3d52a714f44a3e322f4488f1a5c" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:28.440479 32004 raft_consensus.cc:2468] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:28.441188 32492 consensus_queue.cc:1048] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.444185 32125 tablet_bootstrap.cc:654] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.445103 32123 tablet_bootstrap.cc:492] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:28.448210 32513 raft_consensus.cc:493] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.449849 32513 raft_consensus.cc:515] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.452910 32513 leader_election.cc:290] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:28.456492 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e460a405ae485fb15869970e178caf" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:28.457392 32004 raft_consensus.cc:2468] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:28.457245 31848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e460a405ae485fb15869970e178caf" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:28.458927 31896 leader_election.cc:304] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
W20260709 13:16:28.459955 31893 leader_election.cc:343] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.460028 32513 raft_consensus.cc:2804] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.461293 32513 raft_consensus.cc:493] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.461845 32513 raft_consensus.cc:3060] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.467980 32127 tablet_bootstrap.cc:654] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.473167 32123 tablet_bootstrap.cc:654] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.444099 31968 leader_election.cc:304] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:28.494897 32492 raft_consensus.cc:2804] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:16:28.495268 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 51900us
I20260709 13:16:28.495479 32492 raft_consensus.cc:493] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.496058 32492 raft_consensus.cc:3060] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:28.502072 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 58093.2us
I20260709 13:16:28.495841 32514 consensus_queue.cc:1048] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.510013 32505 consensus_queue.cc:1048] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.527259 31750 catalog_manager.cc:5697] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:28.531476 32127 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:28.532325 32127 ts_tablet_manager.cc:1403] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.109s	user 0.005s	sys 0.009s
I20260709 13:16:28.549685 32506 raft_consensus.cc:515] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:28.550948 32458 consensus_queue.cc:1048] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.583407 32126 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:28.585440 32126 ts_tablet_manager.cc:1403] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.223s	user 0.015s	sys 0.011s
I20260709 13:16:28.578897 32127 raft_consensus.cc:359] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.592057 32127 raft_consensus.cc:385] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.592497 32127 raft_consensus.cc:740] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.596238 32127 consensus_queue.cc:260] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.584206 32513 raft_consensus.cc:515] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.600419 32127 ts_tablet_manager.cc:1434] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.003s
I20260709 13:16:28.600714 32506 leader_election.cc:290] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:28.601634 32127 tablet_bootstrap.cc:492] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:28.602212 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c85fd3d52a714f44a3e322f4488f1a5c" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:28.602432 31853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e460a405ae485fb15869970e178caf" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:28.602895 31852 raft_consensus.cc:3060] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.603025 32492 raft_consensus.cc:515] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:28.604568 31893 leader_election.cc:343] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.605283 32492 leader_election.cc:290] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:28.611750 32126 raft_consensus.cc:359] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.612583 32126 raft_consensus.cc:385] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.612869 32126 raft_consensus.cc:740] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.621925 32126 consensus_queue.cc:260] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.622750 32513 leader_election.cc:290] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:28.632995 31969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 66183.1us
I20260709 13:16:28.650657 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e460a405ae485fb15869970e178caf" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:28.651400 32004 raft_consensus.cc:3060] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.653573 32127 tablet_bootstrap.cc:654] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.647403 32408 consensus_queue.cc:1048] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:16:28.658051 32126 ts_tablet_manager.cc:1434] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 13:16:28.658772 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3474410429d7478d8ba2221c7400cb01" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:28.659152 32126 tablet_bootstrap.cc:492] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:28.661055 32125 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:28.661657 32125 ts_tablet_manager.cc:1403] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.315s	user 0.038s	sys 0.024s
I20260709 13:16:28.669721 32123 tablet_bootstrap.cc:492] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:28.670302 32123 ts_tablet_manager.cc:1403] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.225s	user 0.019s	sys 0.016s
I20260709 13:16:28.671974 32126 tablet_bootstrap.cc:654] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.672331 31852 raft_consensus.cc:2468] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:28.642587 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c85fd3d52a714f44a3e322f4488f1a5c" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:28.672611 32123 raft_consensus.cc:359] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.684916 32123 raft_consensus.cc:385] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.685345 32123 raft_consensus.cc:740] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.686259 32123 consensus_queue.cc:260] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.692975 32123 ts_tablet_manager.cc:1434] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:16:28.647622 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3474410429d7478d8ba2221c7400cb01" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:28.700906 32125 raft_consensus.cc:359] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.701956 32125 raft_consensus.cc:385] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.699389 32297 raft_consensus.cc:493] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.703163 32297 raft_consensus.cc:515] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:28.708346 31969 leader_election.cc:343] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.690527 32003 raft_consensus.cc:3060] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.721786 32123 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:28.702291 32125 raft_consensus.cc:740] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.722419 31926 raft_consensus.cc:3060] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.733774 31848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:28.724898 32125 consensus_queue.cc:260] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.734862 31848 raft_consensus.cc:2393] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:28.737529 32125 ts_tablet_manager.cc:1434] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.001s
I20260709 13:16:28.723309 32041 leader_election.cc:304] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:28.740226 32297 leader_election.cc:290] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:28.740474 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:28.743526 32415 raft_consensus.cc:2804] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.746483 32415 raft_consensus.cc:697] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:28.747861 32125 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:28.744968 31968 leader_election.cc:304] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:28.724582 31751 catalog_manager.cc:5697] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.760332 32003 raft_consensus.cc:2468] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:28.754747 32297 raft_consensus.cc:3060] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.747977 32415 consensus_queue.cc:237] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:28.766804 32504 raft_consensus.cc:493] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.771823 32528 log.cc:927] Time spent T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:16:28.788767 32004 raft_consensus.cc:2468] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:28.773591 32504 raft_consensus.cc:515] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.796772 32504 leader_election.cc:290] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:28.797290 31926 raft_consensus.cc:2468] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:28.799662 31968 leader_election.cc:304] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:28.800058 31896 leader_election.cc:304] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:28.800570 32303 raft_consensus.cc:2804] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.801138 32303 raft_consensus.cc:697] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:28.801330 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:28.802604 32515 raft_consensus.cc:2804] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.803192 32515 raft_consensus.cc:697] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:28.804188 32515 consensus_queue.cc:237] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.805603 32126 tablet_bootstrap.cc:492] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
W20260709 13:16:28.805974 31895 leader_election.cc:343] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.806200 32126 ts_tablet_manager.cc:1403] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.147s	user 0.016s	sys 0.017s
I20260709 13:16:28.802168 32303 consensus_queue.cc:237] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.809191 32126 raft_consensus.cc:359] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.818092 32126 raft_consensus.cc:385] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.818435 32126 raft_consensus.cc:740] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.819196 32126 consensus_queue.cc:260] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.822037 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:28.822966 31855 raft_consensus.cc:2468] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:28.823287 32127 tablet_bootstrap.cc:492] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:28.833909 32127 ts_tablet_manager.cc:1403] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.233s	user 0.022s	sys 0.035s
I20260709 13:16:28.824304 32125 tablet_bootstrap.cc:654] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.825206 32123 tablet_bootstrap.cc:654] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.828027 32513 raft_consensus.cc:493] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.839077 32513 raft_consensus.cc:515] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.841871 31893 leader_election.cc:304] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:28.843050 32504 raft_consensus.cc:2804] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.843544 32504 raft_consensus.cc:493] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.844981 32504 raft_consensus.cc:3060] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.846350 32297 raft_consensus.cc:2749] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:28.849404 32127 raft_consensus.cc:359] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.822435 32126 ts_tablet_manager.cc:1434] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:16:28.852677 32126 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:28.850167 32127 raft_consensus.cc:385] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.857916 32127 raft_consensus.cc:740] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.872318 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:28.873102 32127 consensus_queue.cc:260] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:28.874336 32004 raft_consensus.cc:2468] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:28.872272 32504 raft_consensus.cc:515] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.899219 31896 leader_election.cc:304] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:28.900373 32484 raft_consensus.cc:2804] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.900811 32484 raft_consensus.cc:493] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.901157 32484 raft_consensus.cc:3060] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.911761 32127 ts_tablet_manager.cc:1434] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 13:16:28.914548 32504 leader_election.cc:290] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:28.876140 32408 consensus_queue.cc:1048] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 13:16:28.929615 32355 consensus_queue.cc:1048] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.931005 32076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:28.933346 31852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:28.934053 31852 raft_consensus.cc:3060] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.936969 32123 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:28.937664 32123 ts_tablet_manager.cc:1403] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.002s
I20260709 13:16:28.941529 32125 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:28.942153 32125 ts_tablet_manager.cc:1403] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.195s	user 0.018s	sys 0.013s
I20260709 13:16:28.919480 32513 leader_election.cc:290] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:28.929073 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
W20260709 13:16:28.948696 31895 leader_election.cc:343] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.954185 32125 raft_consensus.cc:359] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.955078 32125 raft_consensus.cc:385] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.955384 32125 raft_consensus.cc:740] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.956151 32125 consensus_queue.cc:260] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:28.957540 32127 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
W20260709 13:16:28.963279 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:16:28.965811 31895 leader_election.cc:343] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.959457 32123 raft_consensus.cc:359] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.966998 32123 raft_consensus.cc:385] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.967280 32123 raft_consensus.cc:740] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.967976 32123 consensus_queue.cc:260] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:28.969290 31752 catalog_manager.cc:5697] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.979643 32125 ts_tablet_manager.cc:1434] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.001s
I20260709 13:16:28.986445 32123 ts_tablet_manager.cc:1434] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:16:28.987732 32123 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:29.058159 32125 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:28.984988 32415 raft_consensus.cc:493] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.054863 32491 raft_consensus.cc:493] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.082614 32415 raft_consensus.cc:515] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:29.113379 32123 tablet_bootstrap.cc:654] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.115908 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3474410429d7478d8ba2221c7400cb01" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:29.117211 31929 raft_consensus.cc:2393] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate ccf4710690c7473d814836da1babee51 in this term.
W20260709 13:16:29.122390 32536 log.cc:927] Time spent T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:16:29.094165 32491 raft_consensus.cc:515] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:29.165992 32533 log.cc:927] Time spent T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:16:29.195910 32125 tablet_bootstrap.cc:654] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.188851 32491 leader_election.cc:290] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:29.193459 32519 raft_consensus.cc:493] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.201897 32519 raft_consensus.cc:515] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.204589 32519 leader_election.cc:290] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:29.274372 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:29.275576 31929 raft_consensus.cc:2393] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:29.279078 32126 tablet_bootstrap.cc:654] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.309763 32123 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:29.310638 32123 ts_tablet_manager.cc:1403] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.323s	user 0.015s	sys 0.006s
I20260709 13:16:29.337720 32127 tablet_bootstrap.cc:654] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.286185 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e460a405ae485fb15869970e178caf" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:29.348958 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3474410429d7478d8ba2221c7400cb01" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:29.353107 31852 raft_consensus.cc:2468] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:29.363050 32123 raft_consensus.cc:359] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.364133 32123 raft_consensus.cc:385] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.364490 32123 raft_consensus.cc:740] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.365561 32123 consensus_queue.cc:260] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.358476 32484 raft_consensus.cc:515] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:29.387938 32484 leader_election.cc:290] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:29.396575 32123 ts_tablet_manager.cc:1434] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.003s
I20260709 13:16:29.410204 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:29.410964 32004 raft_consensus.cc:3060] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.409168 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
W20260709 13:16:29.427842 31895 leader_election.cc:343] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.431937 32125 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:29.432715 32125 ts_tablet_manager.cc:1403] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.430s	user 0.038s	sys 0.029s
I20260709 13:16:29.452481 32126 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:29.455289 32126 ts_tablet_manager.cc:1403] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.603s	user 0.024s	sys 0.012s
I20260709 13:16:29.464931 32415 leader_election.cc:290] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:29.459802 32126 raft_consensus.cc:359] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.469058 32126 raft_consensus.cc:385] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.469630 32126 raft_consensus.cc:740] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.472919 32126 consensus_queue.cc:260] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.479468 32004 raft_consensus.cc:2468] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:29.554500 32126 ts_tablet_manager.cc:1434] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.000s
I20260709 13:16:29.532565 32545 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:29.549997 32552 raft_consensus.cc:493] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.562458 32545 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.577908 32545 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:29.582479 32545 consensus_queue.cc:1048] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.568169 32552 raft_consensus.cc:515] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.591735 31925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:29.592563 31925 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:29.593706 31968 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:29.596308 32545 raft_consensus.cc:2804] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:29.596740 32545 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:29.597131 32545 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:29.597571 32073 raft_consensus.cc:3060] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:29.610755 31820 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 352968us
I20260709 13:16:29.612810 32125 raft_consensus.cc:359] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.613600 32125 raft_consensus.cc:385] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.613902 32125 raft_consensus.cc:740] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.614562 32125 consensus_queue.cc:260] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.619782 32545 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.623108 32125 ts_tablet_manager.cc:1434] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.190s	user 0.004s	sys 0.004s
I20260709 13:16:29.629608 31896 leader_election.cc:304] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:29.632094 32125 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:29.632835 32513 raft_consensus.cc:2804] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.633379 32513 raft_consensus.cc:697] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:29.634290 32513 consensus_queue.cc:237] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:29.636731 32514 consensus_queue.cc:1048] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.604681 32552 leader_election.cc:290] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:29.652474 31855 raft_consensus.cc:3060] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.667289 32127 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:29.669056 32127 ts_tablet_manager.cc:1403] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.742s	user 0.030s	sys 0.041s
I20260709 13:16:29.676514 32123 tablet_bootstrap.cc:492] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:29.679241 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:29.679616 32125 tablet_bootstrap.cc:654] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.680285 31849 raft_consensus.cc:2393] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 2: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
W20260709 13:16:29.702335 31821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.008s	sys 0.009s
W20260709 13:16:29.702836 31821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.009s
I20260709 13:16:29.692318 32545 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:29.690783 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:29.727933 31851 raft_consensus.cc:2393] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccf4710690c7473d814836da1babee51 in current term 2: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:29.730116 32548 raft_consensus.cc:493] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.730687 32548 raft_consensus.cc:515] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.734368 32123 tablet_bootstrap.cc:654] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:29.703199 31821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 544196us
I20260709 13:16:29.739992 32127 raft_consensus.cc:359] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.742789 32127 raft_consensus.cc:385] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.743260 32127 raft_consensus.cc:740] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
W20260709 13:16:29.746095 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 57123.3us
I20260709 13:16:29.748472 32078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:29.754137 32125 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:29.754756 32125 ts_tablet_manager.cc:1403] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.128s	user 0.018s	sys 0.029s
I20260709 13:16:29.755636 32000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e460a405ae485fb15869970e178caf" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:29.758158 32125 raft_consensus.cc:359] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.758945 32125 raft_consensus.cc:385] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.759197 32125 raft_consensus.cc:740] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.759855 32125 consensus_queue.cc:260] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:29.752487 31820 leader_election.cc:343] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.744432 32127 consensus_queue.cc:260] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.762656 32125 ts_tablet_manager.cc:1434] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:16:29.766880 32127 ts_tablet_manager.cc:1434] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 13:16:29.768474 32127 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:29.769937 32548 leader_election.cc:290] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:29.772197 31893 leader_election.cc:304] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:29.780552 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7340289c3434c3782e3e367d4de5258" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:29.781517 32002 raft_consensus.cc:2468] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:29.789876 32126 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:29.785993 32515 raft_consensus.cc:2804] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.798506 32500 raft_consensus.cc:493] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.799187 32500 raft_consensus.cc:515] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.799279 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:29.800410 31849 raft_consensus.cc:2393] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:29.801913 32126 tablet_bootstrap.cc:654] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.805156 32515 raft_consensus.cc:697] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:29.807866 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7340289c3434c3782e3e367d4de5258" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:29.807542 32515 consensus_queue.cc:237] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.810474 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:29.781572 32044 leader_election.cc:304] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:29.810062 31849 raft_consensus.cc:2468] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:29.825788 32123 tablet_bootstrap.cc:492] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:29.826514 32123 ts_tablet_manager.cc:1403] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.150s	user 0.010s	sys 0.015s
I20260709 13:16:29.816764 32500 leader_election.cc:290] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:29.824875 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:29.842311 32561 log.cc:927] Time spent T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:16:29.821449 32540 raft_consensus.cc:2749] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:29.844223 31929 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:29.836275 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:29.851006 32526 consensus_queue.cc:1048] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:29.827260 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 45856.8us
W20260709 13:16:29.858343 31893 leader_election.cc:343] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.860890 31968 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:29.863257 32044 leader_election.cc:304] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:29.864133 32540 consensus_queue.cc:1048] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.865871 32415 raft_consensus.cc:2804] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:29.866360 32415 raft_consensus.cc:493] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.866731 32415 raft_consensus.cc:3060] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.869727 32127 tablet_bootstrap.cc:654] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.869764 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:29.872844 31821 leader_election.cc:304] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:29.874586 32567 raft_consensus.cc:2749] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:29.876252 31896 leader_election.cc:343] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.876754 31896 leader_election.cc:304] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:29.880761 32041 leader_election.cc:304] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:29.882089 32514 raft_consensus.cc:2749] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:29.882846 32519 raft_consensus.cc:3060] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.885362 32126 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:29.885967 32126 ts_tablet_manager.cc:1403] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.096s	user 0.023s	sys 0.011s
I20260709 13:16:29.879329 32540 consensus_queue.cc:1048] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:29.859707 32513 consensus_queue.cc:1048] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:16:29.889003 32126 raft_consensus.cc:359] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.889708 32126 raft_consensus.cc:385] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.890003 32126 raft_consensus.cc:740] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.890892 32126 consensus_queue.cc:260] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:29.892875 32126 ts_tablet_manager.cc:1434] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:29.893992 32415 raft_consensus.cc:515] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.896756 32415 leader_election.cc:290] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:29.893345 32123 raft_consensus.cc:359] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:29.898010 32123 raft_consensus.cc:385] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.898285 32123 raft_consensus.cc:740] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.899225 32519 raft_consensus.cc:2749] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:29.898985 32123 consensus_queue.cc:260] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
W20260709 13:16:29.900751 31819 leader_election.cc:343] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.901280 31819 leader_election.cc:304] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:29.903394 32541 raft_consensus.cc:2749] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:29.905817 32123 ts_tablet_manager.cc:1434] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 13:16:29.907178 32123 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:29.911933 31848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7340289c3434c3782e3e367d4de5258" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:29.913101 31848 raft_consensus.cc:3060] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.915102 32123 tablet_bootstrap.cc:654] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.906765 32126 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:29.928046 32123 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:29.928750 32123 ts_tablet_manager.cc:1403] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.008s
I20260709 13:16:29.937362 31848 raft_consensus.cc:2468] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:29.937842 32123 raft_consensus.cc:359] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.938608 32123 raft_consensus.cc:385] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.938730 32041 leader_election.cc:304] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:29.940900 32125 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:29.938925 32123 raft_consensus.cc:740] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.943897 32123 consensus_queue.cc:260] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:29.945252 32519 raft_consensus.cc:2804] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.945657 32127 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:29.945802 32519 raft_consensus.cc:697] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:29.946269 32127 ts_tablet_manager.cc:1403] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.006s
I20260709 13:16:29.964090 32123 ts_tablet_manager.cc:1434] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.004s
I20260709 13:16:29.965884 32123 tablet_bootstrap.cc:492] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:29.971115 31750 catalog_manager.cc:5697] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } }
I20260709 13:16:29.972618 32125 tablet_bootstrap.cc:654] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.975916 32519 consensus_queue.cc:237] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:29.989095 32123 tablet_bootstrap.cc:654] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.991907 32126 tablet_bootstrap.cc:654] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.031116 32125 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:30.031957 32125 ts_tablet_manager.cc:1403] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.003s
I20260709 13:16:30.020771 32506 raft_consensus.cc:493] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.042215 32506 raft_consensus.cc:515] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.045378 32125 raft_consensus.cc:359] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.049535 32125 raft_consensus.cc:385] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.049870 32125 raft_consensus.cc:740] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.051056 32125 consensus_queue.cc:260] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.053958 32125 ts_tablet_manager.cc:1434] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:16:30.110229 32541 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:30.111148 32541 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.114207 32541 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.119127 32555 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:30.119726 32555 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.121407 32555 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.165597 32541 raft_consensus.cc:493] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.166445 32541 raft_consensus.cc:515] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.169484 32541 leader_election.cc:290] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:30.192341 32123 tablet_bootstrap.cc:492] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:30.193401 32123 ts_tablet_manager.cc:1403] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.228s	user 0.040s	sys 0.018s
W20260709 13:16:30.196938 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:30.196835 32123 raft_consensus.cc:359] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.197811 32123 raft_consensus.cc:385] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.198119 32123 raft_consensus.cc:740] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.207163 32541 raft_consensus.cc:493] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:30.207929 32541 raft_consensus.cc:515] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.210685 32541 leader_election.cc:290] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.220764 32488 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.221359 32488 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.223795 32488 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.252225 32484 raft_consensus.cc:493] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.255543 32506 leader_election.cc:290] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:30.258980 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.259788 31922 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:30.261897 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142b461239534bae915d26581eb87d21" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.265018 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.252912 32484 raft_consensus.cc:515] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.269372 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.270346 31922 raft_consensus.cc:2410] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:16:30.271422 32484 leader_election.cc:290] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.272544 31819 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
W20260709 13:16:30.273692 31819 leader_election.cc:343] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.275719 32538 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:30.276129 32538 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:30.276474 32538 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.280257 31848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:30.276585 32126 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:30.286471 32126 ts_tablet_manager.cc:1403] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.392s	user 0.019s	sys 0.005s
I20260709 13:16:30.289044 32126 raft_consensus.cc:359] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.289666 32126 raft_consensus.cc:385] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.289911 32126 raft_consensus.cc:740] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.290612 32126 consensus_queue.cc:260] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.292390 32126 ts_tablet_manager.cc:1434] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:16:30.293304 32126 tablet_bootstrap.cc:492] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:30.295781 32538 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.298367 32538 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.299161 31848 raft_consensus.cc:2393] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 2: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
I20260709 13:16:30.299535 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:30.300312 31922 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.277735 32125 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:30.305492 32126 tablet_bootstrap.cc:654] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.310385 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.311187 31929 raft_consensus.cc:2468] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:30.312659 31922 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:30.314524 31819 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:30.317476 32538 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:30.318612 32538 raft_consensus.cc:697] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:30.320911 32538 consensus_queue.cc:237] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.328752 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:30.331610 32042 leader_election.cc:304] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
W20260709 13:16:30.336162 31893 leader_election.cc:343] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.343426 32125 tablet_bootstrap.cc:654] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.345518 32415 raft_consensus.cc:2804] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.346109 32415 raft_consensus.cc:493] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.346513 32415 raft_consensus.cc:3060] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:30.347023 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 470662us
I20260709 13:16:30.352349 32516 consensus_queue.cc:1048] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:30.352844 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 492033us
I20260709 13:16:30.355139 32297 raft_consensus.cc:2804] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:30.355700 32297 raft_consensus.cc:697] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:30.356499 32297 consensus_queue.cc:237] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.367318 32415 raft_consensus.cc:515] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.372274 32506 raft_consensus.cc:493] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.372836 32506 raft_consensus.cc:515] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.377430 32127 raft_consensus.cc:359] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.378227 32127 raft_consensus.cc:385] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.378487 32127 raft_consensus.cc:740] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.381565 31929 raft_consensus.cc:1240] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Rejecting Update request from peer ccf4710690c7473d814836da1babee51 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:30.381956 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.375011 32506 leader_election.cc:290] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.384619 32000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ebfc33b37b45f69df96a3f4bb8f422" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.384114 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:30.387539 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.395507 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142b461239534bae915d26581eb87d21" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:30.402312 31855 raft_consensus.cc:2468] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 0.
I20260709 13:16:30.404572 31821 leader_election.cc:304] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:30.398756 31998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:30.381618 31850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:30.408592 31850 raft_consensus.cc:3060] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.410146 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.415434 32123 consensus_queue.cc:260] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.416436 32538 raft_consensus.cc:2749] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:30.421335 31849 raft_consensus.cc:1240] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Rejecting Update request from peer ccf4710690c7473d814836da1babee51 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:30.421749 31995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.422998 32593 raft_consensus.cc:493] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 32690b3987094d51bb4f214321a29067)
I20260709 13:16:30.423960 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.423928 32593 raft_consensus.cc:515] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.379233 32127 consensus_queue.cc:260] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.426539 32593 leader_election.cc:290] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:30.426766 31995 raft_consensus.cc:2468] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:30.429085 32127 ts_tablet_manager.cc:1434] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.482s	user 0.011s	sys 0.000s
I20260709 13:16:30.430191 32127 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:30.431195 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7340289c3434c3782e3e367d4de5258" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:30.457888 32004 raft_consensus.cc:3060] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.458334 32123 ts_tablet_manager.cc:1434] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.264s	user 0.010s	sys 0.000s
I20260709 13:16:30.459656 32123 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:30.433202 31821 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:30.440987 31896 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:30.465693 32484 raft_consensus.cc:2749] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:30.467577 32484 consensus_queue.cc:1048] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.465255 31896 leader_election.cc:304] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:30.470484 32504 raft_consensus.cc:2804] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.471022 32504 raft_consensus.cc:493] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.471406 32504 raft_consensus.cc:3060] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.471437 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:30.473464 32488 raft_consensus.cc:2749] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:30.476243 32415 leader_election.cc:290] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:30.444061 32547 consensus_queue.cc:1059] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:30.448115 32074 raft_consensus.cc:3060] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.484252 32594 raft_consensus.cc:3055] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:30.484740 32594 raft_consensus.cc:740] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:30.442490 31998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.486971 31998 raft_consensus.cc:2393] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32690b3987094d51bb4f214321a29067 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:30.489167 32127 tablet_bootstrap.cc:654] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.489886 32594 consensus_queue.cc:260] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.490875 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:30.491255 32594 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:30.493126 31929 raft_consensus.cc:3060] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.494922 32506 consensus_queue.cc:1048] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.498657 32516 consensus_queue.cc:1048] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:30.471268 31820 leader_election.cc:343] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.511245 31820 leader_election.cc:304] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:30.512106 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "658f92ce9d684ffbbec92446706f44c9" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.512619 32123 tablet_bootstrap.cc:654] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.513422 32488 raft_consensus.cc:2749] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:30.511359 32126 tablet_bootstrap.cc:492] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:30.515213 32126 ts_tablet_manager.cc:1403] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.222s	user 0.026s	sys 0.012s
I20260709 13:16:30.516295 31850 raft_consensus.cc:2468] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:30.520329 32504 raft_consensus.cc:515] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.522797 32504 leader_election.cc:290] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.523865 31995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:30.524577 31995 raft_consensus.cc:3060] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.527125 32004 raft_consensus.cc:2468] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:30.531270 32044 leader_election.cc:304] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:30.532199 32519 raft_consensus.cc:2749] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:30.535564 32041 leader_election.cc:304] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:30.536450 32415 raft_consensus.cc:2804] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.537160 32415 raft_consensus.cc:697] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:30.538147 32415 consensus_queue.cc:237] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.539629 32555 consensus_queue.cc:1048] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:30.545799 32123 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:30.546411 32123 ts_tablet_manager.cc:1403] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.000s
I20260709 13:16:30.551369 31995 raft_consensus.cc:2468] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:30.552508 32126 raft_consensus.cc:359] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.553352 32126 raft_consensus.cc:385] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.553653 32126 raft_consensus.cc:740] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.563515 32127 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:30.564253 32127 ts_tablet_manager.cc:1403] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.134s	user 0.011s	sys 0.000s
I20260709 13:16:30.567435 32127 raft_consensus.cc:359] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.568307 32127 raft_consensus.cc:385] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.568620 32127 raft_consensus.cc:740] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.570547 32127 consensus_queue.cc:260] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.573133 31929 raft_consensus.cc:2468] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:30.574121 32127 ts_tablet_manager.cc:1434] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:16:30.577924 31853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:30.578579 32125 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:30.579174 32125 ts_tablet_manager.cc:1403] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.302s	user 0.008s	sys 0.021s
I20260709 13:16:30.579733 32126 consensus_queue.cc:260] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.578397 31968 leader_election.cc:304] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
W20260709 13:16:30.586764 31893 leader_election.cc:343] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.581768 31749 catalog_manager.cc:5697] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.595050 31749 catalog_manager.cc:5697] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.596529 32123 raft_consensus.cc:359] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.599452 32123 raft_consensus.cc:385] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.599794 32123 raft_consensus.cc:740] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.601219 32126 ts_tablet_manager.cc:1434] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.086s	user 0.008s	sys 0.000s
I20260709 13:16:30.588690 32547 raft_consensus.cc:2749] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:30.603160 32127 tablet_bootstrap.cc:492] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:30.600499 32125 raft_consensus.cc:359] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.605540 32125 raft_consensus.cc:385] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.605984 32125 raft_consensus.cc:740] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.615379 32126 tablet_bootstrap.cc:492] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:30.616026 32282 consensus_queue.cc:1048] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:30.622161 32125 consensus_queue.cc:260] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.617244 31896 leader_election.cc:304] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:30.633042 32555 raft_consensus.cc:2804] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:30.633620 32555 raft_consensus.cc:697] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:30.623018 32127 tablet_bootstrap.cc:654] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.615851 32123 consensus_queue.cc:260] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.648512 32519 raft_consensus.cc:493] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.651086 32123 ts_tablet_manager.cc:1434] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.104s	user 0.010s	sys 0.000s
I20260709 13:16:30.653183 32126 tablet_bootstrap.cc:654] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.657693 32125 ts_tablet_manager.cc:1434] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.004s
I20260709 13:16:30.653995 32555 consensus_queue.cc:237] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.662303 32125 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:30.652486 32519 raft_consensus.cc:515] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.665271 32519 leader_election.cc:290] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.659322 32127 tablet_bootstrap.cc:492] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:30.666101 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.667145 32127 ts_tablet_manager.cc:1403] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.019s
I20260709 13:16:30.666256 32004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:30.669132 32044 leader_election.cc:343] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:30.670468 32042 leader_election.cc:343] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.670964 32042 leader_election.cc:304] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:30.671406 32127 raft_consensus.cc:359] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.672350 32519 raft_consensus.cc:2749] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:30.672781 32123 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:30.672438 32127 raft_consensus.cc:385] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.674355 32127 raft_consensus.cc:740] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.678525 32127 consensus_queue.cc:260] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.681357 32127 ts_tablet_manager.cc:1434] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.006s
I20260709 13:16:30.682826 32127 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:30.689320 32127 tablet_bootstrap.cc:654] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.689270 32125 tablet_bootstrap.cc:654] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.697543 32126 tablet_bootstrap.cc:492] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:30.701728 32126 ts_tablet_manager.cc:1403] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.001s
I20260709 13:16:30.707111 32126 raft_consensus.cc:359] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.708168 32126 raft_consensus.cc:385] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.708524 32126 raft_consensus.cc:740] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.716717 32126 consensus_queue.cc:260] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:30.719846 32126 ts_tablet_manager.cc:1434] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:16:30.721244 32126 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:30.736366 32125 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:30.736891 32123 tablet_bootstrap.cc:654] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.736886 32125 ts_tablet_manager.cc:1403] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.000s
I20260709 13:16:30.741659 32126 tablet_bootstrap.cc:654] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.744571 32125 raft_consensus.cc:359] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.745875 32125 raft_consensus.cc:385] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.746327 32125 raft_consensus.cc:740] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.747254 32125 consensus_queue.cc:260] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.752372 32125 ts_tablet_manager.cc:1434] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:16:30.752383 32123 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:30.753136 32123 ts_tablet_manager.cc:1403] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.000s
I20260709 13:16:30.752472 32127 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:30.754487 32127 ts_tablet_manager.cc:1403] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.072s	user 0.035s	sys 0.016s
I20260709 13:16:30.755191 32126 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:30.755733 32126 ts_tablet_manager.cc:1403] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.001s
I20260709 13:16:30.758139 32127 raft_consensus.cc:359] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.759567 32127 raft_consensus.cc:385] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.759372 32126 raft_consensus.cc:359] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.760056 32127 raft_consensus.cc:740] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.760344 32126 raft_consensus.cc:385] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.760687 32126 raft_consensus.cc:740] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.753609 32125 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:30.762331 32127 consensus_queue.cc:260] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.770583 32127 ts_tablet_manager.cc:1434] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.005s
I20260709 13:16:30.771832 32127 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:30.773244 32125 tablet_bootstrap.cc:654] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.776950 32126 consensus_queue.cc:260] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.779125 32126 ts_tablet_manager.cc:1434] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.003s
I20260709 13:16:30.779605 32127 tablet_bootstrap.cc:654] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.781435 32126 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:30.785878 32127 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:30.786528 32127 ts_tablet_manager.cc:1403] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.001s
I20260709 13:16:30.788292 32126 tablet_bootstrap.cc:654] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.788823 32125 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:30.789467 32125 ts_tablet_manager.cc:1403] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.000s
I20260709 13:16:30.790153 32127 raft_consensus.cc:359] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.790956 32127 raft_consensus.cc:385] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.791260 32127 raft_consensus.cc:740] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.792032 32127 consensus_queue.cc:260] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.794788 32127 ts_tablet_manager.cc:1434] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:30.794561 32125 raft_consensus.cc:359] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.795781 32125 raft_consensus.cc:385] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.796229 32125 raft_consensus.cc:740] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.797618 32127 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:30.801573 32126 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:30.802124 32126 ts_tablet_manager.cc:1403] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.011s
I20260709 13:16:30.806077 32127 tablet_bootstrap.cc:654] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.805619 32126 raft_consensus.cc:359] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.806531 32126 raft_consensus.cc:385] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.806849 32126 raft_consensus.cc:740] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.808137 32123 raft_consensus.cc:359] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.808905 32123 raft_consensus.cc:385] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.809237 32123 raft_consensus.cc:740] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.810240 32123 consensus_queue.cc:260] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.801112 32125 consensus_queue.cc:260] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.809238 32126 consensus_queue.cc:260] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.813862 32126 ts_tablet_manager.cc:1434] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:16:30.814643 32125 ts_tablet_manager.cc:1434] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.008s
I20260709 13:16:30.815289 32126 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887: Bootstrap starting.
I20260709 13:16:30.786614 31751 catalog_manager.cc:5697] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from ccf4710690c7473d814836da1babee51 (127.21.33.3) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.822177 32004 raft_consensus.cc:1275] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:30.820039 31751 catalog_manager.cc:5697] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } }
I20260709 13:16:30.818185 32555 raft_consensus.cc:493] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.817803 32125 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:30.829870 32555 raft_consensus.cc:515] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.833356 32566 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:30.848935 32488 raft_consensus.cc:493] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.849612 32488 raft_consensus.cc:515] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:30.851706 32547 raft_consensus.cc:493] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.852319 32547 raft_consensus.cc:515] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.854053 32126 tablet_bootstrap.cc:654] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:30.854260 32555 leader_election.cc:290] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:30.855192 32538 raft_consensus.cc:493] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.855734 32538 raft_consensus.cc:515] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.858489 32538 leader_election.cc:290] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:30.861775 32488 leader_election.cc:290] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:30.866297 31850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142b461239534bae915d26581eb87d21" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:30.868223 31850 raft_consensus.cc:2468] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:30.867444 31928 raft_consensus.cc:1275] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:30.873744 32547 leader_election.cc:290] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:30.874809 32488 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:30.876384 31998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.877135 31998 raft_consensus.cc:2468] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:30.878312 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:30.879477 31996 raft_consensus.cc:2393] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:30.879240 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:30.883836 31967 leader_election.cc:343] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:30.885071 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:30.885802 32074 raft_consensus.cc:2468] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:30.882525 31753 catalog_manager.cc:5697] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.887450 31821 leader_election.cc:304] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:30.888664 32566 raft_consensus.cc:2804] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.899830 32566 raft_consensus.cc:493] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.900189 31752 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34432 (request call id 70) took 1535 ms. Trace:
I20260709 13:16:30.900524 32566 raft_consensus.cc:3060] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.907099 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.907909 31926 raft_consensus.cc:2468] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
I20260709 13:16:30.909282 31851 raft_consensus.cc:1275] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:30.898382 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:30.914189 31819 leader_election.cc:304] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:30.914242 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142b461239534bae915d26581eb87d21" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:30.916229 32077 raft_consensus.cc:2468] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:30.917066 32126 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887: No bootstrap required, opened a new log
I20260709 13:16:30.917133 32538 raft_consensus.cc:3060] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.917769 32126 ts_tablet_manager.cc:1403] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887: Time spent bootstrapping tablet: real 0.103s	user 0.043s	sys 0.028s
I20260709 13:16:30.920904 32126 raft_consensus.cc:359] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.921628 32126 raft_consensus.cc:385] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:30.921940 32126 raft_consensus.cc:740] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Initialized, Role: FOLLOWER
I20260709 13:16:30.922583 32126 consensus_queue.cc:260] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.923964 31750 catalog_manager.cc:5697] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.925091 31895 leader_election.cc:304] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:30.929551 31922 raft_consensus.cc:1275] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:30.930020 32555 raft_consensus.cc:2804] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.930465 32555 raft_consensus.cc:493] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.930843 32555 raft_consensus.cc:3060] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.932500 31968 leader_election.cc:304] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:30.903990 31752 rpcz_store.cc:276] 0709 13:16:29.365261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.937396 (+572135us) service_pool.cc:224] Handling call
0709 13:16:30.900076 (+962680us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8240,"reported_tablets":5}
I20260709 13:16:30.934149 32546 raft_consensus.cc:2804] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:30.936635 32546 raft_consensus.cc:493] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.937005 32546 raft_consensus.cc:3060] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.943468 32555 raft_consensus.cc:515] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:30.935141 32297 consensus_queue.cc:1048] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.946961 32546 raft_consensus.cc:515] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:30.947504 32126 ts_tablet_manager.cc:1434] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:16:30.950482 32555 leader_election.cc:290] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:30.959699 32547 consensus_queue.cc:1048] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:30.985287 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142b461239534bae915d26581eb87d21" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:30.988094 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142b461239534bae915d26581eb87d21" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:30.988903 31855 raft_consensus.cc:3060] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.986120 32074 raft_consensus.cc:3060] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.997485 32546 leader_election.cc:290] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:31.000609 31855 raft_consensus.cc:2468] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:31.004989 32127 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:31.029387 32556 raft_consensus.cc:493] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.030454 32127 ts_tablet_manager.cc:1403] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.235s	user 0.011s	sys 0.016s
I20260709 13:16:31.030387 32556 raft_consensus.cc:515] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:31.033334 32556 leader_election.cc:290] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:31.029631 31850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:31.019323 32538 raft_consensus.cc:2749] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:31.018405 32566 raft_consensus.cc:515] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.037123 32566 leader_election.cc:290] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:31.024060 32074 raft_consensus.cc:2468] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:31.033914 32127 raft_consensus.cc:359] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.037830 32127 raft_consensus.cc:385] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:31.038121 32127 raft_consensus.cc:740] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:31.038848 32127 consensus_queue.cc:260] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.040380 32127 ts_tablet_manager.cc:1434] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.004s
I20260709 13:16:31.041335 32127 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:31.000798 32125 tablet_bootstrap.cc:654] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:31.044874 31967 leader_election.cc:343] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.045032 31998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:31.045853 31998 raft_consensus.cc:3060] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.051748 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:31.052510 32127 tablet_bootstrap.cc:654] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:31.052700 32074 raft_consensus.cc:3060] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.053894 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:31.054668 31996 raft_consensus.cc:2468] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:31.055871 31896 leader_election.cc:304] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:31.058926 32555 raft_consensus.cc:2804] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:31.059337 32555 raft_consensus.cc:493] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.059669 32555 raft_consensus.cc:3060] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.829191 32123 ts_tablet_manager.cc:1434] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 13:16:31.061897 31753 catalog_manager.cc:5697] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.063058 31895 leader_election.cc:304] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:31.077489 31753 catalog_manager.cc:5697] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 1 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:31.077013 31876 debug-util.cc:398] Leaking SignalData structure 0x7b08002fab40 after lost signal to thread 21639
I20260709 13:16:31.086295 32515 raft_consensus.cc:2804] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:31.114450 32123 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:31.085296 32516 raft_consensus.cc:493] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.087142 32355 raft_consensus.cc:493] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.099172 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:31.106251 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:31.111501 32546 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.119124 32594 raft_consensus.cc:493] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.125530 32355 raft_consensus.cc:515] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:31.134693 31876 debug-util.cc:398] Leaking SignalData structure 0x7b0800234340 after lost signal to thread 31787
W20260709 13:16:31.146456 31876 debug-util.cc:398] Leaking SignalData structure 0x7b08003f6040 after lost signal to thread 31879
W20260709 13:16:31.147886 31876 debug-util.cc:398] Leaking SignalData structure 0x7b0800225460 after lost signal to thread 31954
I20260709 13:16:31.148437 32540 consensus_queue.cc:1048] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:31.134984 32594 raft_consensus.cc:515] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.131892 31922 raft_consensus.cc:3060] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.152107 32484 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.130977 32075 raft_consensus.cc:2468] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:31.133847 32515 raft_consensus.cc:697] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:31.124640 32516 raft_consensus.cc:515] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:31.132380 32546 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.176806 32123 tablet_bootstrap.cc:654] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:31.204023 32125 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:31.204741 32125 ts_tablet_manager.cc:1403] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.387s	user 0.012s	sys 0.018s
I20260709 13:16:31.159190 32594 leader_election.cc:290] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:16:31.195577 31876 debug-util.cc:398] Leaking SignalData structure 0x7b08002831c0 after lost signal to thread 32028
I20260709 13:16:31.175575 31850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:31.233433 32127 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:31.234371 32127 ts_tablet_manager.cc:1403] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.193s	user 0.015s	sys 0.010s
W20260709 13:16:31.235831 32175 log.cc:927] Time spent T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:16:31.174036 32484 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:31.236693 31998 raft_consensus.cc:2468] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:31.258147 32516 leader_election.cc:290] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:31.251359 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:31.277915 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:16:31.273025 31876 debug-util.cc:398] Leaking SignalData structure 0x7b08002c2e40 after lost signal to thread 32102
I20260709 13:16:31.281152 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:31.289163 32355 leader_election.cc:290] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:31.290733 32623 log.cc:927] Time spent T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:16:31.291177 32546 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:31.293056 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:31.268769 32074 raft_consensus.cc:2468] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
W20260709 13:16:31.300297 31969 leader_election.cc:343] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.300983 31922 raft_consensus.cc:2468] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
W20260709 13:16:31.303325 31969 leader_election.cc:343] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.304801 32484 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:31.307507 31821 leader_election.cc:304] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:31.308218 32626 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:31.310338 31896 leader_election.cc:343] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:31.312806 31967 leader_election.cc:343] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.313319 31967 leader_election.cc:304] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:31.314600 32127 raft_consensus.cc:359] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:31.315181 32127 raft_consensus.cc:385] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:31.315419 32127 raft_consensus.cc:740] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:31.316963 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:31.316114 32127 consensus_queue.cc:260] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:31.311707 31998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d352eb04d043328db56fab2171304e" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:31.337181 31967 leader_election.cc:343] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.337754 31967 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:31.340656 32297 raft_consensus.cc:2749] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:31.341572 31968 leader_election.cc:304] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:31.281420 32516 consensus_queue.cc:1048] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:16:31.359663 31855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d352eb04d043328db56fab2171304e" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:31.355103 32127 ts_tablet_manager.cc:1434] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.120s	user 0.007s	sys 0.001s
I20260709 13:16:31.319731 32566 raft_consensus.cc:2804] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:31.308794 32626 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:31.329818 32594 raft_consensus.cc:2749] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:31.254684 32515 consensus_queue.cc:237] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:16:31.377287 31893 leader_election.cc:343] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:31.385349 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 102937us
I20260709 13:16:31.363430 32547 raft_consensus.cc:2804] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:31.388475 32125 raft_consensus.cc:359] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.389439 32125 raft_consensus.cc:385] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:31.389758 32125 raft_consensus.cc:740] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:31.392598 32127 tablet_bootstrap.cc:492] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: Bootstrap starting.
I20260709 13:16:31.365137 31998 raft_consensus.cc:2468] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:31.390537 32125 consensus_queue.cc:260] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.381172 32566 raft_consensus.cc:697] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:31.383392 31851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:31.384799 31821 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 76080.5us
I20260709 13:16:31.441721 32125 ts_tablet_manager.cc:1434] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.237s	user 0.009s	sys 0.000s
I20260709 13:16:31.401295 32547 raft_consensus.cc:697] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:31.450910 32123 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:31.451555 32123 ts_tablet_manager.cc:1403] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.354s	user 0.028s	sys 0.025s
I20260709 13:16:31.454869 31896 leader_election.cc:304] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
W20260709 13:16:31.465651 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 65007.6us
I20260709 13:16:31.438261 31851 raft_consensus.cc:2468] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 0.
I20260709 13:16:31.474575 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:31.464114 32642 consensus_queue.cc:1048] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:16:31.478060 32649 raft_consensus.cc:2804] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:31.504479 32649 raft_consensus.cc:493] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.509968 32649 raft_consensus.cc:3060] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.513477 32626 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:31.503103 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:31.478035 32566 consensus_queue.cc:237] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.544370 32123 raft_consensus.cc:359] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.545835 32123 raft_consensus.cc:385] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:31.551103 32555 raft_consensus.cc:515] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:31.554073 32555 leader_election.cc:290] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:31.555663 32125 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:31.577271 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 93220.4us
I20260709 13:16:31.546123 32123 raft_consensus.cc:740] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:31.587338 32127 tablet_bootstrap.cc:654] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:31.632382 31896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 53619.6us
I20260709 13:16:31.637094 32123 consensus_queue.cc:260] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.499235 32547 consensus_queue.cc:237] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:31.661274 31820 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 157796us
W20260709 13:16:31.696403 32042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 242414us
I20260709 13:16:31.706146 32123 ts_tablet_manager.cc:1434] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.254s	user 0.011s	sys 0.000s
I20260709 13:16:31.734205 32123 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:31.806137 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:31.807343 31926 raft_consensus.cc:2468] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:31.822722 32488 raft_consensus.cc:493] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.849329 32125 tablet_bootstrap.cc:654] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:31.854218 31819 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:31.857290 32644 raft_consensus.cc:2804] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:31.844929 32488 raft_consensus.cc:515] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:31.869043 32644 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.879230 32644 raft_consensus.cc:3060] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.879439 32488 leader_election.cc:290] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:31.890373 32123 tablet_bootstrap.cc:654] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:31.892022 32644 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:31.894758 32644 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:31.900812 31893 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:31.904588 32631 raft_consensus.cc:2804] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:31.905052 32631 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.905433 32631 raft_consensus.cc:3060] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.906014 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:31.896970 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 404672us
I20260709 13:16:31.922637 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:31.923692 32077 raft_consensus.cc:2468] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:31.928030 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:31.911957 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:31.931178 32001 raft_consensus.cc:2468] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:31.938862 32649 raft_consensus.cc:515] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:31.947263 31820 leader_election.cc:343] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.956457 31821 leader_election.cc:304] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:31.969726 32488 raft_consensus.cc:2804] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:31.970285 32488 raft_consensus.cc:493] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.970674 32488 raft_consensus.cc:3060] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:31.974730 32127 tablet_bootstrap.cc:492] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: No bootstrap required, opened a new log
I20260709 13:16:31.975467 32127 ts_tablet_manager.cc:1403] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: Time spent bootstrapping tablet: real 0.589s	user 0.012s	sys 0.010s
W20260709 13:16:31.984458 31895 leader_election.cc:343] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:31.985101 31895 leader_election.cc:304] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:31.982343 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:31.988512 32072 raft_consensus.cc:3060] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.005920 32637 raft_consensus.cc:2749] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:31.951234 32555 consensus_queue.cc:1048] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
W20260709 13:16:32.028395 31895 leader_election.cc:343] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:32.087240 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:32.088351 31997 raft_consensus.cc:3060] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.091609 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
W20260709 13:16:32.096114 31820 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 134505us
I20260709 13:16:32.097687 32127 raft_consensus.cc:359] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:32.098773 32127 raft_consensus.cc:385] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:32.099092 32649 leader_election.cc:290] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:32.115731 31998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d352eb04d043328db56fab2171304e" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:32.116441 31998 raft_consensus.cc:3060] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.128401 32633 consensus_queue.cc:1048] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:32.114826 31750 catalog_manager.cc:5697] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:32.143775 31820 leader_election.cc:343] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:32.152055 32566 consensus_queue.cc:1048] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:32.153892 31849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d352eb04d043328db56fab2171304e" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:32.099138 32127 raft_consensus.cc:740] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Initialized, Role: FOLLOWER
I20260709 13:16:32.163146 32127 consensus_queue.cc:260] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:32.271085 32127 ts_tablet_manager.cc:1434] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: Time spent starting tablet: real 0.295s	user 0.010s	sys 0.001s
W20260709 13:16:32.287307 32661 log.cc:927] Time spent T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:32.373560 31998 raft_consensus.cc:2468] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:32.374540 31997 raft_consensus.cc:2468] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
W20260709 13:16:32.375555 32667 log.cc:927] Time spent T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: Append to log took a long time: real 0.349s	user 0.007s	sys 0.000s
W20260709 13:16:32.381230 31893 leader_election.cc:343] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:32.387841 32125 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:32.385438 32488 raft_consensus.cc:515] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:32.388495 32125 ts_tablet_manager.cc:1403] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.833s	user 0.084s	sys 0.108s
I20260709 13:16:32.390668 31896 leader_election.cc:304] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:32.392169 32504 raft_consensus.cc:2804] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:32.392833 32504 raft_consensus.cc:697] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:32.394517 32488 leader_election.cc:290] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:32.394136 32504 consensus_queue.cc:237] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:32.411518 32123 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:32.412272 32123 ts_tablet_manager.cc:1403] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.692s	user 0.025s	sys 0.011s
I20260709 13:16:32.367090 32072 raft_consensus.cc:2468] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:32.392554 32631 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:32.424913 32125 raft_consensus.cc:359] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:32.420341 31895 leader_election.cc:304] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:32.421321 31751 catalog_manager.cc:5697] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } }
I20260709 13:16:32.426512 32125 raft_consensus.cc:385] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:32.426914 32125 raft_consensus.cc:740] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
W20260709 13:16:32.433230 31819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 198476us
I20260709 13:16:32.434358 32125 consensus_queue.cc:260] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:32.438804 32633 raft_consensus.cc:2804] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:32.453408 31924 raft_consensus.cc:2393] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 103d6e4c312a47ff9b258c14ac2871e5 in current term 1: Already voted for candidate a588481f03f6474d9009d5bd5e08e887 in this term.
I20260709 13:16:32.456898 32123 raft_consensus.cc:359] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:32.459810 32123 raft_consensus.cc:385] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:32.460243 32123 raft_consensus.cc:740] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:32.467231 32125 ts_tablet_manager.cc:1434] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.003s
W20260709 13:16:32.518150 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 87926.7us
I20260709 13:16:32.486943 32123 consensus_queue.cc:260] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:16:32.576563 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 113745us
W20260709 13:16:32.582504 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 103319us
W20260709 13:16:32.557113 32673 log.cc:927] Time spent T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:16:32.576295 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:32.623085 32071 raft_consensus.cc:3060] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.659731 32125 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:32.667171 31853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
I20260709 13:16:32.668169 31853 raft_consensus.cc:2393] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a588481f03f6474d9009d5bd5e08e887 in current term 1: Already voted for candidate 103d6e4c312a47ff9b258c14ac2871e5 in this term.
W20260709 13:16:32.685987 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 200054us
I20260709 13:16:32.683153 31855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56618 (request call id 129) took 1052 ms. Trace:
I20260709 13:16:32.661180 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:32.661648 32631 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:32.682636 32633 raft_consensus.cc:697] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:32.691466 31855 rpcz_store.cc:276] 0709 13:16:31.626683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.626883 (+   200us) service_pool.cc:224] Handling call
0709 13:16:31.997887 (+371004us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b5800004298. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e9ccb71 00007f689e9cca22 00007f68a78e5c09 00007f68a6a3fee0 00007f68a6b090ab 00007f68a6b09131 00007f68a6acafc0 00007f68a6ab5929 00007f68a6ab53d2 00007f68a002cbd4
0709 13:16:31.998151 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:31.998634 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:31.998680 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:31.999460 (+   780us) log.cc:844] Serialized 26 byte log entry
0709 13:16:32.071917 (+ 72457us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b18003a53a0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689e9e6e07 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b 00007f68a1690d23 00007f68a16910d1 00007f68a1aaca0f
0709 13:16:32.354606 (+282689us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:32.454313 (+ 99707us) log.cc:844] Serialized 14 byte log entry
0709 13:16:32.678339 (+224026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:32.678431 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:32.678481 (+    50us) raft_consensus.cc:1685] finished
0709 13:16:32.678495 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:32.678621 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4238,"prepare.queue_time_us":164806,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":3904,"raft.queue_time_us":66084,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":2013,"spinlock_wait_cycles":81178624,"thread_start_us":562549,"threads_started":3,"wal-append.queue_time_us":395279}
I20260709 13:16:32.713925 32633 consensus_queue.cc:237] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:32.701938 31996 raft_consensus.cc:3060] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:32.743026 31967 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.744338 32071 raft_consensus.cc:2468] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:32.751628 31996 raft_consensus.cc:2468] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:32.753384 31821 leader_election.cc:304] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:32.755578 32684 raft_consensus.cc:2804] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:32.756074 32684 raft_consensus.cc:697] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:32.756901 32684 consensus_queue.cc:237] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:32.762610 32125 tablet_bootstrap.cc:654] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:32.777582 32123 ts_tablet_manager.cc:1434] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.362s	user 0.022s	sys 0.013s
I20260709 13:16:32.784778 32123 tablet_bootstrap.cc:492] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:32.800211 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:32.801074 32074 raft_consensus.cc:3060] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.819851 32547 consensus_queue.cc:1048] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:32.839920 32566 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.840601 32566 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:32.842836 32566 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:32.847278 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:32.848050 31996 raft_consensus.cc:2468] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 0.
I20260709 13:16:32.847965 32684 consensus_queue.cc:1048] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:32.850360 31819 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:32.851143 32684 raft_consensus.cc:2749] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:32.851972 31821 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:32.852898 32684 raft_consensus.cc:2804] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:32.853353 32684 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.853746 32684 raft_consensus.cc:3060] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.859180 32649 consensus_queue.cc:1048] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:32.867877 32684 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:32.876397 32545 raft_consensus.cc:493] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.879724 32545 raft_consensus.cc:515] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:32.882835 32629 consensus_queue.cc:1048] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:16:32.907579 32545 leader_election.cc:290] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:32.952869 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:32.932520 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:32.954957 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
W20260709 13:16:32.960456 31820 leader_election.cc:343] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:32.972976 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:32.961589 32074 raft_consensus.cc:2468] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 1.
I20260709 13:16:32.954490 32684 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:32.966377 31997 raft_consensus.cc:3060] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:32.994170 32123 tablet_bootstrap.cc:654] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:33.003510 31895 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:33.002177 31922 raft_consensus.cc:2468] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 0.
W20260709 13:16:33.004259 31820 leader_election.cc:343] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:33.009330 31800 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:33.009680 32632 raft_consensus.cc:2804] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:33.023294 31968 leader_election.cc:304] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:33.028805 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:33.029888 32125 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
I20260709 13:16:33.030437 32125 ts_tablet_manager.cc:1403] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.402s	user 0.043s	sys 0.038s
I20260709 13:16:33.031199 32654 raft_consensus.cc:2804] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:33.031620 32654 raft_consensus.cc:493] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.031936 32654 raft_consensus.cc:3060] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:33.039923 31969 leader_election.cc:343] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:33.041275 32632 raft_consensus.cc:697] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:33.043243 32125 raft_consensus.cc:359] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:33.043960 32125 raft_consensus.cc:385] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:33.044241 32125 raft_consensus.cc:740] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:33.044906 32125 consensus_queue.cc:260] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:33.046885 32125 ts_tablet_manager.cc:1434] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:16:33.047945 32125 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:33.042248 32632 consensus_queue.cc:237] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:33.096434 32125 tablet_bootstrap.cc:654] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:33.109009 32123 tablet_bootstrap.cc:492] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:33.109699 32123 ts_tablet_manager.cc:1403] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.325s	user 0.019s	sys 0.003s
I20260709 13:16:33.107645 32654 raft_consensus.cc:515] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:33.112486 32654 leader_election.cc:290] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:33.113580 32125 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: No bootstrap required, opened a new log
W20260709 13:16:33.186344 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:33.188138 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 106136us
I20260709 13:16:33.226691 32125 ts_tablet_manager.cc:1403] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.179s	user 0.014s	sys 0.005s
I20260709 13:16:33.248705 32488 consensus_queue.cc:1048] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:33.213374 31997 raft_consensus.cc:2468] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 1.
I20260709 13:16:33.265911 31821 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 32690b3987094d51bb4f214321a29067
I20260709 13:16:33.271443 32700 raft_consensus.cc:2804] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:33.272491 32700 raft_consensus.cc:697] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:33.274432 32700 consensus_queue.cc:237] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:33.303324 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:33.304505 32123 raft_consensus.cc:359] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:33.307071 32123 raft_consensus.cc:385] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:33.309731 32123 raft_consensus.cc:740] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
I20260709 13:16:33.314635 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:33.315783 31926 raft_consensus.cc:3060] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:33.315868 32684 consensus_queue.cc:1048] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:16:33.309341 31750 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34408 (request call id 69) took 1254 ms. Trace:
I20260709 13:16:33.323858 31752 catalog_manager.cc:5697] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.322604 31750 rpcz_store.cc:276] 0709 13:16:32.065553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.065735 (+   182us) service_pool.cc:224] Handling call
0709 13:16:33.309074 (+1243339us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:32.399701 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:32.400433 (+   732us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:32.400474 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:16:32.400497 (+    23us) tablet.cc:662] Decoding operations
0709 13:16:32.400840 (+   343us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:32.400921 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:32.400957 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:32.401146 (+   189us) tablet.cc:701] Row locks acquired
0709 13:16:32.401170 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:16:32.401309 (+   139us) write_op.cc:270] Start()
0709 13:16:32.401438 (+   129us) write_op.cc:276] Timestamp: P: 1783602992401264 usec, L: 0
0709 13:16:32.401460 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:16:32.457208 (+ 55748us) log.cc:844] Serialized 534 byte log entry
0709 13:16:32.804411 (+347203us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b4400022b4c. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f689e88cdb1 00007f689e92ac85 00007f689e92b6a2 00007f68a169d1db 00007f68a1698aaf 00007f689e9e70ac 00007f68a7841ab9 00007f689ea0369b 00007f689ea0193f 00007f689ea03c02 00007f68a168b20b
0709 13:16:33.088688 (+284277us) op_driver.cc:464] REPLICATION: finished
0709 13:16:33.230988 (+142300us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b18000e47c0. stack: 00007f689e9d2343 00007f689e9d2240 00007f689e3611d4 00007f68a809f304 00007f68a809f299 00007f68a78ccde1 00007f68a78d95b1 00007f689ea043f3 00007f689ea067c1 00007f689ea06779 00007f689ea06709 00007f689ea066d1 00007f689ea059cd 000000000038f354 000000000038f199 00007f689e9e783a
0709 13:16:33.231442 (+   454us) write_op.cc:301] APPLY: starting
0709 13:16:33.231680 (+   238us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:33.231998 (+   318us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:33.232051 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:33.232626 (+   575us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:33.233143 (+   517us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:33.233311 (+   168us) write_op.cc:312] APPLY: finished
0709 13:16:33.260999 (+ 27688us) log.cc:844] Serialized 22 byte log entry
0709 13:16:33.261310 (+   311us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:33.261570 (+   260us) write_op.cc:454] Released schema lock
0709 13:16:33.261649 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:33.262520 (+   871us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:33.262552 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22120,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":22279,"apply.run_cpu_time_us":3921,"apply.run_wall_time_us":31103,"num_ops":1,"prepare.queue_time_us":256389,"prepare.run_cpu_time_us":30179,"prepare.run_wall_time_us":533988,"raft.queue_time_us":107747,"raft.run_cpu_time_us":16993,"raft.run_wall_time_us":23909,"replication_time_us":687057,"spinlock_wait_cycles":20860160,"thread_start_us":815174,"threads_started":4,"wal-append.queue_time_us":501367}]]}
I20260709 13:16:33.341351 32657 consensus_queue.cc:1048] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:33.350560 32123 consensus_queue.cc:260] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:33.376178 32123 ts_tablet_manager.cc:1434] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.266s	user 0.003s	sys 0.009s
I20260709 13:16:33.269145 32125 raft_consensus.cc:359] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:33.377563 32125 raft_consensus.cc:385] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:33.377983 32125 raft_consensus.cc:740] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:33.379092 32125 consensus_queue.cc:260] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:33.376757 32074 raft_consensus.cc:3060] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:33.383984 31926 raft_consensus.cc:2468] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:33.396409 32123 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067: Bootstrap starting.
I20260709 13:16:33.412479 32125 ts_tablet_manager.cc:1434] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.185s	user 0.011s	sys 0.002s
I20260709 13:16:33.444816 32656 raft_consensus.cc:493] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.445906 32656 raft_consensus.cc:515] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:33.449180 32656 leader_election.cc:290] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:33.465517 31968 leader_election.cc:304] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:33.477536 32555 consensus_queue.cc:1048] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:33.479314 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d352eb04d043328db56fab2171304e" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:33.481671 32683 raft_consensus.cc:2804] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:33.482357 32683 raft_consensus.cc:697] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:33.483714 32683 consensus_queue.cc:237] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:33.473067 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48d352eb04d043328db56fab2171304e" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:33.508221 31821 leader_election.cc:304] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:33.517807 32700 raft_consensus.cc:2749] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:33.532018 32699 raft_consensus.cc:493] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.532793 32699 raft_consensus.cc:515] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:33.537873 32699 leader_election.cc:290] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:33.540795 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:33.541514 31853 raft_consensus.cc:3060] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:33.543527 32003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e90fd5526b4a2fbfac6d06dc90f086" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:33.547253 31819 leader_election.cc:304] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:33.573869 32074 raft_consensus.cc:2468] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
W20260709 13:16:33.597220 32725 log.cc:927] Time spent T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: Append to log took a long time: real 0.189s	user 0.002s	sys 0.000s
I20260709 13:16:33.588928 31752 catalog_manager.cc:5697] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.607630 32653 consensus_queue.cc:1048] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:16:33.636255 31752 catalog_manager.cc:5697] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.668347 32123 tablet_bootstrap.cc:654] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:33.668139 32649 consensus_queue.cc:1048] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:16:33.671967 31850 raft_consensus.cc:3060] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:33.696570 32699 consensus_queue.cc:1048] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:33.645733 31752 catalog_manager.cc:5697] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 0 to 1, leader changed from <none> to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.866671 31750 catalog_manager.cc:5697] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.887482 32123 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067: No bootstrap required, opened a new log
I20260709 13:16:33.892081 32123 ts_tablet_manager.cc:1403] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067: Time spent bootstrapping tablet: real 0.496s	user 0.024s	sys 0.053s
I20260709 13:16:33.893074 31750 catalog_manager.cc:5697] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:33.899451 31819 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 49004.9us
I20260709 13:16:33.907366 32123 raft_consensus.cc:359] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:33.915035 32123 raft_consensus.cc:385] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:33.915313 32123 raft_consensus.cc:740] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Initialized, Role: FOLLOWER
W20260709 13:16:33.922901 31820 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 59505.4us
I20260709 13:16:33.934917 32123 consensus_queue.cc:260] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:33.938184 32123 ts_tablet_manager.cc:1434] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067: Time spent starting tablet: real 0.046s	user 0.003s	sys 0.005s
I20260709 13:16:33.947678 32656 raft_consensus.cc:2749] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:33.959062 32076 raft_consensus.cc:3060] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:33.979799 31753 catalog_manager.cc:5697] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:34.027915 32656 consensus_queue.cc:1048] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:34.034307 32642 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.045904 32642 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:34.044725 32545 consensus_queue.cc:1048] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:34.054422 32642 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:34.055474 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:34.057873 31850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:34.061566 32044 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:34.064402 32642 raft_consensus.cc:2749] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:34.119459 32654 consensus_queue.cc:1048] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:35.334686 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
W20260709 13:16:35.356228 31893 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:16:35.371004 31893 consensus_peers.cc:597] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.371701 31893 consensus_peers.cc:597] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.382184 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.415123 31967 consensus_peers.cc:597] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.421186 31967 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.431469 31893 consensus_peers.cc:597] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.435416 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.453703 31967 consensus_peers.cc:597] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.457346 31893 consensus_peers.cc:597] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.458196 31893 consensus_peers.cc:597] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.469913 31967 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.472416 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:35.475643 21636 tablet_replica.cc:333] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.476833 21636 raft_consensus.cc:2243] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.477455 21636 raft_consensus.cc:2272] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.481892 21636 tablet_replica.cc:333] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.482831 21636 raft_consensus.cc:2243] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.483374 21636 raft_consensus.cc:2272] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.484122 32041 consensus_peers.cc:597] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.500622 21636 tablet_replica.cc:333] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.502444 21636 raft_consensus.cc:2243] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.503659 21636 raft_consensus.cc:2272] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.511257 21636 tablet_replica.cc:333] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.512396 21636 raft_consensus.cc:2243] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.513036 21636 raft_consensus.cc:2272] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.516523 21636 tablet_replica.cc:333] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.517660 21636 raft_consensus.cc:2243] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:35.518126 32041 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.518286 21636 raft_consensus.cc:2272] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.521947 21636 tablet_replica.cc:333] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.522967 21636 raft_consensus.cc:2243] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.523590 21636 raft_consensus.cc:2272] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.527477 21636 tablet_replica.cc:333] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.528834 21636 raft_consensus.cc:2243] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.530464 21636 raft_consensus.cc:2272] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.531522 31893 consensus_peers.cc:597] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.534237 21636 tablet_replica.cc:333] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.535413 21636 raft_consensus.cc:2243] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.536113 21636 raft_consensus.cc:2272] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.538551 31893 consensus_peers.cc:597] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.539844 21636 tablet_replica.cc:333] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.540820 21636 raft_consensus.cc:2243] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.541428 21636 raft_consensus.cc:2272] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.546865 21636 tablet_replica.cc:333] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
W20260709 13:16:35.547665 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.547911 21636 raft_consensus.cc:2243] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.548902 21636 raft_consensus.cc:2272] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.553400 21636 tablet_replica.cc:333] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.554517 21636 raft_consensus.cc:2243] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.555431 21636 raft_consensus.cc:2272] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.560321 21636 tablet_replica.cc:333] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.561368 21636 raft_consensus.cc:2243] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.562094 21636 raft_consensus.cc:2272] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.565814 21636 tablet_replica.cc:333] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.566862 21636 raft_consensus.cc:2243] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.568141 21636 raft_consensus.cc:2272] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.572217 21636 tablet_replica.cc:333] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.573885 21636 raft_consensus.cc:2243] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.574586 21636 raft_consensus.cc:2272] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.579126 21636 tablet_replica.cc:333] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.580214 21636 raft_consensus.cc:2243] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.581420 21636 raft_consensus.cc:2272] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.584980 21636 tablet_replica.cc:333] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.585985 21636 raft_consensus.cc:2243] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.586628 21636 raft_consensus.cc:2272] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.586958 31967 consensus_peers.cc:597] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.593034 21636 tablet_replica.cc:333] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.594584 21636 raft_consensus.cc:2243] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.595552 21636 raft_consensus.cc:2272] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.599344 21636 tablet_replica.cc:333] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.600291 21636 raft_consensus.cc:2243] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.602087 21636 raft_consensus.cc:2272] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.604573 31893 consensus_peers.cc:597] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.605639 21636 tablet_replica.cc:333] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.606573 21636 raft_consensus.cc:2243] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.607276 21636 raft_consensus.cc:2272] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.611182 21636 tablet_replica.cc:333] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.612293 21636 raft_consensus.cc:2243] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.612921 21636 raft_consensus.cc:2272] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.616425 21636 tablet_replica.cc:333] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.617547 21636 raft_consensus.cc:2243] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.618170 21636 raft_consensus.cc:2272] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.620236 31893 consensus_peers.cc:597] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.621106 31893 consensus_peers.cc:597] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.621650 21636 tablet_replica.cc:333] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.622597 21636 raft_consensus.cc:2243] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.623327 21636 raft_consensus.cc:2272] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.625056 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.627300 21636 tablet_replica.cc:333] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.628315 21636 raft_consensus.cc:2243] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.629025 21636 raft_consensus.cc:2272] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.632457 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.632758 21636 tablet_replica.cc:333] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.633699 21636 raft_consensus.cc:2243] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.634361 21636 raft_consensus.cc:2272] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.637480 21636 tablet_replica.cc:333] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.638334 21636 raft_consensus.cc:2243] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.638863 21636 raft_consensus.cc:2272] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.641999 21636 tablet_replica.cc:333] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.642923 21636 raft_consensus.cc:2243] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.643535 21636 raft_consensus.cc:2272] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.646721 21636 tablet_replica.cc:333] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.647591 21636 raft_consensus.cc:2243] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:16:35.647701 32041 consensus_peers.cc:597] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.648749 21636 raft_consensus.cc:2272] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.651325 21636 tablet_replica.cc:333] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.652154 21636 raft_consensus.cc:2243] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.652717 21636 raft_consensus.cc:2272] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.655642 21636 tablet_replica.cc:333] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.656462 21636 raft_consensus.cc:2243] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.657106 21636 raft_consensus.cc:2272] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.660064 32041 consensus_peers.cc:597] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.660794 21636 tablet_replica.cc:333] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.661698 21636 raft_consensus.cc:2243] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.662683 21636 raft_consensus.cc:2272] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.665382 21636 tablet_replica.cc:333] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.666225 21636 raft_consensus.cc:2243] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.666836 21636 raft_consensus.cc:2272] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.670665 21636 tablet_replica.cc:333] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.671515 21636 raft_consensus.cc:2243] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.672175 21636 raft_consensus.cc:2272] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.675634 21636 tablet_replica.cc:333] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.676615 21636 raft_consensus.cc:2243] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.677345 21636 raft_consensus.cc:2272] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.681689 21636 tablet_replica.cc:333] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.682648 21636 raft_consensus.cc:2243] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.683399 21636 raft_consensus.cc:2272] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.687561 21636 tablet_replica.cc:333] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.688458 21636 raft_consensus.cc:2243] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.689819 21636 raft_consensus.cc:2272] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.693419 21636 tablet_replica.cc:333] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.694334 21636 raft_consensus.cc:2243] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.695690 21636 raft_consensus.cc:2272] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.696027 31893 consensus_peers.cc:597] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.696761 31893 consensus_peers.cc:597] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.699008 21636 tablet_replica.cc:333] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.699887 21636 raft_consensus.cc:2243] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.700909 21636 raft_consensus.cc:2272] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.704274 21636 tablet_replica.cc:333] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.705084 21636 raft_consensus.cc:2243] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.705770 21636 raft_consensus.cc:2272] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.709479 21636 tablet_replica.cc:333] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.710479 21636 raft_consensus.cc:2243] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.711144 21636 raft_consensus.cc:2272] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.714541 21636 tablet_replica.cc:333] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.715292 21636 raft_consensus.cc:2243] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.715955 21636 raft_consensus.cc:2272] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.719383 21636 tablet_replica.cc:333] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.719983 21636 raft_consensus.cc:2243] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.720561 21636 raft_consensus.cc:2272] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.723487 21636 tablet_replica.cc:333] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.724159 21636 raft_consensus.cc:2243] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.724665 21636 raft_consensus.cc:2272] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.728123 21636 tablet_replica.cc:333] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.728979 21636 raft_consensus.cc:2243] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.730238 21636 raft_consensus.cc:2272] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.731992 31893 consensus_peers.cc:597] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.734048 21636 tablet_replica.cc:333] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.735049 21636 raft_consensus.cc:2243] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:35.736312 21636 raft_consensus.cc:2272] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:35.739477 21636 tablet_replica.cc:333] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: stopping tablet replica
I20260709 13:16:35.740191 21636 raft_consensus.cc:2243] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:35.740746 21636 raft_consensus.cc:2272] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:35.791285 31893 consensus_peers.cc:597] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.807660 31967 consensus_peers.cc:597] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.839504 32041 connection.cc:573] client connection to 127.21.33.1:35847 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:16:35.901939 31967 consensus_peers.cc:597] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.908602 31893 consensus_peers.cc:597] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.917886 32041 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.923040 31893 consensus_peers.cc:597] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:35.937703 31893 consensus_peers.cc:597] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:36.519023 32642 raft_consensus.cc:493] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.519652 32642 raft_consensus.cc:515] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.523259 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:36.526847 32642 leader_election.cc:290] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:16:36.543949 32041 leader_election.cc:336] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.544509 32041 leader_election.cc:304] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:36.547334 32647 raft_consensus.cc:2749] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.553390 32642 raft_consensus.cc:493] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.554081 32642 raft_consensus.cc:515] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.557708 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:36.560578 32642 leader_election.cc:290] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:16:36.567426 32041 leader_election.cc:336] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.568087 32041 leader_election.cc:304] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:36.568915 32642 raft_consensus.cc:2749] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.704913 32555 raft_consensus.cc:493] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.705614 32555 raft_consensus.cc:515] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.708262 32555 leader_election.cc:290] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.712383 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:36.715303 31893 leader_election.cc:336] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.715740 31893 leader_election.cc:304] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:36.716384 32555 raft_consensus.cc:2749] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.727844 32555 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.728395 32555 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.730377 32555 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.734601 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:36.736224 31893 leader_election.cc:336] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.737044 31895 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:36.738061 32555 raft_consensus.cc:2749] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.740810 32642 raft_consensus.cc:493] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.741479 32642 raft_consensus.cc:515] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.744179 32642 leader_election.cc:290] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:36.744800 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:36.745348 31924 raft_consensus.cc:2468] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:36.746799 32042 leader_election.cc:304] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:36.747035 32748 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.747802 32642 raft_consensus.cc:2804] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:36.747575 32748 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:36.748308 32642 raft_consensus.cc:493] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.748764 32642 raft_consensus.cc:3060] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:36.749017 32041 leader_election.cc:336] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.749883 32555 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.751384 32555 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.753608 32555 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.754299 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:36.758214 32748 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:36.759642 32642 raft_consensus.cc:515] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:16:36.762136 31893 leader_election.cc:336] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.766081 32642 leader_election.cc:290] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:36.766350 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc5e72ae63b4c96bc88199a467670e2" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:36.763908 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:36.767035 31924 raft_consensus.cc:3060] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:36.761264 31967 leader_election.cc:336] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.771029 31895 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:36.769945 31967 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:36.773102 32555 raft_consensus.cc:2749] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.774294 32748 raft_consensus.cc:2749] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:36.776685 32041 leader_election.cc:336] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.778823 31924 raft_consensus.cc:2468] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:36.780999 32042 leader_election.cc:304] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:36.787374 32642 raft_consensus.cc:2804] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:36.790222 32642 raft_consensus.cc:697] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:36.791345 32642 consensus_queue.cc:237] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.804210 32654 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.805164 32654 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.808902 32654 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.809947 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:36.813925 32750 raft_consensus.cc:493] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.814574 32750 raft_consensus.cc:515] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.818539 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:36.821169 32750 leader_election.cc:290] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.824154 32555 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.824990 32555 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:36.830896 32041 leader_election.cc:336] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:36.831248 31967 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.831494 32041 leader_election.cc:304] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:36.831679 31967 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:36.832806 32647 raft_consensus.cc:2749] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.832908 32654 raft_consensus.cc:2749] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.845415 32555 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:36.845842 21636 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 31876)
I20260709 13:16:36.849386 32654 raft_consensus.cc:493] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.850039 32654 raft_consensus.cc:515] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:36.852275 31893 leader_election.cc:336] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.853211 31753 catalog_manager.cc:5697] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:36.855533 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:36.856488 32073 raft_consensus.cc:2468] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:36.853246 32654 leader_election.cc:290] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.857905 31969 leader_election.cc:304] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:36.858887 32654 raft_consensus.cc:2804] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:36.859475 32654 raft_consensus.cc:493] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.860040 32654 raft_consensus.cc:3060] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.869549 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:36.872675 32683 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.873420 32683 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.875746 32683 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.876355 31896 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:36.877491 32555 raft_consensus.cc:2749] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:36.878715 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
W20260709 13:16:36.879144 31967 leader_election.cc:336] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.889451 32642 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.890316 32642 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.892367 32642 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:16:36.889446 31967 leader_election.cc:336] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.894789 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:36.895888 31924 raft_consensus.cc:2468] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:36.896104 32748 raft_consensus.cc:493] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.896723 32748 raft_consensus.cc:515] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.897207 32654 raft_consensus.cc:515] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.899624 32654 leader_election.cc:290] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.899768 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:36.900585 32073 raft_consensus.cc:2468] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:36.902326 32042 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
W20260709 13:16:36.905025 32041 leader_election.cc:336] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.905225 31969 leader_election.cc:304] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:36.906309 32654 raft_consensus.cc:2804] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:36.906737 32654 raft_consensus.cc:493] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.908358 32654 raft_consensus.cc:3060] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.908572 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87842118d5214e359e7983d9ede2213b" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:36.909535 32073 raft_consensus.cc:3060] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.907307 32642 raft_consensus.cc:2804] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:36.913705 32642 raft_consensus.cc:493] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.914075 32642 raft_consensus.cc:3060] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.916498 32748 leader_election.cc:290] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.917178 32750 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.918156 32750 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.922992 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:36.923803 32002 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:36.923653 32654 raft_consensus.cc:515] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:36.925535 32044 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:36.894944 31967 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067
I20260709 13:16:36.926430 32647 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:36.954929 32647 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
W20260709 13:16:36.955304 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 65917.5us
I20260709 13:16:36.956581 32073 raft_consensus.cc:2468] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:36.927309 32654 leader_election.cc:290] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.957439 32654 raft_consensus.cc:2749] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:36.923743 32642 raft_consensus.cc:515] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.928798 32750 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:36.927037 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:36.958055 31969 leader_election.cc:304] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:36.958556 32074 raft_consensus.cc:3060] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.959024 32748 raft_consensus.cc:2804] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:36.961014 32642 leader_election.cc:290] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:36.962009 32642 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.955448 32647 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.962746 32642 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:36.963632 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3d708d55c46bcb69cfb239324f375" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:36.964457 31926 raft_consensus.cc:3060] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:36.934870 32041 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:36.974887 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:36.975744 31924 raft_consensus.cc:2468] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 1.
I20260709 13:16:36.977035 32642 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:36.978832 32042 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:36.982583 32750 raft_consensus.cc:2804] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:36.983011 32750 raft_consensus.cc:493] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:36.983429 32750 raft_consensus.cc:3060] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.989815 31926 raft_consensus.cc:2468] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:36.990648 32748 raft_consensus.cc:697] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:37.001315 32748 consensus_queue.cc:237] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:37.004410 31967 leader_election.cc:336] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:36.999801 32041 leader_election.cc:336] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:37.005566 31967 leader_election.cc:336] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:37.005877 32041 leader_election.cc:336] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:37.006458 31967 leader_election.cc:336] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.034274 32042 leader_election.cc:304] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.036244 32764 raft_consensus.cc:2804] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.036958 32764 raft_consensus.cc:697] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:37.009001 32074 raft_consensus.cc:2468] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:37.038882 31969 leader_election.cc:304] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.039778 32748 raft_consensus.cc:2804] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.037897 32764 consensus_queue.cc:237] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:37.040634 32748 raft_consensus.cc:697] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:37.054520 32748 consensus_queue.cc:237] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.041167 32647 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.047092 32750 raft_consensus.cc:515] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:37.060323 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:37.060645 32647 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.061022 31997 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.066808 32750 leader_election.cc:290] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:37.068269 31753 catalog_manager.cc:5697] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.069923 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809ff13e62914dd7854fa320a1173da0" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:37.072471 31926 raft_consensus.cc:3060] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.074256 31997 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:37.085184 32683 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.085865 32683 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:37.088562 32044 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:37.088610 32683 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:37.089488 32750 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.090080 32750 raft_consensus.cc:697] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:37.090973 32750 consensus_queue.cc:237] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:37.103735 31967 leader_election.cc:336] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.105106 32750 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.106635 32750 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.109521 32750 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.116246 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:37.117182 31924 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:37.115906 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:37.118942 32041 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.125319 31968 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.127151 32555 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.133543 32555 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:37.134768 32041 leader_election.cc:336] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.133036 32683 raft_consensus.cc:2804] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:37.148198 32683 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.148638 32683 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.138183 32555 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:37.140491 31753 catalog_manager.cc:5697] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.131567 31926 raft_consensus.cc:2468] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:37.142252 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:37.153707 31997 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 2.
I20260709 13:16:37.154126 32042 leader_election.cc:304] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.159857 31752 catalog_manager.cc:5697] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.165434 32750 raft_consensus.cc:2804] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.167516 32750 raft_consensus.cc:697] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
W20260709 13:16:37.147248 31893 leader_election.cc:336] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.175177 31896 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.175429 32683 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:37.176795 32629 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:37.177198 32629 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.177606 32629 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.178591 32683 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:37.179231 32683 raft_consensus.cc:493] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.179697 32683 raft_consensus.cc:515] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.181958 32683 leader_election.cc:290] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.182698 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:37.183394 32073 raft_consensus.cc:2468] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:37.168876 32750 consensus_queue.cc:237] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:16:37.184361 31967 leader_election.cc:336] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.184579 31969 leader_election.cc:304] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:37.184892   300 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.185539   300 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:37.189558 31967 leader_election.cc:336] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.190281 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:37.190886 31926 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:37.190994 32041 leader_election.cc:336] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.191470 32041 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:37.195998 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:37.196429 32764 raft_consensus.cc:2749] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:37.197573 32073 raft_consensus.cc:2468] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 1.
I20260709 13:16:37.197376   300 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.198752 31969 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:37.199584 31926 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:37.200641 31968 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.202495 32654 raft_consensus.cc:2804] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.203030 32654 raft_consensus.cc:697] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:37.203675 32647 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.204222 32654 consensus_queue.cc:237] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:37.205013   300 raft_consensus.cc:2804] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:37.204406 32647 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.205390   300 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.205847   300 raft_consensus.cc:3060] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.207603 32647 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.213184 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:37.213987 31997 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:37.215451 32629 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:37.216889 32044 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:37.221931 32642 raft_consensus.cc:2804] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:37.222369 32642 raft_consensus.cc:493] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.222781 32642 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.200338 32683 raft_consensus.cc:2804] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:37.223259 32683 raft_consensus.cc:493] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:16:37.223688 32683 raft_consensus.cc:3060] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:37.232493 31967 leader_election.cc:336] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:37.232987 32041 leader_election.cc:336] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:37.236882 32041 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:37.237062 32642 raft_consensus.cc:515] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.239092 32642 leader_election.cc:290] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.239977 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be826572f20c45fe84142ea4f204ba26" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:37.240774 31997 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.241211 31752 catalog_manager.cc:5697] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:37.244679 32002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:37.245357 32002 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:37.248682   300 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.247879 32629 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 election: Requested vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:37.251588 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:37.255693 32074 raft_consensus.cc:3060] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:37.256274 31893 leader_election.cc:336] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.255853 32002 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 3.
I20260709 13:16:37.259411 31896 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.260697 32555 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:37.261257 32555 raft_consensus.cc:697] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:37.263365 32555 consensus_queue.cc:237] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:37.267485 32074 raft_consensus.cc:2468] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
W20260709 13:16:37.271520 32041 leader_election.cc:336] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.273010 31969 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:37.273995   300 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.274242 32748 raft_consensus.cc:2804] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.278591 31997 raft_consensus.cc:2468] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 3.
I20260709 13:16:37.282174 32748 raft_consensus.cc:697] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:37.283457 32748 consensus_queue.cc:237] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.284996 32683 raft_consensus.cc:515] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.290808 31752 catalog_manager.cc:5697] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.292240 32044 leader_election.cc:304] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.293323 31752 catalog_manager.cc:5697] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.305177 32761 raft_consensus.cc:2804] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:37.306151 32761 raft_consensus.cc:697] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 3 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:37.307080 32761 consensus_queue.cc:237] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.332862 31753 catalog_manager.cc:5697] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 2 to 3, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 3 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.355831 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:37.356056 32683 leader_election.cc:290] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:37.356570 32074 raft_consensus.cc:3060] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:37.361573 31967 leader_election.cc:336] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:16:37.365525 31967 leader_election.cc:336] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:16:37.368839 31752 catalog_manager.cc:5697] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 2 to 3, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 3 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.374728 32074 raft_consensus.cc:2468] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
I20260709 13:16:37.376150 31969 leader_election.cc:304] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:37.377369 32748 raft_consensus.cc:2804] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:37.378105 32748 raft_consensus.cc:697] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:37.379024 32748 consensus_queue.cc:237] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:37.387567 31753 catalog_manager.cc:5697] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.407891 31926 raft_consensus.cc:1275] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.409840 32642 consensus_queue.cc:1048] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:37.424782 31967 consensus_peers.cc:597] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:37.417351 31753 catalog_manager.cc:5697] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 1 to 2, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:37.445886 31926 raft_consensus.cc:1275] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.461802 32761 consensus_queue.cc:1048] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:37.513411 32041 consensus_peers.cc:597] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:37.528108 32073 raft_consensus.cc:1275] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.531215 32748 consensus_queue.cc:1048] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:37.588241 32073 raft_consensus.cc:1275] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.589874 32748 consensus_queue.cc:1048] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:37.620824 31997 raft_consensus.cc:1275] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.622368 32764 consensus_queue.cc:1048] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:37.631248 32041 consensus_peers.cc:597] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:37.645210 31967 consensus_peers.cc:597] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:37.658289 31930 raft_consensus.cc:1275] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.660373 32761 consensus_queue.cc:1048] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:37.689354 31967 consensus_peers.cc:597] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:37.696491 32041 consensus_peers.cc:597] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:37.733688 32041 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.738909 31967 consensus_peers.cc:597] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.754428 31893 consensus_peers.cc:597] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.772197 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.775561 32041 consensus_peers.cc:597] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:37.789165 31997 raft_consensus.cc:1275] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:37.792152 31930 raft_consensus.cc:1275] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.792618 32761 consensus_queue.cc:1048] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:37.795261 32762 consensus_queue.cc:1048] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:37.817080 32041 consensus_peers.cc:597] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:37.820878 32002 raft_consensus.cc:1275] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Refusing update from remote peer a588481f03f6474d9009d5bd5e08e887: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:16:37.827450 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:37.830672 32629 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:37.846318 21636 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 31876)
W20260709 13:16:37.853940 31893 consensus_peers.cc:597] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:37.883131 32073 raft_consensus.cc:1275] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:37.887396 32762 consensus_queue.cc:1048] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:16:37.902261 31893 consensus_peers.cc:597] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.903081 31893 consensus_peers.cc:597] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.924641 31967 consensus_peers.cc:597] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:37.936406 31893 consensus_peers.cc:597] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:37.937201 31893 consensus_peers.cc:597] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:37.971957 32073 raft_consensus.cc:1275] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer ccf4710690c7473d814836da1babee51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:37.975090 31967 consensus_peers.cc:597] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:37.976332   300 consensus_queue.cc:1048] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:37.989315 31967 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.006115 32041 consensus_peers.cc:597] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.006331 31893 consensus_peers.cc:597] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.019625 31893 consensus_peers.cc:597] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.024466 31967 consensus_peers.cc:597] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:38.029009 31893 consensus_peers.cc:597] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.040606 31967 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.060699 31893 consensus_peers.cc:597] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.120978 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.121599 31893 consensus_peers.cc:597] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.183899 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.206317 31967 consensus_peers.cc:597] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.214617 31893 consensus_peers.cc:597] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.257337 31893 consensus_peers.cc:597] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.282862 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.298858 31893 consensus_peers.cc:597] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.323442 31893 consensus_peers.cc:597] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.426824 31893 consensus_peers.cc:597] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.444730 32041 consensus_peers.cc:597] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.494946 31967 consensus_peers.cc:597] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.529558 32041 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.542933 31893 consensus_peers.cc:597] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.604377 31893 consensus_peers.cc:597] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:38.847021 21636 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 31876)
W20260709 13:16:39.594630 31967 consensus_peers.cc:597] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:39.658828 32041 consensus_peers.cc:597] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:39.678090 32041 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:39.814307 32041 consensus_peers.cc:597] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:39.847677 21636 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 31876)
W20260709 13:16:39.938297 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.025803 32041 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.068795 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.086122 31893 consensus_peers.cc:597] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.140183 32041 consensus_peers.cc:597] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:40.259011 31893 consensus_peers.cc:597] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.265666 32041 consensus_peers.cc:597] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:40.298393 31893 consensus_peers.cc:597] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.360780 31893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111) [suppressed 425 similar messages]
W20260709 13:16:40.363864 31893 consensus_peers.cc:597] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.398101 31967 consensus_peers.cc:597] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:40.399761 31893 consensus_peers.cc:597] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.461563 32041 consensus_peers.cc:597] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.541568 32041 consensus_peers.cc:597] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.572829 31893 consensus_peers.cc:597] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.584406 31967 consensus_peers.cc:597] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:40.599478 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.599738 31893 consensus_peers.cc:597] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.607127 31893 consensus_peers.cc:597] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.609808 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.620257 31893 consensus_peers.cc:597] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.644029 31893 consensus_peers.cc:597] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.678735 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.694185 32041 consensus_peers.cc:597] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.708364 31893 consensus_peers.cc:597] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:40.725234 31893 consensus_peers.cc:597] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.734232 31967 consensus_peers.cc:597] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:40.759958 31893 consensus_peers.cc:597] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.784010 31893 consensus_peers.cc:597] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.786582 31893 consensus_peers.cc:597] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.848275 21636 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 31876)
W20260709 13:16:40.880896 31893 consensus_peers.cc:597] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.904735 31967 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.975667 31967 consensus_peers.cc:597] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:40.981335 31967 consensus_peers.cc:597] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.019505 31893 consensus_peers.cc:597] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.029544 32041 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.077940 31893 consensus_peers.cc:597] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.091073 31967 consensus_peers.cc:597] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:41.092173 31967 consensus_peers.cc:597] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:41.175381 31967 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.354676 31967 consensus_peers.cc:597] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.524174 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 93550.9us
W20260709 13:16:41.537477 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 106763us
W20260709 13:16:41.546010 31967 consensus_peers.cc:597] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.821269 31967 consensus_peers.cc:597] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:41.848886 21636 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 31876)
W20260709 13:16:42.102649 32041 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:42.184386 32041 consensus_peers.cc:597] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:42.410422 32041 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.478629 32041 consensus_peers.cc:597] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:42.543702 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.587029 31893 consensus_peers.cc:597] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.613819 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.649832 31893 consensus_peers.cc:597] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.654820 31893 consensus_peers.cc:597] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.714548 32041 consensus_peers.cc:597] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:42.810494 31967 consensus_peers.cc:597] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:42.820990 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
W20260709 13:16:42.839238 32041 consensus_peers.cc:597] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:42.870896 32041 consensus_peers.cc:597] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:42.965987 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.045584 32041 consensus_peers.cc:597] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.102216 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:43.118197   340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:43.124590   341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:43.132742   344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:43.134225 21636 server_base.cc:1061] running on GCE node
I20260709 13:16:43.141166 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:43.141597 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:43.141924 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603003141899 us; error 0 us; skew 500 ppm
I20260709 13:16:43.142807 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:43.147853 21636 webserver.cc:533] Webserver started at http://127.21.33.1:33119/ using document root <none> and password file <none>
I20260709 13:16:43.148468 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:43.148703 21636 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:16:43.166828 31893 consensus_peers.cc:597] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.167555 31893 consensus_peers.cc:597] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.198411 31893 consensus_peers.cc:597] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.200112 31893 consensus_peers.cc:597] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.201534 31893 consensus_peers.cc:597] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.207577 21636 fs_manager.cc:714] Time spent opening directory manager: real 0.058s	user 0.016s	sys 0.025s
W20260709 13:16:43.229338 31893 consensus_peers.cc:597] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.233916 31893 consensus_peers.cc:597] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.236613 31893 consensus_peers.cc:597] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.236956 31967 consensus_peers.cc:597] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:43.250528 31893 consensus_peers.cc:597] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.257997   350 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:43.259529 21636 fs_manager.cc:730] Time spent opening block manager: real 0.035s	user 0.006s	sys 0.005s
W20260709 13:16:43.263391 32041 consensus_peers.cc:597] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.263408 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "103d6e4c312a47ff9b258c14ac2871e5"
format_stamp: "Formatted at 2026-07-09 13:16:17 on dist-test-slave-p8rg"
I20260709 13:16:43.264308 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:16:43.273532 31893 consensus_peers.cc:597] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.277058 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.288173 31893 consensus_peers.cc:597] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.349123 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:43.353313 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260709 13:16:43.355127 31967 consensus_peers.cc:597] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:43.356086 31967 consensus_peers.cc:597] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.365643 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260709 13:16:43.376330 31893 consensus_peers.cc:597] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.377130 31893 consensus_peers.cc:597] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.379020 31893 consensus_peers.cc:597] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.383010 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.387070 32041 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.387844   358 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:16:43.414273 31967 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.562337 31967 consensus_peers.cc:597] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:43.613601 31893 consensus_peers.cc:597] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:43.682744 31967 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.687845 31967 consensus_peers.cc:597] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.718627 31967 consensus_peers.cc:597] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:43.750720 31967 consensus_peers.cc:597] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:43.802313 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:16:43.802814 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.421s	user 0.024s	sys 0.001s
I20260709 13:16:43.803303 21636 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:16:43.824354   358 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:43.851303   358 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:43.852490   358 tablet_bootstrap.cc:492] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:43.853256   358 ts_tablet_manager.cc:1403] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.021s
I20260709 13:16:43.858899   358 raft_consensus.cc:359] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:43.860204   358 raft_consensus.cc:740] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.861292   358 consensus_queue.cc:260] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:43.864073   358 ts_tablet_manager.cc:1434] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:16:43.866266   358 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:43.886045   358 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:43.887302   358 tablet_bootstrap.cc:492] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:43.888018   358 ts_tablet_manager.cc:1403] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:16:43.895298   358 raft_consensus.cc:359] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:43.896198   358 raft_consensus.cc:740] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.896910   358 consensus_queue.cc:260] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:43.898856   358 ts_tablet_manager.cc:1434] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:16:43.899624   358 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:43.928316   358 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:43.930009   358 tablet_bootstrap.cc:492] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
W20260709 13:16:43.929993 31967 consensus_peers.cc:597] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:43.931248   358 ts_tablet_manager.cc:1403] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.008s
I20260709 13:16:43.933614   358 raft_consensus.cc:359] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:43.934158   358 raft_consensus.cc:740] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.934728   358 consensus_queue.cc:260] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:43.936314   358 ts_tablet_manager.cc:1434] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:16:43.937145   358 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:43.951272   358 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:43.952385   358 tablet_bootstrap.cc:492] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:43.953105   358 ts_tablet_manager.cc:1403] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:16:43.955921   358 raft_consensus.cc:359] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:43.956725   358 raft_consensus.cc:740] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.957455   358 consensus_queue.cc:260] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:43.959581   358 ts_tablet_manager.cc:1434] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:16:43.960284   358 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:43.975943   358 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:43.976825   358 tablet_bootstrap.cc:492] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:43.977428   358 ts_tablet_manager.cc:1403] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 13:16:43.979949   358 raft_consensus.cc:359] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:43.980566   358 raft_consensus.cc:740] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:43.981244   358 consensus_queue.cc:260] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:43.983263   358 ts_tablet_manager.cc:1434] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:16:43.984319   358 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.013398   358 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.014417   358 tablet_bootstrap.cc:492] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.015029   358 ts_tablet_manager.cc:1403] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.007s
I20260709 13:16:44.017436   358 raft_consensus.cc:359] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.018018   358 raft_consensus.cc:740] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.018719   358 consensus_queue.cc:260] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.020416   358 ts_tablet_manager.cc:1434] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:16:44.021073   358 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.036972   358 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.037956   358 tablet_bootstrap.cc:492] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.038630   358 ts_tablet_manager.cc:1403] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 13:16:44.041369   358 raft_consensus.cc:359] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.042122   358 raft_consensus.cc:740] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.043020   358 consensus_queue.cc:260] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.044978   358 ts_tablet_manager.cc:1434] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:44.045823   358 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:44.057022 31951 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc760 after lost signal to thread 21639
W20260709 13:16:44.058975 31951 debug-util.cc:398] Leaking SignalData structure 0x7b08002e9600 after lost signal to thread 31787
W20260709 13:16:44.062880 31951 debug-util.cc:398] Leaking SignalData structure 0x7b0800110400 after lost signal to thread 31954
I20260709 13:16:44.065335   358 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.067868   358 tablet_bootstrap.cc:492] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
W20260709 13:16:44.068082 31951 debug-util.cc:398] Leaking SignalData structure 0x7b0800239ee0 after lost signal to thread 32028
W20260709 13:16:44.070127 31951 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6cc0 after lost signal to thread 32102
I20260709 13:16:44.070781   358 ts_tablet_manager.cc:1403] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.004s
I20260709 13:16:44.091934   358 raft_consensus.cc:359] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.095525   358 raft_consensus.cc:740] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.102905   358 consensus_queue.cc:260] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.093312 21636 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:16:44.104107 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.301s	user 0.263s	sys 0.012s
I20260709 13:16:44.109207   358 ts_tablet_manager.cc:1434] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.001s
I20260709 13:16:44.110718   358 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.138725   358 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.139847   358 tablet_bootstrap.cc:492] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.140585   358 ts_tablet_manager.cc:1403] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.011s
I20260709 13:16:44.143412   358 raft_consensus.cc:359] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.144152   358 raft_consensus.cc:740] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.144917   358 consensus_queue.cc:260] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.150624   358 ts_tablet_manager.cc:1434] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:16:44.151548   358 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.192076   358 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.193632   358 tablet_bootstrap.cc:492] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.194770   358 ts_tablet_manager.cc:1403] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.000s
I20260709 13:16:44.201126   358 raft_consensus.cc:359] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.202469   358 raft_consensus.cc:740] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.206933   358 consensus_queue.cc:260] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:44.231868 31967 consensus_peers.cc:597] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:44.236819   358 ts_tablet_manager.cc:1434] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:16:44.238528   358 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.271320   358 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.272739   358 tablet_bootstrap.cc:492] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.273821   358 ts_tablet_manager.cc:1403] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.007s
I20260709 13:16:44.278791   358 raft_consensus.cc:359] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.279663   358 raft_consensus.cc:740] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.280627   358 consensus_queue.cc:260] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.283347   358 ts_tablet_manager.cc:1434] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:16:44.284323   358 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:44.305491 32041 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:44.354682   358 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.363137   358 tablet_bootstrap.cc:492] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.363917   358 ts_tablet_manager.cc:1403] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.011s
I20260709 13:16:44.375556   358 raft_consensus.cc:359] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.376420   358 raft_consensus.cc:740] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.377141   358 consensus_queue.cc:260] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.383734   358 ts_tablet_manager.cc:1434] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.004s
I20260709 13:16:44.386688   358 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.428169   358 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.429279   358 tablet_bootstrap.cc:492] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.431231   358 ts_tablet_manager.cc:1403] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.016s
I20260709 13:16:44.437667   358 raft_consensus.cc:359] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.438608   358 raft_consensus.cc:740] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.439677   358 consensus_queue.cc:260] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.442544   358 ts_tablet_manager.cc:1434] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:16:44.443600   358 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.487598   358 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.491385   358 tablet_bootstrap.cc:492] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.492317   358 ts_tablet_manager.cc:1403] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.049s	user 0.025s	sys 0.004s
I20260709 13:16:44.495898   358 raft_consensus.cc:359] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.496852   358 raft_consensus.cc:740] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.497968   358 consensus_queue.cc:260] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.500766   358 ts_tablet_manager.cc:1434] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:16:44.501875   358 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.526238   358 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.527428   358 tablet_bootstrap.cc:492] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.528275   358 ts_tablet_manager.cc:1403] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.000s
I20260709 13:16:44.532886   358 raft_consensus.cc:359] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.533898   358 raft_consensus.cc:740] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.534802   358 consensus_queue.cc:260] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.537581   358 ts_tablet_manager.cc:1434] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:16:44.539489   358 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.589443   358 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.590911   358 tablet_bootstrap.cc:492] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.591758   358 ts_tablet_manager.cc:1403] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.053s	user 0.021s	sys 0.023s
I20260709 13:16:44.594878   358 raft_consensus.cc:359] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.595919   358 raft_consensus.cc:740] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.597123   358 consensus_queue.cc:260] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:44.602737   358 ts_tablet_manager.cc:1434] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:16:44.604101   358 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.660429   358 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.663542   358 tablet_bootstrap.cc:492] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.664695   358 ts_tablet_manager.cc:1403] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.014s
I20260709 13:16:44.678157   358 raft_consensus.cc:359] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.679124   358 raft_consensus.cc:740] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.680192   358 consensus_queue.cc:260] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.683560   358 ts_tablet_manager.cc:1434] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:16:44.684850   358 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.722684   358 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.724392   358 tablet_bootstrap.cc:492] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.725529   358 ts_tablet_manager.cc:1403] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.011s
I20260709 13:16:44.729780   358 raft_consensus.cc:359] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.730962   358 raft_consensus.cc:740] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.732177   358 consensus_queue.cc:260] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.735563   358 ts_tablet_manager.cc:1434] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:16:44.736609   358 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:44.755687 32041 consensus_peers.cc:597] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:44.781932   358 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.783393   358 tablet_bootstrap.cc:492] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.784194   358 ts_tablet_manager.cc:1403] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.008s
I20260709 13:16:44.787436   358 raft_consensus.cc:359] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.788738   358 raft_consensus.cc:740] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.789592   358 consensus_queue.cc:260] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:44.791208   439 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:35847 every 8 connection(s)
I20260709 13:16:44.792357 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:35847
I20260709 13:16:44.792956   358 ts_tablet_manager.cc:1434] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:16:44.794257   358 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:44.916194   406 raft_consensus.cc:3060] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.917202   358 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:44.918469   358 tablet_bootstrap.cc:492] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:44.919413   358 ts_tablet_manager.cc:1403] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.125s	user 0.026s	sys 0.015s
I20260709 13:16:44.932176   358 raft_consensus.cc:359] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.933117   358 raft_consensus.cc:740] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:44.934170   358 consensus_queue.cc:260] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:44.937073   358 ts_tablet_manager.cc:1434] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:16:44.938159   358 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:45.075809 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 155327us
I20260709 13:16:45.148110   405 raft_consensus.cc:3060] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.150663   362 raft_consensus.cc:493] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.151360   362 raft_consensus.cc:515] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.163225   362 leader_election.cc:290] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:45.192592 31895 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 50347.8us
I20260709 13:16:45.206404   362 raft_consensus.cc:493] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.207298   456 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.207229   362 raft_consensus.cc:515] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:45.207924   456 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:45.214668   362 leader_election.cc:290] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:45.227283   358 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:45.248565   358 tablet_bootstrap.cc:492] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.249532   358 ts_tablet_manager.cc:1403] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.312s	user 0.025s	sys 0.003s
I20260709 13:16:45.246932   456 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:45.269274   358 raft_consensus.cc:359] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.292819   358 raft_consensus.cc:740] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
W20260709 13:16:45.300925 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 83717.7us
I20260709 13:16:45.299849   358 consensus_queue.cc:260] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.330734   456 raft_consensus.cc:493] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.331543   456 raft_consensus.cc:515] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:16:45.317109 31967 consensus_peers.cc:597] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:45.345386 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:45.350541 32041 consensus_peers.cc:597] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:45.355166   456 leader_election.cc:290] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:16:45.363188 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:45.364027 32041 consensus_peers.cc:597] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:45.372967   358 ts_tablet_manager.cc:1434] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.123s	user 0.009s	sys 0.000s
I20260709 13:16:45.374194   358 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:45.401113 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:45.421252 32073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:45.452373   402 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.454403   405 raft_consensus.cc:3060] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.529081 31925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:45.529670 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:45.531783 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:45.529165 31922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:45.536063   352 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:45.537701   455 raft_consensus.cc:2749] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:45.550575 31893 consensus_peers.cc:597] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:45.554946 31967 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.560915   441 heartbeater.cc:344] Connected to a master server at 127.21.33.62:43497
I20260709 13:16:45.561424   441 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:45.562642   441 heartbeater.cc:507] Master 127.21.33.62:43497 requested a full tablet report, sending...
I20260709 13:16:45.577818 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:45.594938   354 leader_election.cc:304] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:45.596634   362 raft_consensus.cc:2749] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:45.597566 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125eaf8f67bd4727b6d49528c95ca751" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:45.601150 31893 consensus_peers.cc:597] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.608989   358 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:16:45.610569   352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 74740.4us
I20260709 13:16:45.611734   352 leader_election.cc:304] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:45.613561   358 tablet_bootstrap.cc:492] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.607817   456 raft_consensus.cc:493] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:45.614548   358 ts_tablet_manager.cc:1403] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.241s	user 0.038s	sys 0.015s
I20260709 13:16:45.614207   456 raft_consensus.cc:515] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:45.619225   362 raft_consensus.cc:2749] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:45.620223   354 leader_election.cc:304] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:45.626273   362 raft_consensus.cc:2749] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:45.621223   358 raft_consensus.cc:359] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.632104   358 raft_consensus.cc:740] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
W20260709 13:16:45.638280 31967 consensus_peers.cc:597] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:45.639338 31893 consensus_peers.cc:597] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:45.641063 31893 consensus_peers.cc:597] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.632845   358 consensus_queue.cc:260] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.647810   456 leader_election.cc:290] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
W20260709 13:16:45.641659 31893 consensus_peers.cc:597] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.648651 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:45.651925 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "604a6aa17b764d61b1403b11157ddbe7" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:45.658375   358 ts_tablet_manager.cc:1434] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:16:45.659449   358 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:45.659190   354 leader_election.cc:304] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:45.661572   362 raft_consensus.cc:2749] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:45.711526 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.723883   358 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:45.725063   358 tablet_bootstrap.cc:492] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.725788   358 ts_tablet_manager.cc:1403] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.008s
I20260709 13:16:45.728706   358 raft_consensus.cc:359] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.729468   358 raft_consensus.cc:740] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:45.752444   358 consensus_queue.cc:260] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.754446   358 ts_tablet_manager.cc:1434] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.028s	user 0.002s	sys 0.004s
I20260709 13:16:45.756285   358 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:45.781397 31893 consensus_peers.cc:597] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.807520   358 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:45.808871   358 tablet_bootstrap.cc:492] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.810182   358 ts_tablet_manager.cc:1403] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.005s
I20260709 13:16:45.814383   358 raft_consensus.cc:359] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:45.815266   358 raft_consensus.cc:740] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:45.816375   358 consensus_queue.cc:260] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:45.826157 31753 ts_manager.cc:194] Re-registered known tserver with Master: 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:45.827512   358 ts_tablet_manager.cc:1434] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:16:45.828544   358 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:45.875198   358 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:45.876341   358 tablet_bootstrap.cc:492] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.877054   358 ts_tablet_manager.cc:1403] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.049s	user 0.025s	sys 0.000s
I20260709 13:16:45.880458   358 raft_consensus.cc:359] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.881253   358 raft_consensus.cc:740] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:45.882197   358 consensus_queue.cc:260] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:45.885005   358 ts_tablet_manager.cc:1434] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:16:45.886037   358 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:45.913383   358 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:45.914496   358 tablet_bootstrap.cc:492] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.915794   358 ts_tablet_manager.cc:1403] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.007s
I20260709 13:16:45.918699   358 raft_consensus.cc:359] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:45.919581   358 raft_consensus.cc:740] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:45.920476   358 consensus_queue.cc:260] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:45.923384   358 ts_tablet_manager.cc:1434] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:16:45.924311   358 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:45.929921   405 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:45.942258 31893 consensus_peers.cc:597] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:45.971037   358 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:45.973238   358 tablet_bootstrap.cc:492] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:45.987679   358 ts_tablet_manager.cc:1403] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.011s
I20260709 13:16:45.992849   358 raft_consensus.cc:359] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:45.993691   358 raft_consensus.cc:740] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:45.994763   358 consensus_queue.cc:260] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.002349   358 ts_tablet_manager.cc:1434] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:16:46.003168   358 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.005120 31753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52416
I20260709 13:16:46.011574   441 heartbeater.cc:499] Master 127.21.33.62:43497 was elected leader, sending a full tablet report...
W20260709 13:16:46.078903 31893 consensus_peers.cc:597] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:46.097538   358 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.098996   358 tablet_bootstrap.cc:492] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.100675   358 ts_tablet_manager.cc:1403] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.098s	user 0.029s	sys 0.034s
I20260709 13:16:46.103821   358 raft_consensus.cc:359] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:46.104705   358 raft_consensus.cc:740] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.105952   358 consensus_queue.cc:260] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:46.109472   358 ts_tablet_manager.cc:1434] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:46.115156   358 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.160111   405 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:46.161175   358 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:16:46.165593   431 debug-util.cc:398] Leaking SignalData structure 0x7b080027d380 after lost signal to thread 21639
W20260709 13:16:46.167083   431 debug-util.cc:398] Leaking SignalData structure 0x7b0800335b40 after lost signal to thread 31787
I20260709 13:16:46.168699   358 tablet_bootstrap.cc:492] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.169721   358 ts_tablet_manager.cc:1403] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.057s	user 0.022s	sys 0.010s
I20260709 13:16:46.174494   358 raft_consensus.cc:359] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.175424   358 raft_consensus.cc:740] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
W20260709 13:16:46.168820   431 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8dc0 after lost signal to thread 31954
W20260709 13:16:46.177205   431 debug-util.cc:398] Leaking SignalData structure 0x7b080003d240 after lost signal to thread 32028
W20260709 13:16:46.178992   431 debug-util.cc:398] Leaking SignalData structure 0x7b0800351340 after lost signal to thread 32102
W20260709 13:16:46.181332   431 debug-util.cc:398] Leaking SignalData structure 0x7b08002320c0 after lost signal to thread 439
W20260709 13:16:46.187352 32041 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:46.188084   358 consensus_queue.cc:260] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.199159   358 ts_tablet_manager.cc:1434] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:16:46.201670   358 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.278890   358 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.279994   358 tablet_bootstrap.cc:492] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.280629   358 ts_tablet_manager.cc:1403] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.079s	user 0.023s	sys 0.017s
I20260709 13:16:46.283012   358 raft_consensus.cc:359] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.283636   358 raft_consensus.cc:740] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.284265   358 consensus_queue.cc:260] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.286573   358 ts_tablet_manager.cc:1434] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.006s
I20260709 13:16:46.287483   358 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.319921   358 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.320979   358 tablet_bootstrap.cc:492] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.321689   358 ts_tablet_manager.cc:1403] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.034s	user 0.023s	sys 0.008s
I20260709 13:16:46.325426   358 raft_consensus.cc:359] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.326277   358 raft_consensus.cc:740] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.327100   358 consensus_queue.cc:260] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.329478   358 ts_tablet_manager.cc:1434] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:16:46.330369   358 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.364559   358 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.365924   358 tablet_bootstrap.cc:492] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.366760   358 ts_tablet_manager.cc:1403] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.005s
I20260709 13:16:46.369941   358 raft_consensus.cc:359] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.371315   358 raft_consensus.cc:740] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.372318   358 consensus_queue.cc:260] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.374198   358 ts_tablet_manager.cc:1434] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:16:46.375058   358 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
W20260709 13:16:46.381754 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:46.429123   358 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.430428   358 tablet_bootstrap.cc:492] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.431145   358 ts_tablet_manager.cc:1403] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.009s
I20260709 13:16:46.434295   358 raft_consensus.cc:359] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.434989   358 raft_consensus.cc:740] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.435803   358 consensus_queue.cc:260] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.438055   358 ts_tablet_manager.cc:1434] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:46.438865   358 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.467442   358 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.468508   358 tablet_bootstrap.cc:492] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.469453   358 ts_tablet_manager.cc:1403] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.009s
I20260709 13:16:46.472242   358 raft_consensus.cc:359] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.472975   358 raft_consensus.cc:740] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.473718   358 consensus_queue.cc:260] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.475963   358 ts_tablet_manager.cc:1434] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:46.476814   358 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.515698   358 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.516819   358 tablet_bootstrap.cc:492] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.517629   358 ts_tablet_manager.cc:1403] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.008s
I20260709 13:16:46.520947   358 raft_consensus.cc:359] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.521848   358 raft_consensus.cc:740] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.523013   358 consensus_queue.cc:260] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.525652   358 ts_tablet_manager.cc:1434] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:16:46.538690   358 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.576444   358 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.675531   358 tablet_bootstrap.cc:492] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.676465   358 ts_tablet_manager.cc:1403] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.008s
I20260709 13:16:46.679504   358 raft_consensus.cc:359] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.680218   358 raft_consensus.cc:740] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.707712   358 consensus_queue.cc:260] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:46.710341   358 ts_tablet_manager.cc:1434] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:16:46.715869   358 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.761233   358 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:16:46.763924 31893 consensus_peers.cc:597] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:46.768332 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 110441us
I20260709 13:16:46.771883   358 tablet_bootstrap.cc:492] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.773079   358 ts_tablet_manager.cc:1403] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.003s
I20260709 13:16:46.784622   358 raft_consensus.cc:359] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.786751   358 raft_consensus.cc:740] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.787665   358 consensus_queue.cc:260] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:46.791270   358 ts_tablet_manager.cc:1434] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:16:46.800740   358 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.842598   399 raft_consensus.cc:3060] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.872125   405 raft_consensus.cc:3060] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.873404   358 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.874605   358 tablet_bootstrap.cc:492] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.875433   358 ts_tablet_manager.cc:1403] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.011s
I20260709 13:16:46.883011   358 raft_consensus.cc:359] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:46.883841   358 raft_consensus.cc:740] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.885016   358 consensus_queue.cc:260] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:46.887370   358 ts_tablet_manager.cc:1434] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.008s
I20260709 13:16:46.888391   358 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:46.926635   358 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:46.927835   358 tablet_bootstrap.cc:492] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:46.928681   358 ts_tablet_manager.cc:1403] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.008s
I20260709 13:16:46.933000   358 raft_consensus.cc:359] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:46.933738   358 raft_consensus.cc:740] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:46.934561   358 consensus_queue.cc:260] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:46.936875   358 ts_tablet_manager.cc:1434] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:16:46.937835   358 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:47.029999   358 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:47.031376   358 tablet_bootstrap.cc:492] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:47.032161   358 ts_tablet_manager.cc:1403] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.095s	user 0.029s	sys 0.018s
I20260709 13:16:47.034993   358 raft_consensus.cc:359] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:47.035691   358 raft_consensus.cc:740] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.036504   358 consensus_queue.cc:260] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:47.044028   358 ts_tablet_manager.cc:1434] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:16:47.044945   358 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:47.073848   358 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:47.074971   358 tablet_bootstrap.cc:492] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:47.075709   358 ts_tablet_manager.cc:1403] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.005s
I20260709 13:16:47.078547   358 raft_consensus.cc:359] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:47.079254   358 raft_consensus.cc:740] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.079999   358 consensus_queue.cc:260] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:47.082558   358 ts_tablet_manager.cc:1434] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:47.087611   358 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:47.142422   358 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:47.143441   358 tablet_bootstrap.cc:492] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:47.144161   358 ts_tablet_manager.cc:1403] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.002s
I20260709 13:16:47.147028   358 raft_consensus.cc:359] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:47.147738   358 raft_consensus.cc:740] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.148643   358 consensus_queue.cc:260] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:47.156096   358 ts_tablet_manager.cc:1434] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:16:47.179522   358 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:47.224406   358 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:47.225580   358 tablet_bootstrap.cc:492] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:47.226401   358 ts_tablet_manager.cc:1403] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.008s
I20260709 13:16:47.229300   358 raft_consensus.cc:359] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:47.230087   358 raft_consensus.cc:740] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.236129   358 consensus_queue.cc:260] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:47.247120   358 ts_tablet_manager.cc:1434] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:16:47.248167   358 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:47.296926   358 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:47.322567   358 tablet_bootstrap.cc:492] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:47.323483   358 ts_tablet_manager.cc:1403] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.076s	user 0.026s	sys 0.008s
W20260709 13:16:47.333657   354 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.33.62:43497, user_credentials={real_user=slave}} blocked reactor thread for 59020.7us
I20260709 13:16:47.336912   358 raft_consensus.cc:359] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:47.337880   358 raft_consensus.cc:740] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.338646   358 consensus_queue.cc:260] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:47.340474   358 ts_tablet_manager.cc:1434] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.001s
W20260709 13:16:47.354404 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 62404.1us
I20260709 13:16:47.341396   358 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap starting.
I20260709 13:16:47.420048   358 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:47.421152   358 tablet_bootstrap.cc:492] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Bootstrap complete.
I20260709 13:16:47.421950   358 ts_tablet_manager.cc:1403] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent bootstrapping tablet: real 0.081s	user 0.027s	sys 0.006s
I20260709 13:16:47.425117   358 raft_consensus.cc:359] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:47.425927   358 raft_consensus.cc:740] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Initialized, Role: FOLLOWER
I20260709 13:16:47.426694   358 consensus_queue.cc:260] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:47.444835   358 ts_tablet_manager.cc:1434] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:16:47.496017   402 raft_consensus.cc:3060] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.501335   399 raft_consensus.cc:3060] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.944222   399 raft_consensus.cc:3060] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:48.008455 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 54713.9us
W20260709 13:16:49.013031 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 104325us
W20260709 13:16:50.273936 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 66469.8us
W20260709 13:16:50.424022 31896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 233618us
W20260709 13:16:53.611570 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 53432.2us
W20260709 13:16:53.656438 31969 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 66429us
I20260709 13:16:56.242354   557 raft_consensus.cc:493] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:56.243714   557 raft_consensus.cc:515] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.252045 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247a65da5884caba020ed13be64352f" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.270745   557 leader_election.cc:290] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:56.301244   559 raft_consensus.cc:493] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:56.302114   559 raft_consensus.cc:515] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:56.306138   401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247a65da5884caba020ed13be64352f" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:56.332234 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:56.332901   559 leader_election.cc:290] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.363703   527 raft_consensus.cc:493] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
W20260709 13:16:56.365247 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 45751.3us
I20260709 13:16:56.367355   527 raft_consensus.cc:515] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:56.369894   527 leader_election.cc:290] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.380357   563 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:56.381179   563 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.386068   563 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.386904   563 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 32690b3987094d51bb4f214321a29067)
I20260709 13:16:56.387588 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.387730   563 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:16:56.393751 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 107776us
I20260709 13:16:56.396982   558 raft_consensus.cc:493] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:56.397954   558 raft_consensus.cc:515] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:56.400616 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.400923   558 leader_election.cc:290] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:16:56.410547 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 83280us
I20260709 13:16:56.412460 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.414711   527 raft_consensus.cc:493] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:56.415076   565 raft_consensus.cc:493] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:56.415474   527 raft_consensus.cc:515] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:56.415936   565 raft_consensus.cc:515] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:56.417637 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "039ecd5ff518405fb0fc382c46e7e7c4" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.418160   527 leader_election.cc:290] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:56.418407   565 leader_election.cc:290] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:56.419656   547 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:56.419898   565 raft_consensus.cc:493] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:16:56.420258   547 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:56.420701 32075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:56.424496   547 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:56.431152 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.406425   563 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.446076   563 raft_consensus.cc:493] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 32690b3987094d51bb4f214321a29067)
I20260709 13:16:56.420460   565 raft_consensus.cc:515] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:56.438839 31925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c11d6a5d0564e90b9ceda978d9225cb" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.441041 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e16bbde8ad04886af986347646976b7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.465921   565 leader_election.cc:290] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.447821 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.435336 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "039ecd5ff518405fb0fc382c46e7e7c4" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.449806 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.470383   401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7811f2bbfe41aea0eba9a542d7dee8" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
W20260709 13:16:56.457073 31893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 46869.2us
I20260709 13:16:56.458863   558 raft_consensus.cc:493] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:56.459509   566 raft_consensus.cc:493] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ccf4710690c7473d814836da1babee51)
I20260709 13:16:56.472177   566 raft_consensus.cc:515] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:56.472535   401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441576026a7f44f990e9aa61c86435d6" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:56.473304   563 raft_consensus.cc:515] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:56.474673   401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e16bbde8ad04886af986347646976b7" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:56.476672   405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34769db9ad7a46a8b075b865d2a2b6ea" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:56.480531   353 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.472185   558 raft_consensus.cc:515] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:56.502257   567 raft_consensus.cc:2749] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:56.504801   558 leader_election.cc:290] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.483685   566 leader_election.cc:290] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:16:56.496753 31968 leader_election.cc:304] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:56.506093   401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5" is_pre_election: true
I20260709 13:16:56.510362   564 raft_consensus.cc:2749] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.514035 32077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:16:56.525277   563 leader_election.cc:290] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:56.530694 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2ce179ed2a94df48e1b5f00fec58804" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.530848 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.533015 32042 leader_election.cc:304] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
W20260709 13:16:56.570271 32041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 51930.4us
I20260709 13:16:56.536207 32042 leader_election.cc:304] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.581142   352 leader_election.cc:304] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:56.574329 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.576543 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a588481f03f6474d9009d5bd5e08e887" is_pre_election: true
I20260709 13:16:56.537285   558 raft_consensus.cc:2749] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.583338   352 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.584335   567 raft_consensus.cc:2749] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.584618   558 raft_consensus.cc:2749] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.585026   567 raft_consensus.cc:2749] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:56.586155 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad8727419f947cc957c6638ca5fd93a" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.586663 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a207058f154c27b443ceacc88454fe" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:16:56.586257 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
W20260709 13:16:56.590250 32042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 54137.6us
I20260709 13:16:56.591463 32042 leader_election.cc:304] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.596349 32042 leader_election.cc:304] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.597640   558 raft_consensus.cc:2749] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.596007   570 raft_consensus.cc:2749] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.608768 31929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:56.610266 31929 raft_consensus.cc:606] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:56.610862 32041 leader_election.cc:304] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.611976   570 raft_consensus.cc:2749] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:56.612398 32041 leader_election.cc:304] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:56.613317   570 raft_consensus.cc:2749] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.613878 32041 leader_election.cc:304] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:56.614423 31929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62e37d9df96544f08314921b5ba47624"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:56.614784   570 raft_consensus.cc:2749] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.615252 32041 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:56.616161   570 raft_consensus.cc:2749] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:56.615172 31929 raft_consensus.cc:606] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:56.616621 32041 leader_election.cc:304] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:16:56.618084   570 raft_consensus.cc:2749] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:56.645892 32077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d"
dest_uuid: "32690b3987094d51bb4f214321a29067"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:60614
I20260709 13:16:56.646620 32077 raft_consensus.cc:606] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:56.678277 32001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15"
dest_uuid: "ccf4710690c7473d814836da1babee51"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 13:16:56.678982 32001 raft_consensus.cc:606] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:56.682890   543 raft_consensus.cc:993] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.683554   543 raft_consensus.cc:1081] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.684811 31926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:56.685776 31926 raft_consensus.cc:606] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:56.685662   405 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48192
I20260709 13:16:56.686434   405 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.686863   405 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:56.691812 31926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:56.692652 31926 raft_consensus.cc:606] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:56.694020 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:56.696100 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:56.697698   405 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.702332 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:56.703233   405 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.703557 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd8ceaec471453d9530212df301bc15" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:56.704632 31999 raft_consensus.cc:3055] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:56.705047 31999 raft_consensus.cc:740] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:56.706416 31999 consensus_queue.cc:260] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.708801 31999 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:56.703436 31926 raft_consensus.cc:3060] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:56.726593 31999 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 3.
I20260709 13:16:56.730161   354 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:56.731714   563 raft_consensus.cc:2804] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:56.732405   563 raft_consensus.cc:697] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:56.733541   563 consensus_queue.cc:237] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.735540 31926 raft_consensus.cc:2468] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 3.
W20260709 13:16:56.750686 32025 debug-util.cc:398] Leaking SignalData structure 0x7b0800165680 after lost signal to thread 21639
W20260709 13:16:56.752220 32025 debug-util.cc:398] Leaking SignalData structure 0x7b080020bec0 after lost signal to thread 31787
W20260709 13:16:56.765558 32025 debug-util.cc:398] Leaking SignalData structure 0x7b0800218480 after lost signal to thread 439
I20260709 13:16:56.785069   560 raft_consensus.cc:993] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.785989   552 raft_consensus.cc:1081] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.788548   403 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:56.793115   552 raft_consensus.cc:993] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.793953   552 raft_consensus.cc:1081] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.828127   398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:56.828943   550 raft_consensus.cc:993] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.829602   550 raft_consensus.cc:1081] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.844871   398 raft_consensus.cc:493] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.845952   398 raft_consensus.cc:3060] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.847625   397 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:56.854707   403 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.856060   558 raft_consensus.cc:993] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.854403   397 raft_consensus.cc:493] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.862183   558 raft_consensus.cc:1081] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:56.862421   397 raft_consensus.cc:3060] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.863935   403 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.866777   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48202
I20260709 13:16:56.866607 31750 catalog_manager.cc:5697] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 2 to 3, leader changed from ccf4710690c7473d814836da1babee51 (127.21.33.3) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 3 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.871228   406 raft_consensus.cc:493] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:56.871727   406 raft_consensus.cc:3060] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.882081   397 raft_consensus.cc:515] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.888154 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:56.888430   397 leader_election.cc:290] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.889072 31929 raft_consensus.cc:3055] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:56.889480 31929 raft_consensus.cc:740] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:56.892342 31929 consensus_queue.cc:260] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.894071 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4c038e4f034907a5fcaf6ab632a887" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:56.894995 31999 raft_consensus.cc:3060] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.888005   406 raft_consensus.cc:515] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:56.901880 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:56.902570 32070 raft_consensus.cc:3055] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:56.902426   398 raft_consensus.cc:515] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.903029 32070 raft_consensus.cc:740] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:56.904521 31929 raft_consensus.cc:3060] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.904664 32070 consensus_queue.cc:260] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:56.908511 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:56.909247 31924 raft_consensus.cc:3055] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:56.909356 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1bcd4895944417f88a1d53c87eb4a22" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:56.906901 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7460aac4590c4910abca0a74a4863b5d" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:56.910331 32001 raft_consensus.cc:3060] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.910403 31926 raft_consensus.cc:3060] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.906602   406 leader_election.cc:290] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:56.909626 31924 raft_consensus.cc:740] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:56.919919 31924 consensus_queue.cc:260] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.921816 31924 raft_consensus.cc:3060] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.923010 31926 raft_consensus.cc:2468] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:56.908279   398 leader_election.cc:290] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.909729 32070 raft_consensus.cc:3060] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.926126   352 leader_election.cc:304] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:56.928169   563 raft_consensus.cc:2804] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:56.904819   403 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.928810   563 raft_consensus.cc:697] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:56.930063   563 consensus_queue.cc:237] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:56.933287 32001 raft_consensus.cc:2468] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:56.936412 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:56.938290 31926 raft_consensus.cc:3055] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:56.938499 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a930e148e8204d8db397cb765fbef4ef" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:56.938669 31926 raft_consensus.cc:740] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:56.939107 31996 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.939635 31926 consensus_queue.cc:260] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.940937 31926 raft_consensus.cc:3060] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.937362   403 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 4 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:56.945053   354 leader_election.cc:304] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:56.946121   567 raft_consensus.cc:2804] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:56.951432 31924 raft_consensus.cc:2468] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:56.955083 31926 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 4.
I20260709 13:16:56.961076 31996 raft_consensus.cc:2468] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 4.
I20260709 13:16:56.963958 31999 raft_consensus.cc:2468] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:56.966197 32070 raft_consensus.cc:2468] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:56.970499 31929 raft_consensus.cc:2468] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:56.972018   354 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:56.968868   567 raft_consensus.cc:697] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:56.994712   567 consensus_queue.cc:237] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:56.997339   354 leader_election.cc:304] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:57.004264   563 raft_consensus.cc:2804] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:57.006119   579 raft_consensus.cc:2804] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:57.028846   563 raft_consensus.cc:697] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:57.030789   563 consensus_queue.cc:237] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:57.044070   579 raft_consensus.cc:697] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 4 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:57.047622   579 consensus_queue.cc:237] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:57.069155 31750 catalog_manager.cc:5697] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.071308 31750 catalog_manager.cc:5697] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from 32690b3987094d51bb4f214321a29067 (127.21.33.4) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.073109 31750 catalog_manager.cc:5697] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.075188 31750 catalog_manager.cc:5697] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 3 to 4, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 4 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.167889   560 raft_consensus.cc:993] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:57.169255   560 raft_consensus.cc:1081] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:57.174532   403 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62e37d9df96544f08314921b5ba47624"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:57.175161   403 raft_consensus.cc:493] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:57.175516   403 raft_consensus.cc:3060] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:57.186009   403 raft_consensus.cc:515] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:57.190865   403 leader_election.cc:290] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:57.191514 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:57.192322 31996 raft_consensus.cc:3060] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:57.194144 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e37d9df96544f08314921b5ba47624" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:57.194965 31929 raft_consensus.cc:3055] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:57.195489 31929 raft_consensus.cc:740] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:57.196756 31929 consensus_queue.cc:260] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:57.199069 31929 raft_consensus.cc:3060] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:57.208022 31996 raft_consensus.cc:2468] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:57.209013 31929 raft_consensus.cc:2468] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:57.209278   354 leader_election.cc:304] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:57.210402   579 raft_consensus.cc:2804] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:57.211006   579 raft_consensus.cc:697] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:57.211988   579 consensus_queue.cc:237] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:57.231858 31750 catalog_manager.cc:5697] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.255466 31999 raft_consensus.cc:1275] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:57.261775   579 consensus_queue.cc:1048] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:57.271718 31929 raft_consensus.cc:1275] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:57.274417   579 consensus_queue.cc:1048] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:57.418783 31968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:37703, user_credentials={real_user=slave}} blocked reactor thread for 141510us
I20260709 13:16:57.431205 32001 raft_consensus.cc:1275] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.446714   563 consensus_queue.cc:1048] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.486985 31929 raft_consensus.cc:1275] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.506528 31926 raft_consensus.cc:1275] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 4 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:57.507900   579 consensus_queue.cc:1048] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.509953   579 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:57.542780 31997 raft_consensus.cc:1275] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.549795 32070 raft_consensus.cc:1275] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.555447   563 consensus_queue.cc:1048] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.565131   583 consensus_queue.cc:1048] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.570863 31997 raft_consensus.cc:1275] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:57.586786   583 consensus_queue.cc:1048] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:57.594923 31929 raft_consensus.cc:1275] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.599442   583 consensus_queue.cc:1048] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.656620 31997 raft_consensus.cc:1275] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.658478   579 consensus_queue.cc:1048] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:57.658464 31930 raft_consensus.cc:1275] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.666441   567 consensus_queue.cc:1048] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.697410 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:57.710578 31926 raft_consensus.cc:1275] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.714076   563 consensus_queue.cc:1048] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:57.858777 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:57.935532 31926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:57.936419 31926 raft_consensus.cc:606] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:57.940265 31930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:57.940975 31930 raft_consensus.cc:606] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:57.944168 32001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7"
dest_uuid: "ccf4710690c7473d814836da1babee51"
mode: GRACEFUL
new_leader_uuid: "a588481f03f6474d9009d5bd5e08e887"
 from {username='slave'} at 127.0.0.1:38940
I20260709 13:16:57.944823 32001 raft_consensus.cc:606] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Received request to transfer leadership to TS a588481f03f6474d9009d5bd5e08e887
I20260709 13:16:57.949267 31930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:57.950048 31930 raft_consensus.cc:606] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:57.952839 31930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91b9568c210d4ef387297f17cde15389"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "32690b3987094d51bb4f214321a29067"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:57.953552 31930 raft_consensus.cc:606] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 32690b3987094d51bb4f214321a29067
I20260709 13:16:57.962132 31930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:57.962891 31930 raft_consensus.cc:606] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:57.966147 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:57.971853 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:58.054895   543 raft_consensus.cc:993] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: : Instructing follower a588481f03f6474d9009d5bd5e08e887 to start an election
I20260709 13:16:58.055609   543 raft_consensus.cc:1081] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Signalling peer a588481f03f6474d9009d5bd5e08e887 to start an election
I20260709 13:16:58.058095 31930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
 from {username='slave'} at 127.0.0.1:57716
I20260709 13:16:58.058799 31930 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.059208 31930 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:58.073048 31930 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:58.077283 31930 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:16:58.077896 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:58.078811 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4df0e0908b634f91a377721d9f0e3ef7" candidate_uuid: "a588481f03f6474d9009d5bd5e08e887" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:58.079317 32070 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:58.079502 32001 raft_consensus.cc:3055] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:58.080034 32001 raft_consensus.cc:740] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:58.080986 32001 consensus_queue.cc:260] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:58.082574 32001 raft_consensus.cc:3060] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:58.090271 32070 raft_consensus.cc:2468] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 3.
I20260709 13:16:58.094244 31895 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:58.094421 32001 raft_consensus.cc:2468] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a588481f03f6474d9009d5bd5e08e887 in term 3.
I20260709 13:16:58.096728   552 raft_consensus.cc:2804] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:58.097437   552 raft_consensus.cc:697] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Becoming Leader. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:58.098608   552 consensus_queue.cc:237] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:58.161700 31751 catalog_manager.cc:5697] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 reported cstate change: term changed from 2 to 3, leader changed from ccf4710690c7473d814836da1babee51 (127.21.33.3) to a588481f03f6474d9009d5bd5e08e887 (127.21.33.2). New cstate: current_term: 3 leader_uuid: "a588481f03f6474d9009d5bd5e08e887" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.355795   552 raft_consensus.cc:993] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.356482   552 raft_consensus.cc:1081] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.364348   560 raft_consensus.cc:993] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.364970   560 raft_consensus.cc:1081] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.373126   560 raft_consensus.cc:993] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.373998   560 raft_consensus.cc:1081] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.386360   399 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:58.388314   399 raft_consensus.cc:493] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.393362   399 raft_consensus.cc:3060] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.394330   398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:58.394862   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:58.395663   398 raft_consensus.cc:493] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.396193   398 raft_consensus.cc:3060] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.396041   406 raft_consensus.cc:493] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.397826   406 raft_consensus.cc:3060] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.410519   399 raft_consensus.cc:515] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.414187 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:58.414934 32074 raft_consensus.cc:3060] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.427969 31926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3127b63807542e6bb6a2a33677baabc" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:58.428658   406 raft_consensus.cc:515] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.429189 31926 raft_consensus.cc:3055] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.434712   399 leader_election.cc:290] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:58.434446 32074 raft_consensus.cc:2468] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.448524   406 leader_election.cc:290] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:58.434540 31926 raft_consensus.cc:740] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:58.450527   353 leader_election.cc:304] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:58.453882 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:58.454248   583 raft_consensus.cc:2804] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.454654 32074 raft_consensus.cc:3060] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.450455 31926 consensus_queue.cc:260] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.455559   583 raft_consensus.cc:697] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:58.456697 31926 raft_consensus.cc:3060] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.435995   552 raft_consensus.cc:993] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.457031   583 consensus_queue.cc:237] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.458762   551 raft_consensus.cc:1081] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:58.431638   398 raft_consensus.cc:515] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:58.462021   398 leader_election.cc:290] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:58.463425 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfe802c323741a9b432e1eca10b2698" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:58.464181 31929 raft_consensus.cc:3055] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.464524 31929 raft_consensus.cc:740] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:58.465762 31929 consensus_queue.cc:260] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.467694 31929 raft_consensus.cc:3060] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.474823 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:58.475715 32001 raft_consensus.cc:3060] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.487510   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:58.490438   552 raft_consensus.cc:993] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 32690b3987094d51bb4f214321a29067 to start an election
I20260709 13:16:58.494190   545 raft_consensus.cc:1081] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 32690b3987094d51bb4f214321a29067 to start an election
I20260709 13:16:58.493202 32001 raft_consensus.cc:2468] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.496692 32074 raft_consensus.cc:2468] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.491389   406 raft_consensus.cc:493] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.497566 32070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91b9568c210d4ef387297f17cde15389"
dest_uuid: "32690b3987094d51bb4f214321a29067"
 from {username='slave'} at 127.0.0.1:57638
I20260709 13:16:58.497932   406 raft_consensus.cc:3060] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.498242 32070 raft_consensus.cc:493] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:58.498612 32070 raft_consensus.cc:3060] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.498795   353 leader_election.cc:304] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:58.500006   579 raft_consensus.cc:2804] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.500548   579 raft_consensus.cc:697] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:58.501618   579 consensus_queue.cc:237] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.514281   354 leader_election.cc:304] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:58.515244   563 raft_consensus.cc:2804] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.516189   563 raft_consensus.cc:697] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:58.517745   563 consensus_queue.cc:237] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:58.530349   406 raft_consensus.cc:515] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.535207 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:58.536289 32074 raft_consensus.cc:3060] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.550598 32001 raft_consensus.cc:1275] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Refusing update from remote peer a588481f03f6474d9009d5bd5e08e887: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:58.550499   406 leader_election.cc:290] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:58.553041 32074 raft_consensus.cc:2468] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.552803   545 consensus_queue.cc:1048] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:58.554539   353 leader_election.cc:304] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:58.555614   579 raft_consensus.cc:2804] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.556167   579 raft_consensus.cc:697] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:58.557147   579 consensus_queue.cc:237] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.587666 32070 raft_consensus.cc:515] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:58.598160 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08f4aafcb3434b9badfaffebc83bd15c" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:58.599022 31930 raft_consensus.cc:3055] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.590785 31926 raft_consensus.cc:2468] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.599495 31930 raft_consensus.cc:740] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:58.602694 31924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:58.603791 31924 raft_consensus.cc:3055] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.604557 31924 raft_consensus.cc:740] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:58.599087 32070 leader_election.cc:290] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:58.605716 31924 consensus_queue.cc:260] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:58.607303 31924 raft_consensus.cc:3060] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.580524 31929 raft_consensus.cc:2468] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.604688 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91b9568c210d4ef387297f17cde15389" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:58.615451 31997 raft_consensus.cc:3060] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.617667 31929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "395cb97d328c4ccc9cb0b96fe5d83dd6" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:58.622784 31929 raft_consensus.cc:3055] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:58.623438 31929 raft_consensus.cc:740] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:58.627905 31929 consensus_queue.cc:260] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:58.622007 31930 consensus_queue.cc:260] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:58.636664 31930 raft_consensus.cc:3060] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.640842 31929 raft_consensus.cc:3060] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.642534 31924 raft_consensus.cc:2468] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:58.643785 32042 leader_election.cc:304] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:58.644758   570 raft_consensus.cc:2804] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.652457   570 raft_consensus.cc:697] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:58.661181   570 consensus_queue.cc:237] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:58.663514 31997 raft_consensus.cc:2468] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:58.664711 31999 raft_consensus.cc:1240] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Rejecting Update request from peer a588481f03f6474d9009d5bd5e08e887 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:58.667019 31929 raft_consensus.cc:2468] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.673547 31930 raft_consensus.cc:2468] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:58.728230 31750 catalog_manager.cc:5697] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.751183 31751 catalog_manager.cc:5697] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.816480 32070 raft_consensus.cc:1275] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Refusing update from remote peer a588481f03f6474d9009d5bd5e08e887: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:58.825258   551 consensus_queue.cc:1048] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:58.925359 32074 raft_consensus.cc:1275] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:58.929144   567 consensus_queue.cc:1048] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:58.932679 31751 catalog_manager.cc:5697] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.943441 31751 catalog_manager.cc:5697] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.949378 32001 raft_consensus.cc:1275] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:58.959244 31751 catalog_manager.cc:5697] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.964421   567 consensus_queue.cc:1048] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:58.973616 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:59.025848 31924 raft_consensus.cc:1275] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.055351   579 consensus_queue.cc:1048] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.061136 31924 raft_consensus.cc:1275] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.063593 31926 raft_consensus.cc:1275] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.065768   567 consensus_queue.cc:1048] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.069779   567 consensus_queue.cc:1048] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.074762 31924 raft_consensus.cc:1275] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.103943   567 consensus_queue.cc:1048] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.106629 32070 raft_consensus.cc:1275] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.109675   579 consensus_queue.cc:1048] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:16:59.116664 32001 raft_consensus.cc:1275] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.150018   558 consensus_queue.cc:1048] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.225150 31929 raft_consensus.cc:1275] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:59.255739   354 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 79345.1us
W20260709 13:16:59.259399   353 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.4:34551, user_credentials={real_user=slave}} blocked reactor thread for 101189us
I20260709 13:16:59.262362 32077 raft_consensus.cc:1275] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.269589   570 consensus_queue.cc:1048] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.311165   563 consensus_queue.cc:1048] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:59.323124   636 log.cc:927] Time spent T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 13:16:59.444751 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:59.506453 32074 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed"
dest_uuid: "32690b3987094d51bb4f214321a29067"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:60614
I20260709 13:16:59.507057 32074 raft_consensus.cc:606] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:59.510162 31930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391"
dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:50232
I20260709 13:16:59.510814 31930 raft_consensus.cc:606] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:59.514132 32001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5"
dest_uuid: "ccf4710690c7473d814836da1babee51"
mode: GRACEFUL
new_leader_uuid: "32690b3987094d51bb4f214321a29067"
 from {username='slave'} at 127.0.0.1:38940
I20260709 13:16:59.514824 32001 raft_consensus.cc:606] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Received request to transfer leadership to TS 32690b3987094d51bb4f214321a29067
I20260709 13:16:59.519130 32001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe"
dest_uuid: "ccf4710690c7473d814836da1babee51"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:38940
I20260709 13:16:59.519817 32001 raft_consensus.cc:606] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:16:59.521348 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:16:59.523730 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:59.640477   552 raft_consensus.cc:993] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:59.641038   552 raft_consensus.cc:1081] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:59.649097   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48214
I20260709 13:16:59.649833   406 raft_consensus.cc:493] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.650244   406 raft_consensus.cc:3060] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.662221   406 raft_consensus.cc:515] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:59.665091   406 leader_election.cc:290] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:16:59.665887 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:59.666435 32074 raft_consensus.cc:3060] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.669530 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f88110f6d141dc97dbef5c1c4f8391" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:59.670240 31930 raft_consensus.cc:3055] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.670526 31930 raft_consensus.cc:740] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a588481f03f6474d9009d5bd5e08e887, State: Running, Role: LEADER
I20260709 13:16:59.671304 31930 consensus_queue.cc:260] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:59.672611 31930 raft_consensus.cc:3060] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.676947 32074 raft_consensus.cc:2468] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:59.678341   353 leader_election.cc:304] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:59.679682   635 raft_consensus.cc:2804] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.680310   635 raft_consensus.cc:697] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:59.681435   635 consensus_queue.cc:237] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:16:59.688632 31930 raft_consensus.cc:2468] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:16:59.697824   570 raft_consensus.cc:993] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:59.698385   570 raft_consensus.cc:1081] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:59.702611   398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48202
I20260709 13:16:59.703321   398 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.703720   398 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.705271 31750 catalog_manager.cc:5697] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from a588481f03f6474d9009d5bd5e08e887 (127.21.33.2) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.712659   398 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:59.716231   398 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:59.716408 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:16:59.717483 32074 raft_consensus.cc:3055] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:59.717305 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:59.718022 32074 raft_consensus.cc:740] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:59.718268 31996 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.719028 32074 consensus_queue.cc:260] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:59.720367 32074 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.740268 32074 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 3.
I20260709 13:16:59.742324   353 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:16:59.743304   635 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:59.744019   635 raft_consensus.cc:697] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:59.744119 31996 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 3.
I20260709 13:16:59.745124   635 consensus_queue.cc:237] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:16:59.760973 31750 catalog_manager.cc:5697] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 2 to 3, leader changed from 32690b3987094d51bb4f214321a29067 (127.21.33.4) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 3 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:59.846333   543 raft_consensus.cc:993] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:59.847008   543 raft_consensus.cc:1081] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:16:59.848819   398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48192
I20260709 13:16:59.849534   398 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.849933   398 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.860827   398 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:59.868041 32001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:59.870424 32001 raft_consensus.cc:3055] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:59.870671 32001 raft_consensus.cc:740] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:59.871497 32001 consensus_queue.cc:260] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:59.872609 32001 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.881762 32001 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 3.
I20260709 13:16:59.868465 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49fb2266a654cc9bdc77936ae8f38fe" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:59.882813   354 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:16:59.883195 31930 raft_consensus.cc:3060] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.883819   635 raft_consensus.cc:2804] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:59.866637   398 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 3 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:59.884380   635 raft_consensus.cc:697] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:16:59.885381   635 consensus_queue.cc:237] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:16:59.896176 31930 raft_consensus.cc:2468] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 3.
I20260709 13:16:59.910422   543 raft_consensus.cc:993] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: : Instructing follower 32690b3987094d51bb4f214321a29067 to start an election
I20260709 13:16:59.910966   543 raft_consensus.cc:1081] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Signalling peer 32690b3987094d51bb4f214321a29067 to start an election
I20260709 13:16:59.914146 32077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5"
dest_uuid: "32690b3987094d51bb4f214321a29067"
 from {username='slave'} at 127.0.0.1:57636
I20260709 13:16:59.916961 32077 raft_consensus.cc:493] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.918339 32077 raft_consensus.cc:3060] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.924718 31750 catalog_manager.cc:5697] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 2 to 3, leader changed from ccf4710690c7473d814836da1babee51 (127.21.33.3) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 3 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.956151 32077 raft_consensus.cc:515] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:59.959390 31930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a588481f03f6474d9009d5bd5e08e887"
I20260709 13:16:59.960280 31930 raft_consensus.cc:3060] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.966813 32077 leader_election.cc:290] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Requested vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:16:59.967247 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5b26df57d6488798ed73660ded04a5" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:16:59.968187 31999 raft_consensus.cc:3055] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.968535 31999 raft_consensus.cc:740] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:16:59.969427 31999 consensus_queue.cc:260] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:16:59.971032 31930 raft_consensus.cc:2468] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:16:59.972354 32042 leader_election.cc:304] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, a588481f03f6474d9009d5bd5e08e887; no voters: 
I20260709 13:16:59.973269   623 raft_consensus.cc:2804] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.973796   623 raft_consensus.cc:697] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Becoming Leader. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:16:59.975114   623 consensus_queue.cc:237] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:17:00.010701 31999 raft_consensus.cc:3060] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:00.039752 31999 raft_consensus.cc:2468] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32690b3987094d51bb4f214321a29067 in term 2.
I20260709 13:17:00.059942 31750 catalog_manager.cc:5697] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 reported cstate change: term changed from 1 to 2, leader changed from ccf4710690c7473d814836da1babee51 (127.21.33.3) to 32690b3987094d51bb4f214321a29067 (127.21.33.4). New cstate: current_term: 2 leader_uuid: "32690b3987094d51bb4f214321a29067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:00.099802 31929 raft_consensus.cc:1275] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.102900   583 consensus_queue.cc:1048] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:00.122231 32070 raft_consensus.cc:1275] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.123998   637 consensus_queue.cc:1048] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.166491 31999 raft_consensus.cc:1275] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:00.175128   563 consensus_queue.cc:1048] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:00.206693 32070 raft_consensus.cc:1275] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:00.209009   563 consensus_queue.cc:1048] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:00.401031 31999 raft_consensus.cc:1275] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.402489   623 consensus_queue.cc:1048] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.430521 31929 raft_consensus.cc:1275] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Refusing update from remote peer 32690b3987094d51bb4f214321a29067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.431521 31926 raft_consensus.cc:1275] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:00.432310   623 consensus_queue.cc:1048] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.434549   563 consensus_queue.cc:1048] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:17:00.519678 32044 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:38635, user_credentials={real_user=slave}} blocked reactor thread for 43592.6us
I20260709 13:17:00.525182 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:00.531345 31967 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:35847, user_credentials={real_user=slave}} blocked reactor thread for 52676.9us
I20260709 13:17:00.616571 31997 raft_consensus.cc:1275] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:00.665942 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:00.679754   663 consensus_queue.cc:1048] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:00.792583 32070 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84"
dest_uuid: "32690b3987094d51bb4f214321a29067"
mode: GRACEFUL
new_leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:60614
I20260709 13:17:00.793258 32070 raft_consensus.cc:606] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Received request to transfer leadership to TS 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:17:00.795346 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:00.799469 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:01.060338   570 raft_consensus.cc:993] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: : Instructing follower 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:17:01.060999   570 raft_consensus.cc:1081] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Signalling peer 103d6e4c312a47ff9b258c14ac2871e5 to start an election
I20260709 13:17:01.064306   403 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84"
dest_uuid: "103d6e4c312a47ff9b258c14ac2871e5"
 from {username='slave'} at 127.0.0.1:48202
I20260709 13:17:01.065012   403 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:01.065431   403 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:01.074936   403 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:01.083006 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:17:01.083725 32070 raft_consensus.cc:3055] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:01.083981 32070 raft_consensus.cc:740] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 32690b3987094d51bb4f214321a29067, State: Running, Role: LEADER
I20260709 13:17:01.084182 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "103d6e4c312a47ff9b258c14ac2871e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccf4710690c7473d814836da1babee51"
I20260709 13:17:01.084905 31996 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:01.084720 32070 consensus_queue.cc:260] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:01.086001 32070 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:01.086067   403 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Requested vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551)
I20260709 13:17:01.102334 32070 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:17:01.103737   353 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103d6e4c312a47ff9b258c14ac2871e5, 32690b3987094d51bb4f214321a29067; no voters: 
I20260709 13:17:01.104699   663 raft_consensus.cc:2804] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:01.105223   663 raft_consensus.cc:697] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Becoming Leader. State: Replica: 103d6e4c312a47ff9b258c14ac2871e5, State: Running, Role: LEADER
I20260709 13:17:01.106688   663 consensus_queue.cc:237] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:01.108207 31996 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103d6e4c312a47ff9b258c14ac2871e5 in term 2.
I20260709 13:17:01.211310 31750 catalog_manager.cc:5697] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 reported cstate change: term changed from 1 to 2, leader changed from 32690b3987094d51bb4f214321a29067 (127.21.33.4) to 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "103d6e4c312a47ff9b258c14ac2871e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.633399 32070 raft_consensus.cc:1275] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:01.634845   664 consensus_queue.cc:1048] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:01.671115 31996 raft_consensus.cc:1275] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Refusing update from remote peer 103d6e4c312a47ff9b258c14ac2871e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:01.676740   663 consensus_queue.cc:1048] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:01.800720 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:01.843706 21636 tablet_server.cc:179] TabletServer@127.21.33.1:35847 shutting down...
W20260709 13:17:01.851678 32041 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 401 similar messages]
W20260709 13:17:01.859923 31893 consensus_peers.cc:597] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.871665 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.876859 31967 consensus_peers.cc:597] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.894460 31893 consensus_peers.cc:597] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.898514 31893 consensus_peers.cc:597] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.899792 31893 consensus_peers.cc:597] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.900249 32041 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.901995 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.940065 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:01.967003 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:01.969478 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.971249 21636 raft_consensus.cc:2243] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:01.971886 32041 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:01.972354 21636 raft_consensus.cc:2272] T 809ff13e62914dd7854fa320a1173da0 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:01.972381 31967 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:01.976919 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.978118 21636 raft_consensus.cc:2243] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.979285 21636 raft_consensus.cc:2272] T 87842118d5214e359e7983d9ede2213b P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.985638 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.986714 21636 raft_consensus.cc:2243] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.987305 21636 raft_consensus.cc:2272] T 2b97d68a6e314f6792e4c8bfa99d3025 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:01.990617 31893 consensus_peers.cc:597] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:01.990833 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.993405 21636 raft_consensus.cc:2243] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:01.994576 21636 raft_consensus.cc:2272] T 08f4aafcb3434b9badfaffebc83bd15c P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:01.996413 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:01.998037 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.999053 21636 raft_consensus.cc:2243] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.999521 21636 raft_consensus.cc:2272] T ea7811f2bbfe41aea0eba9a542d7dee8 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.002868 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.003820 21636 raft_consensus.cc:2243] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.004387 21636 raft_consensus.cc:2272] T 58ea2a2bfc574918bd4f2bf7f2839e9c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.007630 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.008644 21636 raft_consensus.cc:2243] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.010341 21636 raft_consensus.cc:2272] T 48d352eb04d043328db56fab2171304e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.014092 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.014889 21636 raft_consensus.cc:2243] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.015398 21636 raft_consensus.cc:2272] T 77e32837dc3e4e7d99a94d477a4d50d1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.019809 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.020855 21636 raft_consensus.cc:2243] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.021471 21636 raft_consensus.cc:2272] T 1fc5e72ae63b4c96bc88199a467670e2 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.026976 21636 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:02.027514 31967 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.027956 21636 raft_consensus.cc:2243] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.028611 21636 raft_consensus.cc:2272] T 8e16bbde8ad04886af986347646976b7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.031841 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.032546 21636 raft_consensus.cc:2243] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.033079 21636 raft_consensus.cc:2272] T 4b0b712922784fd8a86a1c486c9d372e P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.035938 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.036679 21636 raft_consensus.cc:2243] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.037381 21636 raft_consensus.cc:2272] T c85fd3d52a714f44a3e322f4488f1a5c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.040571 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.041412 21636 raft_consensus.cc:2243] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.042521 21636 raft_consensus.cc:2272] T 4dfe802c323741a9b432e1eca10b2698 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.044019 32041 consensus_peers.cc:597] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.045760 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.046808 21636 raft_consensus.cc:2243] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 4 LEADER]: Raft consensus shutting down.
W20260709 13:17:02.047408 31967 consensus_peers.cc:597] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.048132 21636 raft_consensus.cc:2272] T a930e148e8204d8db397cb765fbef4ef P 103d6e4c312a47ff9b258c14ac2871e5 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.050704 31967 consensus_peers.cc:597] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.051501 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.052281 21636 raft_consensus.cc:2243] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.053395 21636 raft_consensus.cc:2272] T 75f88110f6d141dc97dbef5c1c4f8391 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.056782 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.057658 21636 raft_consensus.cc:2243] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.058416 21636 raft_consensus.cc:2272] T dea199ae81974f50bda37760537988a5 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.060988 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.061735 21636 raft_consensus.cc:2243] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.062182 21636 raft_consensus.cc:2272] T 142b461239534bae915d26581eb87d21 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.066828 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.067737 21636 raft_consensus.cc:2243] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.068503 21636 raft_consensus.cc:2272] T 79ebfc33b37b45f69df96a3f4bb8f422 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.071949 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.072659 21636 raft_consensus.cc:2243] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:17:02.073349 31893 consensus_peers.cc:597] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.073781 21636 raft_consensus.cc:2272] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.076844 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.077769 21636 raft_consensus.cc:2243] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.078254 21636 raft_consensus.cc:2272] T e9e460a405ae485fb15869970e178caf P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.079905 32041 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.081987 21636 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:02.081910 32041 consensus_peers.cc:597] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.083087 21636 raft_consensus.cc:2243] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.084447 21636 raft_consensus.cc:2272] T d3127b63807542e6bb6a2a33677baabc P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.086411 32041 consensus_peers.cc:597] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.087450 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.088243 21636 raft_consensus.cc:2243] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.089293 21636 raft_consensus.cc:2272] T c49fb2266a654cc9bdc77936ae8f38fe P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.092701 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.093612 21636 raft_consensus.cc:2243] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.095264 21636 raft_consensus.cc:2272] T c1bcd4895944417f88a1d53c87eb4a22 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.095549 31893 consensus_peers.cc:597] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.098433 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.099488 21636 raft_consensus.cc:2243] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.100737 21636 raft_consensus.cc:2272] T fb6e2830158d41d3bb09aeaad85a5f84 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.106379 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.107340 21636 raft_consensus.cc:2243] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.107861 21636 raft_consensus.cc:2272] T a2ce179ed2a94df48e1b5f00fec58804 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.111399 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.112219 21636 raft_consensus.cc:2243] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.112710 21636 raft_consensus.cc:2272] T d7340289c3434c3782e3e367d4de5258 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.116159 21636 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:02.116645 31967 consensus_peers.cc:597] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.117085 21636 raft_consensus.cc:2243] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.117749 21636 raft_consensus.cc:2272] T b90a6b486bef42019735b76ff4da04a1 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.121070 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.121989 21636 raft_consensus.cc:2243] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.122503 21636 raft_consensus.cc:2272] T 1b4e043220524dfa8c9106bfe7690bcd P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.125881 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.126747 21636 raft_consensus.cc:2243] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.127283 21636 raft_consensus.cc:2272] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.131477 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.132378 21636 raft_consensus.cc:2243] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.133410 21636 raft_consensus.cc:2272] T 7dd8ceaec471453d9530212df301bc15 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.136165 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.136785 21636 raft_consensus.cc:2243] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.137236 21636 raft_consensus.cc:2272] T 441576026a7f44f990e9aa61c86435d6 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.140276 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.141026 21636 raft_consensus.cc:2243] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.141462 21636 raft_consensus.cc:2272] T 2611ce5875fe4e55a870d70749bd1ec4 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.143968 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.144562 21636 raft_consensus.cc:2243] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.145628 21636 raft_consensus.cc:2272] T 7460aac4590c4910abca0a74a4863b5d P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.149292 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.149935 21636 raft_consensus.cc:2243] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.150599 21636 raft_consensus.cc:2272] T 604a6aa17b764d61b1403b11157ddbe7 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.153143 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.153846 21636 raft_consensus.cc:2243] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.154692 21636 raft_consensus.cc:2272] T 62e37d9df96544f08314921b5ba47624 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.157292 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.157960 21636 raft_consensus.cc:2243] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.158443 21636 raft_consensus.cc:2272] T bfc3d708d55c46bcb69cfb239324f375 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.162029 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.163211 21636 raft_consensus.cc:2243] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.163765 21636 raft_consensus.cc:2272] T 8ad8727419f947cc957c6638ca5fd93a P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.166755 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.167438 21636 raft_consensus.cc:2243] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.167984 21636 raft_consensus.cc:2272] T be826572f20c45fe84142ea4f204ba26 P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.170605 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.171226 21636 raft_consensus.cc:2243] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.171679 21636 raft_consensus.cc:2272] T 6247a65da5884caba020ed13be64352f P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.173955 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.174942 21636 raft_consensus.cc:2243] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.175470 21636 raft_consensus.cc:2272] T 34769db9ad7a46a8b075b865d2a2b6ea P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.178794 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.179622 21636 raft_consensus.cc:2243] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.180711 21636 raft_consensus.cc:2272] T 221d9ca0b1f9450ead6ef9e225fe17ed P 103d6e4c312a47ff9b258c14ac2871e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.183733 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.184614 21636 raft_consensus.cc:2243] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.185878 21636 raft_consensus.cc:2272] T 0c4c038e4f034907a5fcaf6ab632a887 P 103d6e4c312a47ff9b258c14ac2871e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.189021 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.189872 21636 raft_consensus.cc:2243] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.190490 21636 raft_consensus.cc:2272] T 125eaf8f67bd4727b6d49528c95ca751 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.192905 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.193441 21636 raft_consensus.cc:2243] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.193994 21636 raft_consensus.cc:2272] T 87e90fd5526b4a2fbfac6d06dc90f086 P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.196519 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.197204 21636 raft_consensus.cc:2243] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.197742 21636 raft_consensus.cc:2272] T 14858ce2d22c4dba9d14295ca6c2b23c P 103d6e4c312a47ff9b258c14ac2871e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.261591 31893 consensus_peers.cc:597] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.288599 31967 consensus_peers.cc:597] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.321586 32041 consensus_peers.cc:597] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.322402 31893 consensus_peers.cc:597] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.323370 31967 consensus_peers.cc:597] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.345839 32041 connection.cc:573] client connection to 127.21.33.1:35847 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:17:02.402328 31893 consensus_peers.cc:597] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.439240 21636 tablet_server.cc:196] TabletServer@127.21.33.1:35847 shutdown complete.
W20260709 13:17:02.501573 31967 consensus_peers.cc:597] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.507375 31967 consensus_peers.cc:597] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.706055 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
W20260709 13:17:02.715030 31968 consensus_peers.cc:597] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.739352 31968 consensus_peers.cc:597] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:02.747195 32042 consensus_peers.cc:597] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.771248 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:02.774333 21636 tablet_replica.cc:333] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.775454 21636 raft_consensus.cc:2243] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.777007 21636 raft_consensus.cc:2272] T 48d352eb04d043328db56fab2171304e P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.780411 21636 tablet_replica.cc:333] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.781374 21636 raft_consensus.cc:2243] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.782670 21636 raft_consensus.cc:2272] T 142b461239534bae915d26581eb87d21 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.785655 21636 tablet_replica.cc:333] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.786574 21636 raft_consensus.cc:2243] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.787125 21636 raft_consensus.cc:2272] T 7a5b26df57d6488798ed73660ded04a5 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.790158 21636 tablet_replica.cc:333] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.790783 21636 raft_consensus.cc:2243] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:02.790722 31968 consensus_peers.cc:597] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.791611 21636 raft_consensus.cc:2272] T 87e90fd5526b4a2fbfac6d06dc90f086 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.794179 21636 tablet_replica.cc:333] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.794945 21636 raft_consensus.cc:2243] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.795473 21636 raft_consensus.cc:2272] T 3474410429d7478d8ba2221c7400cb01 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.798800 21636 tablet_replica.cc:333] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
W20260709 13:17:02.799059 32042 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.799733 21636 raft_consensus.cc:2243] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.800781 21636 raft_consensus.cc:2272] T 1b4e043220524dfa8c9106bfe7690bcd P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.803249 21636 tablet_replica.cc:333] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.803853 21636 raft_consensus.cc:2243] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.804270 21636 raft_consensus.cc:2272] T 4c11d6a5d0564e90b9ceda978d9225cb P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.806547 21636 tablet_replica.cc:333] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.807156 21636 raft_consensus.cc:2243] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.808172 21636 raft_consensus.cc:2272] T b90a6b486bef42019735b76ff4da04a1 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.811257 21636 tablet_replica.cc:333] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.812320 21636 raft_consensus.cc:2243] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.812937 21636 raft_consensus.cc:2272] T 1fc5e72ae63b4c96bc88199a467670e2 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.816025 21636 tablet_replica.cc:333] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.816874 21636 raft_consensus.cc:2243] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:02.817055 31968 consensus_peers.cc:597] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.817416 21636 raft_consensus.cc:2272] T 658f92ce9d684ffbbec92446706f44c9 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.820230 21636 tablet_replica.cc:333] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.820950 21636 raft_consensus.cc:2243] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.821425 21636 raft_consensus.cc:2272] T dea199ae81974f50bda37760537988a5 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.893302 21636 tablet_replica.cc:333] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.894307 21636 raft_consensus.cc:2243] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.894881 21636 raft_consensus.cc:2272] T a930e148e8204d8db397cb765fbef4ef P a588481f03f6474d9009d5bd5e08e887 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.898118 21636 tablet_replica.cc:333] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.899089 21636 raft_consensus.cc:2243] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.899650 21636 raft_consensus.cc:2272] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.903060 21636 tablet_replica.cc:333] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.903909 21636 raft_consensus.cc:2243] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.904425 21636 raft_consensus.cc:2272] T c1bcd4895944417f88a1d53c87eb4a22 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.912880 21636 tablet_replica.cc:333] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.914717 21636 raft_consensus.cc:2243] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.915344 21636 raft_consensus.cc:2272] T c49fb2266a654cc9bdc77936ae8f38fe P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.918378 21636 tablet_replica.cc:333] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.919025 21636 raft_consensus.cc:2243] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.919492 21636 raft_consensus.cc:2272] T d3127b63807542e6bb6a2a33677baabc P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.921905 21636 tablet_replica.cc:333] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.922572 21636 raft_consensus.cc:2243] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.923038 21636 raft_consensus.cc:2272] T 75f88110f6d141dc97dbef5c1c4f8391 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.925484 21636 tablet_replica.cc:333] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.926321 21636 raft_consensus.cc:2243] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.926872 21636 raft_consensus.cc:2272] T 58ea2a2bfc574918bd4f2bf7f2839e9c P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.929876 21636 tablet_replica.cc:333] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.930629 21636 raft_consensus.cc:2243] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.931166 21636 raft_consensus.cc:2272] T 4dfe802c323741a9b432e1eca10b2698 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.933998 21636 tablet_replica.cc:333] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.934773 21636 raft_consensus.cc:2243] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.935887 21636 raft_consensus.cc:2272] T e9e460a405ae485fb15869970e178caf P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.938808 21636 tablet_replica.cc:333] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.939591 21636 raft_consensus.cc:2243] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.940102 21636 raft_consensus.cc:2272] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.940475 31968 consensus_peers.cc:597] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.942972 21636 tablet_replica.cc:333] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.943965 21636 raft_consensus.cc:2243] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.944559 21636 raft_consensus.cc:2272] T bfc3d708d55c46bcb69cfb239324f375 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.945963 31968 consensus_peers.cc:597] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.947778 21636 tablet_replica.cc:333] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.949653 21636 raft_consensus.cc:2243] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.950652 21636 raft_consensus.cc:2272] T 34769db9ad7a46a8b075b865d2a2b6ea P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.953562 21636 tablet_replica.cc:333] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.954349 21636 raft_consensus.cc:2243] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.955806 21636 raft_consensus.cc:2272] T 6247a65da5884caba020ed13be64352f P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.959290 21636 tablet_replica.cc:333] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.960142 21636 raft_consensus.cc:2243] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.960719 21636 raft_consensus.cc:2272] T 62477156fe044dad9635981556678bef P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.963796 21636 tablet_replica.cc:333] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
W20260709 13:17:02.963980 31968 consensus_peers.cc:597] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.964649 21636 raft_consensus.cc:2243] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.965245 21636 raft_consensus.cc:2272] T 125eaf8f67bd4727b6d49528c95ca751 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.968181 21636 tablet_replica.cc:333] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.969076 21636 raft_consensus.cc:2243] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.969659 21636 raft_consensus.cc:2272] T 0c4c038e4f034907a5fcaf6ab632a887 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.972023 32042 consensus_peers.cc:597] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:02.972846 21636 tablet_replica.cc:333] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.973623 21636 raft_consensus.cc:2243] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.974467 21636 raft_consensus.cc:2272] T 4df0e0908b634f91a377721d9f0e3ef7 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.977442 21636 tablet_replica.cc:333] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.978120 21636 raft_consensus.cc:2243] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.978645 21636 raft_consensus.cc:2272] T 7460aac4590c4910abca0a74a4863b5d P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.981252 21636 tablet_replica.cc:333] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.982115 21636 raft_consensus.cc:2243] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.982681 21636 raft_consensus.cc:2272] T 91b9568c210d4ef387297f17cde15389 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.985744 21636 tablet_replica.cc:333] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.986533 21636 raft_consensus.cc:2243] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.987087 21636 raft_consensus.cc:2272] T 62e37d9df96544f08314921b5ba47624 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.991509 21636 tablet_replica.cc:333] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.992308 21636 raft_consensus.cc:2243] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.993407 21636 raft_consensus.cc:2272] T 340107361dee4b3997a43799863f36e8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.996178 21636 tablet_replica.cc:333] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:02.996837 21636 raft_consensus.cc:2243] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.997864 21636 raft_consensus.cc:2272] T 79ebfc33b37b45f69df96a3f4bb8f422 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.000237 21636 tablet_replica.cc:333] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.000880 21636 raft_consensus.cc:2243] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.001362 21636 raft_consensus.cc:2272] T 809ff13e62914dd7854fa320a1173da0 P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.003535 21636 tablet_replica.cc:333] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.004108 21636 raft_consensus.cc:2243] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.005108 21636 raft_consensus.cc:2272] T 716630c6a4e249aeb96ef3ac276290c8 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.007812 21636 tablet_replica.cc:333] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.008584 21636 raft_consensus.cc:2243] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.009749 21636 raft_consensus.cc:2272] T 039ecd5ff518405fb0fc382c46e7e7c4 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.012683 21636 tablet_replica.cc:333] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.013446 21636 raft_consensus.cc:2243] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.014595 21636 raft_consensus.cc:2272] T 441576026a7f44f990e9aa61c86435d6 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.015588 32042 consensus_peers.cc:597] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.017292 21636 tablet_replica.cc:333] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.018189 21636 raft_consensus.cc:2243] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.018774 21636 raft_consensus.cc:2272] T 7dd8ceaec471453d9530212df301bc15 P a588481f03f6474d9009d5bd5e08e887 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.021401 21636 tablet_replica.cc:333] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.022220 21636 raft_consensus.cc:2243] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.022702 21636 raft_consensus.cc:2272] T ab0ae780de5843cea623c39fd867cc57 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.024910 21636 tablet_replica.cc:333] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.025489 21636 raft_consensus.cc:2243] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.026010 21636 raft_consensus.cc:2272] T 08f4aafcb3434b9badfaffebc83bd15c P a588481f03f6474d9009d5bd5e08e887 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.028438 21636 tablet_replica.cc:333] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.028983 21636 raft_consensus.cc:2243] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.029492 21636 raft_consensus.cc:2272] T 973ae55c06694504ba369b35b010b993 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.032372 21636 tablet_replica.cc:333] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.033075 21636 raft_consensus.cc:2243] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.034214 21636 raft_consensus.cc:2272] T 56a207058f154c27b443ceacc88454fe P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.037071 21636 tablet_replica.cc:333] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.037796 21636 raft_consensus.cc:2243] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.038956 21636 raft_consensus.cc:2272] T 8e16bbde8ad04886af986347646976b7 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.041952 21636 tablet_replica.cc:333] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887: stopping tablet replica
I20260709 13:17:03.042608 21636 raft_consensus.cc:2243] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.043188 21636 raft_consensus.cc:2272] T a2ce179ed2a94df48e1b5f00fec58804 P a588481f03f6474d9009d5bd5e08e887 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.049706 32042 consensus_peers.cc:597] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:03.067322 31968 consensus_peers.cc:597] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:03.090168 32042 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.115842   610 raft_consensus.cc:493] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.116475   610 raft_consensus.cc:515] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:17:03.119220   610 leader_election.cc:290] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:17:03.134100 31968 leader_election.cc:336] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
W20260709 13:17:03.141582 31967 leader_election.cc:336] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.142107 31967 leader_election.cc:304] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.143564   610 raft_consensus.cc:2749] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.150453 32042 consensus_peers.cc:597] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.158913   558 raft_consensus.cc:493] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.159525   558 raft_consensus.cc:515] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:03.161799   658 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.162472   658 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:03.164829   658 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:17:03.167820 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:17:03.169394 31999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "32690b3987094d51bb4f214321a29067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ccf4710690c7473d814836da1babee51" is_pre_election: true
I20260709 13:17:03.176648   558 leader_election.cc:290] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.181973   610 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.182634   610 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:03.185660 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:17:03.186304 32070 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 2.
W20260709 13:17:03.187291 32042 leader_election.cc:336] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.188284 31969 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:17:03.189265   543 raft_consensus.cc:2804] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:03.186261   610 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:17:03.189751   543 raft_consensus.cc:493] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.190279   543 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:03.193614 31967 leader_election.cc:336] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.204188   558 raft_consensus.cc:493] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.204846   558 raft_consensus.cc:515] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:03.205900   610 raft_consensus.cc:493] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.206517   610 raft_consensus.cc:515] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:17:03.208714   610 leader_election.cc:290] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:03.213485 32042 leader_election.cc:336] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.214517   558 leader_election.cc:290] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:03.215443 31967 leader_election.cc:336] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.217585   558 raft_consensus.cc:493] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.218227   558 raft_consensus.cc:515] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
W20260709 13:17:03.220474 31968 consensus_peers.cc:597] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:03.226125 31968 leader_election.cc:336] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.226642 31968 leader_election.cc:304] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.229908   610 raft_consensus.cc:2749] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.234884 32042 leader_election.cc:336] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.249166   690 raft_consensus.cc:493] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.249933   690 raft_consensus.cc:515] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:17:03.257874 31967 leader_election.cc:336] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.200778   543 raft_consensus.cc:515] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:03.220504   558 leader_election.cc:290] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:03.262976 32042 consensus_peers.cc:597] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 -> Peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Couldn't send request to peer a588481f03f6474d9009d5bd5e08e887. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.263270   690 leader_election.cc:290] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.264283 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb6e2830158d41d3bb09aeaad85a5f84" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:17:03.264912 32074 raft_consensus.cc:3060] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:03.265641 31968 leader_election.cc:336] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.266165 31968 leader_election.cc:304] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.270074   691 raft_consensus.cc:2749] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:03.271903 32074 raft_consensus.cc:2468] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 3.
I20260709 13:17:03.273013   543 leader_election.cc:290] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:17:03.273144 31969 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 
I20260709 13:17:03.274119   543 raft_consensus.cc:2804] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:03.275409   543 raft_consensus.cc:697] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 3 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:17:03.276307   543 consensus_queue.cc:237] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:17:03.280323 31967 leader_election.cc:336] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:03.295060 32041 leader_election.cc:336] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.302582 32041 leader_election.cc:304] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:17:03.303614   658 raft_consensus.cc:2749] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.308624 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
W20260709 13:17:03.320533 32041 leader_election.cc:336] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.321281 32041 leader_election.cc:304] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.334372   570 raft_consensus.cc:2749] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.341280 32044 consensus_peers.cc:597] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:03.342078 32044 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.346148   543 raft_consensus.cc:493] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.347195   543 raft_consensus.cc:515] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:17:03.348757 32041 leader_election.cc:336] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.349303 32041 leader_election.cc:304] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.349975   543 leader_election.cc:290] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.349390 31752 catalog_manager.cc:5697] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 reported cstate change: term changed from 2 to 3, leader changed from 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1) to ccf4710690c7473d814836da1babee51 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "ccf4710690c7473d814836da1babee51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.352602   570 raft_consensus.cc:2749] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.353880 32041 leader_election.cc:336] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.354380 32041 leader_election.cc:304] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.355185   570 raft_consensus.cc:2749] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.365056 31968 leader_election.cc:336] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.367445   658 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.368132   658 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
W20260709 13:17:03.369379 31967 leader_election.cc:336] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.369910 31967 leader_election.cc:304] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.370831   689 raft_consensus.cc:2749] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:03.370878   658 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ccf4710690c7473d814836da1babee51 (127.21.33.3:38635), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:17:03.377669 32044 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:03.355968   543 raft_consensus.cc:493] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.383500   658 raft_consensus.cc:493] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.384147   658 raft_consensus.cc:515] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:03.383755   543 raft_consensus.cc:515] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:17:03.386770   658 leader_election.cc:290] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.387529   543 leader_election.cc:290] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:03.394836 31967 leader_election.cc:336] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.395442   543 raft_consensus.cc:493] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
W20260709 13:17:03.397305 32042 leader_election.cc:336] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
W20260709 13:17:03.398623 32041 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.399262 32041 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, ccf4710690c7473d814836da1babee51
I20260709 13:17:03.399535   543 raft_consensus.cc:515] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
W20260709 13:17:03.399986 31968 leader_election.cc:336] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.401959   543 leader_election.cc:290] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.403258   658 raft_consensus.cc:2749] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:03.408259 31968 leader_election.cc:304] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.409053   543 raft_consensus.cc:2749] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:03.410831 31967 leader_election.cc:336] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:03.411537 32041 leader_election.cc:336] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.411967 32041 leader_election.cc:304] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.412582   658 raft_consensus.cc:2749] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.426934 31968 leader_election.cc:336] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.427485 31968 leader_election.cc:304] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.428261   691 raft_consensus.cc:2749] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.450239 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:03.451300 21636 tablet_replica.cc:333] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.452199 21636 raft_consensus.cc:2243] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.452786 21636 raft_consensus.cc:2272] T 48d352eb04d043328db56fab2171304e P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.456143 21636 tablet_replica.cc:333] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.457008 21636 raft_consensus.cc:2243] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.458087 21636 raft_consensus.cc:2272] T 87842118d5214e359e7983d9ede2213b P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.459131 32044 consensus_peers.cc:597] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.460974 21636 tablet_replica.cc:333] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.462039 21636 raft_consensus.cc:2243] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.463225 21636 raft_consensus.cc:2272] T 87e90fd5526b4a2fbfac6d06dc90f086 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.466959 21636 tablet_replica.cc:333] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.467837 21636 raft_consensus.cc:2243] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.468438 21636 raft_consensus.cc:2272] T 56a207058f154c27b443ceacc88454fe P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.471789 21636 tablet_replica.cc:333] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.472721 21636 raft_consensus.cc:2243] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.473289 21636 raft_consensus.cc:2272] T 7a5b26df57d6488798ed73660ded04a5 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.476354 21636 tablet_replica.cc:333] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.477442 21636 raft_consensus.cc:2243] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.478104 21636 raft_consensus.cc:2272] T d7340289c3434c3782e3e367d4de5258 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.479045   691 raft_consensus.cc:493] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.479581   691 raft_consensus.cc:515] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } }
I20260709 13:17:03.481300 21636 tablet_replica.cc:333] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.482491 21636 raft_consensus.cc:2243] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.483292 21636 raft_consensus.cc:2272] T e9e460a405ae485fb15869970e178caf P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.486629 21636 tablet_replica.cc:333] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51: stopping tablet replica
W20260709 13:17:03.487353 31967 leader_election.cc:336] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.487533 21636 raft_consensus.cc:2243] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.488842 21636 raft_consensus.cc:2272] T 3474410429d7478d8ba2221c7400cb01 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.489195   691 leader_election.cc:290] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.492439 21636 tablet_replica.cc:333] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51: stopping tablet replica
W20260709 13:17:03.493032 31968 leader_election.cc:336] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.493877 31968 leader_election.cc:304] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.496459   691 raft_consensus.cc:2749] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.497924 21636 raft_consensus.cc:2243] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.498586 21636 raft_consensus.cc:2272] T c85fd3d52a714f44a3e322f4488f1a5c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.501823 21636 tablet_replica.cc:333] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.502699 21636 raft_consensus.cc:2243] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.503877 21636 raft_consensus.cc:2272] T 4c11d6a5d0564e90b9ceda978d9225cb P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.504112 32044 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.507067 21636 tablet_replica.cc:333] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.508095 21636 raft_consensus.cc:2243] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.508648 21636 raft_consensus.cc:2272] T c1bcd4895944417f88a1d53c87eb4a22 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.511811 21636 tablet_replica.cc:333] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.512655 21636 raft_consensus.cc:2243] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.513197 21636 raft_consensus.cc:2272] T c49fb2266a654cc9bdc77936ae8f38fe P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.516305 21636 tablet_replica.cc:333] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51: stopping tablet replica
W20260709 13:17:03.517135 32044 consensus_peers.cc:597] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.517191 21636 raft_consensus.cc:2243] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.518817 21636 raft_consensus.cc:2272] T 2b97d68a6e314f6792e4c8bfa99d3025 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.522089 21636 tablet_replica.cc:333] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.523058 21636 raft_consensus.cc:2243] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.523624 21636 raft_consensus.cc:2272] T 658f92ce9d684ffbbec92446706f44c9 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.565459 21636 tablet_replica.cc:333] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.567149   558 raft_consensus.cc:493] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.568943   558 raft_consensus.cc:515] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:03.570245 21636 raft_consensus.cc:2243] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.573539   558 leader_election.cc:290] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:03.575249 21636 pending_rounds.cc:70] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Trying to abort 1 pending ops.
I20260709 13:17:03.578145 21636 pending_rounds.cc:77] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305637983354781696 op_type: NO_OP noop_request { }
I20260709 13:17:03.578979 21636 raft_consensus.cc:2889] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.579605 21636 raft_consensus.cc:2272] T fb6e2830158d41d3bb09aeaad85a5f84 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.582561 32044 consensus_peers.cc:597] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:03.584899 32044 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.591964 21636 tablet_replica.cc:333] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51: stopping tablet replica
W20260709 13:17:03.593861 32041 leader_election.cc:336] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:03.599294 32042 leader_election.cc:336] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.599895 32042 leader_election.cc:304] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.595638 21636 raft_consensus.cc:2243] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.601081   570 raft_consensus.cc:2749] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.604583 21636 raft_consensus.cc:2272] T 973ae55c06694504ba369b35b010b993 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.610873 21636 tablet_replica.cc:333] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.610951   691 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.612507   691 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:03.613057 21636 raft_consensus.cc:2243] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.614408 21636 raft_consensus.cc:2272] T 08f4aafcb3434b9badfaffebc83bd15c P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.617677   691 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:17:03.619941 21636 tablet_replica.cc:333] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.621582 21636 raft_consensus.cc:2243] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.622475 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067" is_pre_election: true
I20260709 13:17:03.622831 21636 raft_consensus.cc:2272] T a930e148e8204d8db397cb765fbef4ef P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.623507 32074 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 3.
W20260709 13:17:03.626526 31967 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.627872 31969 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:17:03.629133   689 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:03.629310 21636 tablet_replica.cc:333] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.630642   689 raft_consensus.cc:493] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 103d6e4c312a47ff9b258c14ac2871e5)
I20260709 13:17:03.631147   689 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.631150 21636 raft_consensus.cc:2243] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.632459 21636 raft_consensus.cc:2272] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.633666 32044 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.636865 21636 tablet_replica.cc:333] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.638278 21636 raft_consensus.cc:2243] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.640983 21636 raft_consensus.cc:2272] T dea199ae81974f50bda37760537988a5 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.646479 21636 tablet_replica.cc:333] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.646782   689 raft_consensus.cc:515] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:03.649222 21636 raft_consensus.cc:2243] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.650735 32074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "221d9ca0b1f9450ead6ef9e225fe17ed" candidate_uuid: "ccf4710690c7473d814836da1babee51" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "32690b3987094d51bb4f214321a29067"
I20260709 13:17:03.651549 32074 raft_consensus.cc:3060] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:03.656287 32044 consensus_peers.cc:597] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.657579 21636 raft_consensus.cc:2272] T 6247a65da5884caba020ed13be64352f P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.658771 31967 leader_election.cc:336] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:03.649569   689 leader_election.cc:290] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 election: Requested vote from peers 32690b3987094d51bb4f214321a29067 (127.21.33.4:34551), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
I20260709 13:17:03.662626 21636 tablet_replica.cc:333] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.664316 21636 raft_consensus.cc:2243] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.666221 32074 raft_consensus.cc:2468] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccf4710690c7473d814836da1babee51 in term 4.
I20260709 13:17:03.667050 21636 raft_consensus.cc:2272] T 62477156fe044dad9635981556678bef P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.668121 31969 leader_election.cc:304] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32690b3987094d51bb4f214321a29067, ccf4710690c7473d814836da1babee51; no voters: 103d6e4c312a47ff9b258c14ac2871e5
I20260709 13:17:03.669689   689 raft_consensus.cc:2804] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:03.670300   689 raft_consensus.cc:697] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 4 LEADER]: Becoming Leader. State: Replica: ccf4710690c7473d814836da1babee51, State: Running, Role: LEADER
I20260709 13:17:03.672003 21636 tablet_replica.cc:333] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.672047   689 consensus_queue.cc:237] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:03.673348 21636 raft_consensus.cc:2243] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.676608 21636 raft_consensus.cc:2272] T 125eaf8f67bd4727b6d49528c95ca751 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.681598 21636 tablet_replica.cc:333] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.682704 21636 raft_consensus.cc:2243] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.683419 21636 raft_consensus.cc:2272] T 77e32837dc3e4e7d99a94d477a4d50d1 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.686599 21636 tablet_replica.cc:333] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.687412 21636 raft_consensus.cc:2243] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.688105 21636 raft_consensus.cc:2272] T a6aaf6fc3e504bc79c75c64bf99d89e7 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.691450 21636 tablet_replica.cc:333] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.692407 21636 raft_consensus.cc:2243] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.694021 21636 raft_consensus.cc:2272] T ea7811f2bbfe41aea0eba9a542d7dee8 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.698208 21636 tablet_replica.cc:333] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.699056 21636 raft_consensus.cc:2243] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.701017 21636 raft_consensus.cc:2272] T 58ea2a2bfc574918bd4f2bf7f2839e9c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.705319 21636 tablet_replica.cc:333] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.707273 21636 raft_consensus.cc:2243] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.708387 21636 raft_consensus.cc:2272] T 4df0e0908b634f91a377721d9f0e3ef7 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.713865 21636 tablet_replica.cc:333] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.715013 21636 raft_consensus.cc:2243] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.716017 21636 raft_consensus.cc:2272] T 4b0b712922784fd8a86a1c486c9d372e P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.719089 32044 consensus_peers.cc:597] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.720863 21636 tablet_replica.cc:333] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.722460 21636 raft_consensus.cc:2243] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.723642 21636 raft_consensus.cc:2272] T 0c4c038e4f034907a5fcaf6ab632a887 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.728557 21636 tablet_replica.cc:333] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: stopping tablet replica
W20260709 13:17:03.729272 32044 consensus_peers.cc:597] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 -> Peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Couldn't send request to peer ccf4710690c7473d814836da1babee51. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.730158 21636 raft_consensus.cc:2243] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.731303 21636 pending_rounds.cc:70] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: Trying to abort 1 pending ops.
I20260709 13:17:03.731616 21636 pending_rounds.cc:77] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305637984974499840 op_type: NO_OP noop_request { }
I20260709 13:17:03.732010 21636 raft_consensus.cc:2889] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.732391 21636 raft_consensus.cc:2272] T 221d9ca0b1f9450ead6ef9e225fe17ed P ccf4710690c7473d814836da1babee51 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.737074 21636 tablet_replica.cc:333] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.738234 21636 raft_consensus.cc:2243] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.739130 21636 raft_consensus.cc:2272] T 79ebfc33b37b45f69df96a3f4bb8f422 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.743717 21636 tablet_replica.cc:333] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.745007 21636 raft_consensus.cc:2243] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.748461 21636 raft_consensus.cc:2272] T 604a6aa17b764d61b1403b11157ddbe7 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.753682 21636 tablet_replica.cc:333] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.755287 21636 raft_consensus.cc:2243] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.756652 21636 raft_consensus.cc:2272] T 91b9568c210d4ef387297f17cde15389 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.761471 21636 tablet_replica.cc:333] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.762647 21636 raft_consensus.cc:2243] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.769119 21636 raft_consensus.cc:2272] T 62e37d9df96544f08314921b5ba47624 P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.777289 21636 tablet_replica.cc:333] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.778486 21636 raft_consensus.cc:2243] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.779807 21636 raft_consensus.cc:2272] T 340107361dee4b3997a43799863f36e8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.784137 21636 tablet_replica.cc:333] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.785257 21636 raft_consensus.cc:2243] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.787943 21636 raft_consensus.cc:2272] T 14858ce2d22c4dba9d14295ca6c2b23c P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.793408 21636 tablet_replica.cc:333] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.794508 21636 raft_consensus.cc:2243] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.795434 21636 raft_consensus.cc:2272] T 716630c6a4e249aeb96ef3ac276290c8 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.800125 21636 tablet_replica.cc:333] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.801059 21636 raft_consensus.cc:2243] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.801867 21636 raft_consensus.cc:2272] T 039ecd5ff518405fb0fc382c46e7e7c4 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.805606 21636 tablet_replica.cc:333] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.806582 21636 raft_consensus.cc:2243] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.807551 21636 raft_consensus.cc:2272] T 7dd8ceaec471453d9530212df301bc15 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.812441 21636 tablet_replica.cc:333] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.813354 21636 raft_consensus.cc:2243] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.815043 21636 raft_consensus.cc:2272] T ab0ae780de5843cea623c39fd867cc57 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.819403 21636 tablet_replica.cc:333] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.820278 21636 raft_consensus.cc:2243] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.821877 21636 raft_consensus.cc:2272] T 2611ce5875fe4e55a870d70749bd1ec4 P ccf4710690c7473d814836da1babee51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.826310 21636 tablet_replica.cc:333] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.827104 21636 raft_consensus.cc:2243] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:03.828760 21636 raft_consensus.cc:2272] T 8ad8727419f947cc957c6638ca5fd93a P ccf4710690c7473d814836da1babee51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.832655 21636 tablet_replica.cc:333] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51: stopping tablet replica
I20260709 13:17:03.833395 21636 raft_consensus.cc:2243] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.834226 21636 raft_consensus.cc:2272] T be826572f20c45fe84142ea4f204ba26 P ccf4710690c7473d814836da1babee51 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.834729   570 raft_consensus.cc:493] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:03.835296   570 raft_consensus.cc:515] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:03.837779   570 leader_election.cc:290] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:03.845767 32041 leader_election.cc:336] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:03.847270 32042 leader_election.cc:336] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.847810 32042 leader_election.cc:304] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.849684   570 raft_consensus.cc:2749] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:03.914163   658 raft_consensus.cc:493] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:03.915500   658 raft_consensus.cc:515] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:17:03.919188   658 leader_election.cc:290] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:17:03.927060 32044 leader_election.cc:336] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:03.939452 32042 leader_election.cc:336] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.939985 32042 leader_election.cc:304] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:17:03.941216   658 raft_consensus.cc:2749] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:03.961776   558 raft_consensus.cc:493] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:03.962862   558 raft_consensus.cc:515] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:03.965185   558 leader_election.cc:290] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:03.973019 32041 leader_election.cc:336] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:03.973639 32042 leader_election.cc:336] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:03.974149 32042 leader_election.cc:304] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:03.975003   558 raft_consensus.cc:2749] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:04.094749   570 raft_consensus.cc:493] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.095332   570 raft_consensus.cc:515] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:04.097489   570 leader_election.cc:290] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:04.102962 32042 leader_election.cc:336] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
W20260709 13:17:04.103928 32041 leader_election.cc:336] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:04.105247 32041 leader_election.cc:304] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:04.105919   570 raft_consensus.cc:2749] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:04.146495   699 raft_consensus.cc:493] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.147265   699 raft_consensus.cc:515] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } }
I20260709 13:17:04.149963   699 leader_election.cc:290] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847)
W20260709 13:17:04.155594 32041 leader_election.cc:336] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:04.156195 32042 leader_election.cc:336] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:04.156687 32042 leader_election.cc:304] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:04.157415   699 raft_consensus.cc:2749] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:04.219758   699 raft_consensus.cc:493] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.220361   699 raft_consensus.cc:515] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:17:04.222402   699 leader_election.cc:290] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:17:04.226529 32042 leader_election.cc:336] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
W20260709 13:17:04.227074 32044 leader_election.cc:336] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Network error: Client connection negotiation failed: client connection to 127.21.33.3:38635: connect: Connection refused (error 111)
I20260709 13:17:04.227440 32044 leader_election.cc:304] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:17:04.228112   699 raft_consensus.cc:2749] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:04.263543   558 raft_consensus.cc:493] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.264262   558 raft_consensus.cc:515] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:04.266469   558 leader_election.cc:290] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
W20260709 13:17:04.270295 32041 leader_election.cc:336] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
W20260709 13:17:04.271045 32042 leader_election.cc:336] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:04.271428 32042 leader_election.cc:304] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:04.272022   558 raft_consensus.cc:2749] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:04.300346   558 raft_consensus.cc:493] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.301008   558 raft_consensus.cc:515] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:17:04.302071 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:17:04.303331   558 leader_election.cc:290] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
W20260709 13:17:04.309008 32044 leader_election.cc:336] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Network error: Client connection negotiation failed: client connection to 127.21.33.3:38635: connect: Connection refused (error 111)
W20260709 13:17:04.309957 32042 leader_election.cc:336] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:04.310516 32042 leader_election.cc:304] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:17:04.311885   558 raft_consensus.cc:2749] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:04.322885 32041 consensus_peers.cc:597] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:04.400836 32041 consensus_peers.cc:597] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:04.401396 32041 consensus_peers.cc:597] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:04.413758 21636 tablet_server.cc:179] TabletServer@127.21.33.4:0 shutting down...
W20260709 13:17:04.428714 32041 consensus_peers.cc:597] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:04.466987 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:04.467975 21636 tablet_replica.cc:333] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.468885 21636 raft_consensus.cc:2243] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.469651 21636 raft_consensus.cc:2272] T ea7811f2bbfe41aea0eba9a542d7dee8 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.472157   558 raft_consensus.cc:493] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.472653 21636 tablet_replica.cc:333] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.472685   558 raft_consensus.cc:515] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:17:04.473649 21636 raft_consensus.cc:2243] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.474268 21636 raft_consensus.cc:2272] T 34769db9ad7a46a8b075b865d2a2b6ea P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.475126   558 leader_election.cc:290] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:17:04.478204 21636 tablet_replica.cc:333] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.478359   699 raft_consensus.cc:493] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.478219   558 raft_consensus.cc:493] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a588481f03f6474d9009d5bd5e08e887)
I20260709 13:17:04.479802 21636 raft_consensus.cc:2243] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.479597   699 raft_consensus.cc:515] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } } peers { permanent_uuid: "103d6e4c312a47ff9b258c14ac2871e5" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 35847 } } peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } }
I20260709 13:17:04.480674 21636 raft_consensus.cc:2272] T 87842118d5214e359e7983d9ede2213b P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:04.481175 32042 leader_election.cc:336] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
W20260709 13:17:04.481750 32044 leader_election.cc:336] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Network error: Client connection negotiation failed: client connection to 127.21.33.3:38635: connect: Connection refused (error 111)
I20260709 13:17:04.479748   558 raft_consensus.cc:515] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a588481f03f6474d9009d5bd5e08e887" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 37703 } } peers { permanent_uuid: "ccf4710690c7473d814836da1babee51" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 38635 } } peers { permanent_uuid: "32690b3987094d51bb4f214321a29067" member_type: VOTER last_known_addr { host: "127.21.33.4" port: 34551 } }
I20260709 13:17:04.482388 32044 leader_election.cc:304] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
I20260709 13:17:04.483409   570 raft_consensus.cc:2749] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:04.483668   699 leader_election.cc:290] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847), a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703)
I20260709 13:17:04.484649 21636 tablet_replica.cc:333] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.484465   558 leader_election.cc:290] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703), ccf4710690c7473d814836da1babee51 (127.21.33.3:38635)
I20260709 13:17:04.485821 21636 raft_consensus.cc:2243] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.486574 21636 raft_consensus.cc:2272] T 142b461239534bae915d26581eb87d21 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.491047 21636 tablet_replica.cc:333] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
W20260709 13:17:04.491297 32044 leader_election.cc:336] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ccf4710690c7473d814836da1babee51 (127.21.33.3:38635): Network error: Client connection negotiation failed: client connection to 127.21.33.3:38635: connect: Connection refused (error 111)
I20260709 13:17:04.492036 21636 raft_consensus.cc:2243] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:17:04.492455 32042 leader_election.cc:336] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
W20260709 13:17:04.492936 32042 leader_election.cc:336] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a588481f03f6474d9009d5bd5e08e887 (127.21.33.2:37703): Network error: Client connection negotiation failed: client connection to 127.21.33.2:37703: connect: Connection refused (error 111)
I20260709 13:17:04.493204 21636 raft_consensus.cc:2272] T 7a5b26df57d6488798ed73660ded04a5 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.493230 32042 leader_election.cc:304] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: a588481f03f6474d9009d5bd5e08e887, ccf4710690c7473d814836da1babee51
W20260709 13:17:04.494280 32041 leader_election.cc:336] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111)
I20260709 13:17:04.494422   558 raft_consensus.cc:2749] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:04.494750 32041 leader_election.cc:304] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32690b3987094d51bb4f214321a29067; no voters: 103d6e4c312a47ff9b258c14ac2871e5, a588481f03f6474d9009d5bd5e08e887
I20260709 13:17:04.495664   558 raft_consensus.cc:2749] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:04.495796 32041 consensus_peers.cc:597] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:04.496865 21636 tablet_replica.cc:333] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.497730 21636 raft_consensus.cc:2243] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.498680 21636 raft_consensus.cc:2272] T d7340289c3434c3782e3e367d4de5258 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.501808 21636 tablet_replica.cc:333] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.502840 21636 raft_consensus.cc:2243] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.503515 21636 raft_consensus.cc:2272] T 3474410429d7478d8ba2221c7400cb01 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.506073 21636 tablet_replica.cc:333] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.506721 21636 raft_consensus.cc:2243] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.507643 21636 raft_consensus.cc:2272] T c85fd3d52a714f44a3e322f4488f1a5c P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.510012 21636 tablet_replica.cc:333] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.510654 21636 raft_consensus.cc:2243] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.511224 21636 raft_consensus.cc:2272] T 4c11d6a5d0564e90b9ceda978d9225cb P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.513417 21636 tablet_replica.cc:333] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.514148 21636 raft_consensus.cc:2243] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.514600 21636 raft_consensus.cc:2272] T b90a6b486bef42019735b76ff4da04a1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.516924 21636 tablet_replica.cc:333] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.517588 21636 raft_consensus.cc:2243] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.518481 21636 raft_consensus.cc:2272] T 2b97d68a6e314f6792e4c8bfa99d3025 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.520704 21636 tablet_replica.cc:333] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.521332 21636 raft_consensus.cc:2243] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.522344 21636 raft_consensus.cc:2272] T 809ff13e62914dd7854fa320a1173da0 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.524605 21636 tablet_replica.cc:333] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.525285 21636 raft_consensus.cc:2243] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.528033 21636 raft_consensus.cc:2272] T 1fc5e72ae63b4c96bc88199a467670e2 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.530642 21636 tablet_replica.cc:333] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.531293 21636 raft_consensus.cc:2243] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.532582 21636 raft_consensus.cc:2272] T 658f92ce9d684ffbbec92446706f44c9 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.535488 21636 tablet_replica.cc:333] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.536239 21636 raft_consensus.cc:2243] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.536775 21636 raft_consensus.cc:2272] T 75f88110f6d141dc97dbef5c1c4f8391 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.539636 21636 tablet_replica.cc:333] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.540365 21636 raft_consensus.cc:2243] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.541374 21636 raft_consensus.cc:2272] T 372dd26a25ab4cf686e6c74fcc9b6bb6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.544036 21636 tablet_replica.cc:333] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.544732 21636 raft_consensus.cc:2243] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.545336 21636 raft_consensus.cc:2272] T d3127b63807542e6bb6a2a33677baabc P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.547850 21636 tablet_replica.cc:333] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.548527 21636 raft_consensus.cc:2243] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.549165 21636 raft_consensus.cc:2272] T 62477156fe044dad9635981556678bef P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.551676 21636 tablet_replica.cc:333] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.552335 21636 raft_consensus.cc:2243] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.553457 21636 raft_consensus.cc:2272] T 77e32837dc3e4e7d99a94d477a4d50d1 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.555806 21636 tablet_replica.cc:333] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.556468 21636 raft_consensus.cc:2243] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.556983 21636 raft_consensus.cc:2272] T 14858ce2d22c4dba9d14295ca6c2b23c P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.559252 21636 tablet_replica.cc:333] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.559955 21636 raft_consensus.cc:2243] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.560573 21636 raft_consensus.cc:2272] T 716630c6a4e249aeb96ef3ac276290c8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.564682 21636 tablet_replica.cc:333] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.565531 21636 raft_consensus.cc:2243] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.566392 21636 raft_consensus.cc:2272] T 039ecd5ff518405fb0fc382c46e7e7c4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:04.568902 32041 consensus_peers.cc:597] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:04.570426 21636 tablet_replica.cc:333] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.571552 21636 raft_consensus.cc:2243] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.572316 21636 raft_consensus.cc:2272] T 4df0e0908b634f91a377721d9f0e3ef7 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.575145 21636 tablet_replica.cc:333] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.575942 21636 raft_consensus.cc:2243] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.576990 21636 raft_consensus.cc:2272] T 4b0b712922784fd8a86a1c486c9d372e P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.579723 21636 tablet_replica.cc:333] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.580361 21636 raft_consensus.cc:2243] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.581108 21636 raft_consensus.cc:2272] T 221d9ca0b1f9450ead6ef9e225fe17ed P 32690b3987094d51bb4f214321a29067 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.583511 21636 tablet_replica.cc:333] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.584120 21636 raft_consensus.cc:2243] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.584654 21636 raft_consensus.cc:2272] T 604a6aa17b764d61b1403b11157ddbe7 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.587119 21636 tablet_replica.cc:333] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.587733 21636 raft_consensus.cc:2243] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.588222 21636 raft_consensus.cc:2272] T 7460aac4590c4910abca0a74a4863b5d P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.590646 21636 tablet_replica.cc:333] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.591301 21636 raft_consensus.cc:2243] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.592195 21636 raft_consensus.cc:2272] T 91b9568c210d4ef387297f17cde15389 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.594414 21636 tablet_replica.cc:333] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.595005 21636 raft_consensus.cc:2243] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.595498 21636 raft_consensus.cc:2272] T 340107361dee4b3997a43799863f36e8 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.598078 21636 tablet_replica.cc:333] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.598685 21636 raft_consensus.cc:2243] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.599205 21636 raft_consensus.cc:2272] T 1b4e043220524dfa8c9106bfe7690bcd P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.602114 21636 tablet_replica.cc:333] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.602869 21636 raft_consensus.cc:2243] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.603861 21636 raft_consensus.cc:2272] T a6aaf6fc3e504bc79c75c64bf99d89e7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.606029 21636 tablet_replica.cc:333] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.606592 21636 raft_consensus.cc:2243] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.607120 21636 raft_consensus.cc:2272] T fb6e2830158d41d3bb09aeaad85a5f84 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.609359 21636 tablet_replica.cc:333] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.610010 21636 raft_consensus.cc:2243] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.610498 21636 raft_consensus.cc:2272] T 56a207058f154c27b443ceacc88454fe P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.613095 21636 tablet_replica.cc:333] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.613822 21636 raft_consensus.cc:2243] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.614331 21636 raft_consensus.cc:2272] T 8e16bbde8ad04886af986347646976b7 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.617734 21636 tablet_replica.cc:333] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.618626 21636 raft_consensus.cc:2243] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.619235 21636 raft_consensus.cc:2272] T 395cb97d328c4ccc9cb0b96fe5d83dd6 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.622468 21636 tablet_replica.cc:333] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.623337 21636 raft_consensus.cc:2243] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.624118 21636 raft_consensus.cc:2272] T 8ad8727419f947cc957c6638ca5fd93a P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.627475 21636 tablet_replica.cc:333] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.628324 21636 raft_consensus.cc:2243] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.629846 21636 raft_consensus.cc:2272] T bfc3d708d55c46bcb69cfb239324f375 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.633351 21636 tablet_replica.cc:333] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.634231 21636 raft_consensus.cc:2243] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.635529 21636 raft_consensus.cc:2272] T a2ce179ed2a94df48e1b5f00fec58804 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.638942 21636 tablet_replica.cc:333] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.639776 21636 raft_consensus.cc:2243] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.640409 21636 raft_consensus.cc:2272] T 4dfe802c323741a9b432e1eca10b2698 P 32690b3987094d51bb4f214321a29067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.643918 21636 tablet_replica.cc:333] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.644727 21636 raft_consensus.cc:2243] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.645349 21636 raft_consensus.cc:2272] T 441576026a7f44f990e9aa61c86435d6 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.648706 21636 tablet_replica.cc:333] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.649461 21636 raft_consensus.cc:2243] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.650136 21636 raft_consensus.cc:2272] T 2611ce5875fe4e55a870d70749bd1ec4 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.653425 21636 tablet_replica.cc:333] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.654240 21636 raft_consensus.cc:2243] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:04.654534 32041 consensus_peers.cc:597] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 -> Peer 103d6e4c312a47ff9b258c14ac2871e5 (127.21.33.1:35847): Couldn't send request to peer 103d6e4c312a47ff9b258c14ac2871e5. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:35847: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:04.654865 21636 raft_consensus.cc:2272] T ab0ae780de5843cea623c39fd867cc57 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.658291 21636 tablet_replica.cc:333] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.659013 21636 raft_consensus.cc:2243] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:04.659653 21636 raft_consensus.cc:2272] T 973ae55c06694504ba369b35b010b993 P 32690b3987094d51bb4f214321a29067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.662801 21636 tablet_replica.cc:333] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067: stopping tablet replica
I20260709 13:17:04.663436 21636 raft_consensus.cc:2243] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.664613 21636 raft_consensus.cc:2272] T be826572f20c45fe84142ea4f204ba26 P 32690b3987094d51bb4f214321a29067 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.761783 21636 tablet_server.cc:196] TabletServer@127.21.33.4:0 shutdown complete.
I20260709 13:17:04.842242 21636 master.cc:562] Master@127.21.33.62:43497 shutting down...
I20260709 13:17:04.882033 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:04.882781 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:04.883141 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5f865afa026547d5aa8394713f4c8ddf: stopping tablet replica
I20260709 13:17:04.908759 21636 master.cc:584] Master@127.21.33.62:43497 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (47601 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:17:05.018360 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:34643
I20260709 13:17:05.019671 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:05.026211   701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.026621   702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.026714   704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:05.028443 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:05.029909 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:05.030102 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:05.030212 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603025030201 us; error 0 us; skew 500 ppm
I20260709 13:17:05.030694 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:05.033885 21636 webserver.cc:533] Webserver started at http://127.21.33.62:35519/ using document root <none> and password file <none>
I20260709 13:17:05.034406 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:05.034561 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:05.034802 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:05.036095 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "77b1bea634234737a3ee2938c35732de"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.040876 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:05.044606   709 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.045439 21636 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:05.045749 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "77b1bea634234737a3ee2938c35732de"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.046010 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:05.072978 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:05.074486 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:05.121958 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:34643
I20260709 13:17:05.122092   760 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:34643 every 8 connection(s)
I20260709 13:17:05.126591   761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.140092   761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de: Bootstrap starting.
I20260709 13:17:05.146211   761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:05.151986   761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de: No bootstrap required, opened a new log
I20260709 13:17:05.154718   761 raft_consensus.cc:359] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b1bea634234737a3ee2938c35732de" member_type: VOTER }
I20260709 13:17:05.155206   761 raft_consensus.cc:385] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:05.155470   761 raft_consensus.cc:740] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b1bea634234737a3ee2938c35732de, State: Initialized, Role: FOLLOWER
I20260709 13:17:05.156117   761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b1bea634234737a3ee2938c35732de" member_type: VOTER }
I20260709 13:17:05.156644   761 raft_consensus.cc:399] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:05.156898   761 raft_consensus.cc:493] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:05.157210   761 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:05.163553   761 raft_consensus.cc:515] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b1bea634234737a3ee2938c35732de" member_type: VOTER }
I20260709 13:17:05.164165   761 leader_election.cc:304] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77b1bea634234737a3ee2938c35732de; no voters: 
I20260709 13:17:05.165566   761 leader_election.cc:290] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:05.165953   764 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:05.167435   764 raft_consensus.cc:697] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 1 LEADER]: Becoming Leader. State: Replica: 77b1bea634234737a3ee2938c35732de, State: Running, Role: LEADER
I20260709 13:17:05.168171   764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b1bea634234737a3ee2938c35732de" member_type: VOTER }
I20260709 13:17:05.169088   761 sys_catalog.cc:565] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:05.175813   766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77b1bea634234737a3ee2938c35732de. Latest consensus state: current_term: 1 leader_uuid: "77b1bea634234737a3ee2938c35732de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b1bea634234737a3ee2938c35732de" member_type: VOTER } }
I20260709 13:17:05.176728   766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:05.178318   765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77b1bea634234737a3ee2938c35732de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b1bea634234737a3ee2938c35732de" member_type: VOTER } }
I20260709 13:17:05.179054   765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:05.183768   770 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:05.193452   770 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:05.195118 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:05.204571   770 catalog_manager.cc:1383] Generated new cluster ID: 87c63139951f4696a33124a8a8f74c04
I20260709 13:17:05.204965   770 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:05.243395   770 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:05.245034   770 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:05.258550   770 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de: Generated new TSK 0
I20260709 13:17:05.259281   770 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:05.326705 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:05.334906   782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.335865   783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:05.341637 21636 server_base.cc:1061] running on GCE node
W20260709 13:17:05.342928   785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:05.344064 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:05.344264 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:05.344439 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603025344421 us; error 0 us; skew 500 ppm
I20260709 13:17:05.345110 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:05.348539 21636 webserver.cc:533] Webserver started at http://127.21.33.1:38945/ using document root <none> and password file <none>
I20260709 13:17:05.349076 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:05.349260 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:05.349481 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:05.350736 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "f30dd7659dd84ca7bb677ab9804924af"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.356079 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:17:05.360350   790 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.361296 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:17:05.361691 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "f30dd7659dd84ca7bb677ab9804924af"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.362020 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:05.376454 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:05.378118 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:05.379853 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:05.382539 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:05.382746 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.382939 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:05.383118 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.431770 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:46789
I20260709 13:17:05.431906   852 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:46789 every 8 connection(s)
I20260709 13:17:05.437966 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:05.446190   857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.447530   858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.449313   860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:05.451175 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:05.453029 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:05.453302 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:05.453552 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603025453506 us; error 0 us; skew 500 ppm
I20260709 13:17:05.454412 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:05.457111 21636 webserver.cc:533] Webserver started at http://127.21.33.2:39019/ using document root <none> and password file <none>
I20260709 13:17:05.457734 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:05.457957 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:05.458257 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:05.459640 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "ccbdbc0a613a499e831042aca13d367e"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.461318   853 heartbeater.cc:344] Connected to a master server at 127.21.33.62:34643
I20260709 13:17:05.461870   853 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:05.462651   853 heartbeater.cc:507] Master 127.21.33.62:34643 requested a full tablet report, sending...
I20260709 13:17:05.465085   726 ts_manager.cc:194] Registered new tserver with Master: f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789)
I20260709 13:17:05.465131 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:17:05.467226   726 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38610
I20260709 13:17:05.469837   865 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.470865 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:05.471248 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "ccbdbc0a613a499e831042aca13d367e"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.471557 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:05.486485 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:05.488034 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:05.489784 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:05.492503 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:05.492780 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.493054 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:05.493250 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.545774 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:41721
I20260709 13:17:05.545894   927 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:41721 every 8 connection(s)
I20260709 13:17:05.550943 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:05.560385   931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.561522   932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:05.564457   934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:05.564671 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:05.566167 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:05.566448 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:05.566694 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603025566669 us; error 0 us; skew 500 ppm
I20260709 13:17:05.567445   928 heartbeater.cc:344] Connected to a master server at 127.21.33.62:34643
I20260709 13:17:05.567525 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:05.568040   928 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:05.568933   928 heartbeater.cc:507] Master 127.21.33.62:34643 requested a full tablet report, sending...
I20260709 13:17:05.571188 21636 webserver.cc:533] Webserver started at http://127.21.33.3:39097/ using document root <none> and password file <none>
I20260709 13:17:05.571883 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:05.571805   726 ts_manager.cc:194] Registered new tserver with Master: ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:05.572261 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:05.572651 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:05.573861   726 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38620
I20260709 13:17:05.574183 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.579722 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:17:05.583824   939 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.584859 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:17:05.585201 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:05.585628 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:05.614295 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:05.615825 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:05.617647 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:05.620240 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:05.620493 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.620751 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:05.620939 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:05.671604 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:42819
I20260709 13:17:05.671711  1001 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:42819 every 8 connection(s)
I20260709 13:17:05.689116  1002 heartbeater.cc:344] Connected to a master server at 127.21.33.62:34643
I20260709 13:17:05.689651  1002 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:05.690497  1002 heartbeater.cc:507] Master 127.21.33.62:34643 requested a full tablet report, sending...
I20260709 13:17:05.692842   726 ts_manager.cc:194] Registered new tserver with Master: 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:05.693182 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017669694s
I20260709 13:17:05.694904   726 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38626
I20260709 13:17:05.723070   726 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38630:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:05.726402   726 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:05.817287   893 tablet_service.cc:1511] Processing CreateTablet for tablet 8c1a020d7eea4c1aaf57aaef1f7a2242 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:05.819129   893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c1a020d7eea4c1aaf57aaef1f7a2242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.823939   892 tablet_service.cc:1511] Processing CreateTablet for tablet d5dbb685cfd941618e5faf493d1824d7 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:05.825820   892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5dbb685cfd941618e5faf493d1824d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.826818   891 tablet_service.cc:1511] Processing CreateTablet for tablet 20e4db9ddb954998a5233fe1682e94cb (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:05.828512   891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20e4db9ddb954998a5233fe1682e94cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.833930   967 tablet_service.cc:1511] Processing CreateTablet for tablet 8c1a020d7eea4c1aaf57aaef1f7a2242 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:05.835558   967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c1a020d7eea4c1aaf57aaef1f7a2242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.836802   890 tablet_service.cc:1511] Processing CreateTablet for tablet 26359eb1650e41ee99a2f5e88a6d3a20 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:05.838371   890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26359eb1650e41ee99a2f5e88a6d3a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.850683   889 tablet_service.cc:1511] Processing CreateTablet for tablet 1734ca1fafaa47f28f09883752366ef3 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:05.852735   889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1734ca1fafaa47f28f09883752366ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.855463   888 tablet_service.cc:1511] Processing CreateTablet for tablet 68dfa173666343c7acbc683137e46389 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:05.857189   888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dfa173666343c7acbc683137e46389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.861307   966 tablet_service.cc:1511] Processing CreateTablet for tablet d5dbb685cfd941618e5faf493d1824d7 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:05.863030   814 tablet_service.cc:1511] Processing CreateTablet for tablet 1734ca1fafaa47f28f09883752366ef3 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:05.864655   814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1734ca1fafaa47f28f09883752366ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.865043   965 tablet_service.cc:1511] Processing CreateTablet for tablet 20e4db9ddb954998a5233fe1682e94cb (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:05.865531   817 tablet_service.cc:1511] Processing CreateTablet for tablet d5dbb685cfd941618e5faf493d1824d7 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:05.867025   817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5dbb685cfd941618e5faf493d1824d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.866863   813 tablet_service.cc:1511] Processing CreateTablet for tablet 68dfa173666343c7acbc683137e46389 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:05.868355   813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dfa173666343c7acbc683137e46389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.868865   964 tablet_service.cc:1511] Processing CreateTablet for tablet 26359eb1650e41ee99a2f5e88a6d3a20 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:05.869527   816 tablet_service.cc:1511] Processing CreateTablet for tablet 20e4db9ddb954998a5233fe1682e94cb (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:05.871095   816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20e4db9ddb954998a5233fe1682e94cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.863077   966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5dbb685cfd941618e5faf493d1824d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.870838   812 tablet_service.cc:1511] Processing CreateTablet for tablet 38c50490ed8f45b394bba5138382e651 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:05.872023   965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20e4db9ddb954998a5233fe1682e94cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.872388   812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38c50490ed8f45b394bba5138382e651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.874169   964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26359eb1650e41ee99a2f5e88a6d3a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.874600   887 tablet_service.cc:1511] Processing CreateTablet for tablet a917d956d957469eacc1bacf690a53ff (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:05.876133   887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a917d956d957469eacc1bacf690a53ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.875905   818 tablet_service.cc:1511] Processing CreateTablet for tablet 8c1a020d7eea4c1aaf57aaef1f7a2242 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:05.873752   815 tablet_service.cc:1511] Processing CreateTablet for tablet 26359eb1650e41ee99a2f5e88a6d3a20 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:05.878257   818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c1a020d7eea4c1aaf57aaef1f7a2242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.878933   886 tablet_service.cc:1511] Processing CreateTablet for tablet fdc01e2a1c574e12a1288e9ce08da385 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:05.880501   886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc01e2a1c574e12a1288e9ce08da385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.881915   815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26359eb1650e41ee99a2f5e88a6d3a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.883548   885 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0c817782084112a28fa822071ccc71 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:05.884485   811 tablet_service.cc:1511] Processing CreateTablet for tablet a917d956d957469eacc1bacf690a53ff (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:05.885581   885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0c817782084112a28fa822071ccc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.887998   810 tablet_service.cc:1511] Processing CreateTablet for tablet fdc01e2a1c574e12a1288e9ce08da385 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:05.890611   810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc01e2a1c574e12a1288e9ce08da385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.891939   811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a917d956d957469eacc1bacf690a53ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.898310   963 tablet_service.cc:1511] Processing CreateTablet for tablet 1734ca1fafaa47f28f09883752366ef3 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:05.900187   963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1734ca1fafaa47f28f09883752366ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.903002   962 tablet_service.cc:1511] Processing CreateTablet for tablet 68dfa173666343c7acbc683137e46389 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:05.904743   962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68dfa173666343c7acbc683137e46389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.907850   961 tablet_service.cc:1511] Processing CreateTablet for tablet 38c50490ed8f45b394bba5138382e651 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:05.910933   960 tablet_service.cc:1511] Processing CreateTablet for tablet a917d956d957469eacc1bacf690a53ff (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:05.913976   959 tablet_service.cc:1511] Processing CreateTablet for tablet fdc01e2a1c574e12a1288e9ce08da385 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:05.915787   958 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0c817782084112a28fa822071ccc71 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:05.909416   961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38c50490ed8f45b394bba5138382e651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.903499   893 tablet_service.cc:1511] Processing CreateTablet for tablet 38c50490ed8f45b394bba5138382e651 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:05.918437   959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc01e2a1c574e12a1288e9ce08da385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.919256   960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a917d956d957469eacc1bacf690a53ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.919436   893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38c50490ed8f45b394bba5138382e651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.921839   958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0c817782084112a28fa822071ccc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.942426   809 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0c817782084112a28fa822071ccc71 (DEFAULT_TABLE table=test-workload [id=b1d9cec8edc5488eacf523c7d971ae43]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:05.944082   809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0c817782084112a28fa822071ccc71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:05.959211  1017 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:05.963040  1018 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:05.972072  1018 tablet_bootstrap.cc:654] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:05.980680  1017 tablet_bootstrap.cc:654] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.013454  1017 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.014700  1017 ts_tablet_manager.cc:1403] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.056s	user 0.009s	sys 0.013s
I20260709 13:17:06.077453  1021 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.087605  1021 tablet_bootstrap.cc:654] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.098905  1017 raft_consensus.cc:359] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.100054  1017 raft_consensus.cc:385] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.100435  1017 raft_consensus.cc:740] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.102229  1017 consensus_queue.cc:260] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.119740  1021 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.120467  1021 ts_tablet_manager.cc:1403] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.019s
I20260709 13:17:06.122610  1018 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.123286  1018 ts_tablet_manager.cc:1403] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.161s	user 0.054s	sys 0.070s
I20260709 13:17:06.124161  1021 raft_consensus.cc:359] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.124969  1021 raft_consensus.cc:385] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.125255  1021 raft_consensus.cc:740] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.126137  1021 consensus_queue.cc:260] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.126510  1018 raft_consensus.cc:359] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.127259  1018 raft_consensus.cc:385] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.127583  1018 raft_consensus.cc:740] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.128512  1018 consensus_queue.cc:260] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.171787  1018 ts_tablet_manager.cc:1434] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.048s	user 0.010s	sys 0.014s
I20260709 13:17:06.173285  1002 heartbeater.cc:499] Master 127.21.33.62:34643 was elected leader, sending a full tablet report...
I20260709 13:17:06.174005  1018 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
W20260709 13:17:06.176769  1003 tablet.cc:2406] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:06.182825  1017 ts_tablet_manager.cc:1434] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.168s	user 0.009s	sys 0.019s
I20260709 13:17:06.187651   928 heartbeater.cc:499] Master 127.21.33.62:34643 was elected leader, sending a full tablet report...
W20260709 13:17:06.189334   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:06.181434  1018 tablet_bootstrap.cc:654] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.194839  1017 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.200366  1021 ts_tablet_manager.cc:1434] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.080s	user 0.017s	sys 0.017s
I20260709 13:17:06.201469   853 heartbeater.cc:499] Master 127.21.33.62:34643 was elected leader, sending a full tablet report...
I20260709 13:17:06.201979  1021 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.210522  1021 tablet_bootstrap.cc:654] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.216732  1018 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.217422  1018 ts_tablet_manager.cc:1403] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.016s
I20260709 13:17:06.223055  1017 tablet_bootstrap.cc:654] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.222234  1018 raft_consensus.cc:359] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.223623  1018 raft_consensus.cc:385] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.224092  1018 raft_consensus.cc:740] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.225383  1018 consensus_queue.cc:260] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.228981  1018 ts_tablet_manager.cc:1434] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:17:06.230425  1018 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.240420  1018 tablet_bootstrap.cc:654] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.269321  1021 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.270179  1021 ts_tablet_manager.cc:1403] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.069s	user 0.026s	sys 0.034s
I20260709 13:17:06.271489  1018 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.272269  1018 ts_tablet_manager.cc:1403] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.019s
I20260709 13:17:06.272689  1017 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.273263  1017 ts_tablet_manager.cc:1403] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.079s	user 0.029s	sys 0.019s
I20260709 13:17:06.273932  1021 raft_consensus.cc:359] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.274837  1021 raft_consensus.cc:385] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.275190  1021 raft_consensus.cc:740] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.275729  1018 raft_consensus.cc:359] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.276649  1018 raft_consensus.cc:385] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.276036  1021 consensus_queue.cc:260] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.277040  1018 raft_consensus.cc:740] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.277141  1017 raft_consensus.cc:359] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.277922  1017 raft_consensus.cc:385] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.278245  1017 raft_consensus.cc:740] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.278509  1018 consensus_queue.cc:260] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.279244  1021 ts_tablet_manager.cc:1434] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:17:06.279268  1017 consensus_queue.cc:260] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.280692  1021 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.281801  1017 ts_tablet_manager.cc:1434] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:06.283205  1017 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.284891  1018 ts_tablet_manager.cc:1434] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:17:06.286293  1018 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.293674  1018 tablet_bootstrap.cc:654] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.294183  1017 tablet_bootstrap.cc:654] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.294660  1024 raft_consensus.cc:493] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.295316  1024 raft_consensus.cc:515] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.301625  1021 tablet_bootstrap.cc:654] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.306366  1024 leader_election.cc:290] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.317342  1023 raft_consensus.cc:493] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.318040  1023 raft_consensus.cc:515] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.319336  1021 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.320199  1021 ts_tablet_manager.cc:1403] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.010s
I20260709 13:17:06.323686  1023 leader_election.cc:290] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.330494  1021 raft_consensus.cc:359] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.331697  1021 raft_consensus.cc:385] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.332095  1021 raft_consensus.cc:740] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.333464  1021 consensus_queue.cc:260] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.338635  1021 ts_tablet_manager.cc:1434] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.006s
I20260709 13:17:06.340754  1025 raft_consensus.cc:493] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.341427  1025 raft_consensus.cc:515] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.347509  1021 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.351027  1018 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.351105   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.352578  1018 ts_tablet_manager.cc:1403] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.019s
W20260709 13:17:06.354135   869 leader_election.cc:343] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.357491  1017 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.359119  1025 leader_election.cc:290] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.361138  1024 raft_consensus.cc:493] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.365553  1017 ts_tablet_manager.cc:1403] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.083s	user 0.029s	sys 0.024s
I20260709 13:17:06.367352  1025 raft_consensus.cc:493] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.368244  1024 raft_consensus.cc:515] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.368497  1025 raft_consensus.cc:515] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.371627  1025 leader_election.cc:290] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.372480  1021 tablet_bootstrap.cc:654] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.380491  1024 leader_election.cc:290] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.390514  1017 raft_consensus.cc:359] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.391840  1017 raft_consensus.cc:385] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.392241  1017 raft_consensus.cc:740] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.395460  1018 raft_consensus.cc:359] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.396991  1018 raft_consensus.cc:385] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.402210  1017 consensus_queue.cc:260] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.411887   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.416473  1017 ts_tablet_manager.cc:1434] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 13:17:06.402454  1018 raft_consensus.cc:740] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.419275   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
W20260709 13:17:06.421264   793 leader_election.cc:343] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.418159   902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
W20260709 13:17:06.421286   868 leader_election.cc:343] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.417822  1017 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.423337   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.424015   828 raft_consensus.cc:2468] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 0.
I20260709 13:17:06.425205   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.425393   942 leader_election.cc:304] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.426038   827 raft_consensus.cc:2468] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 0.
I20260709 13:17:06.426431  1024 raft_consensus.cc:2804] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.427115  1024 raft_consensus.cc:493] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.427496  1024 raft_consensus.cc:3060] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.422698   868 leader_election.cc:304] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:06.429276  1023 raft_consensus.cc:2749] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:06.430718   793 leader_election.cc:343] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.418071  1018 consensus_queue.cc:260] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.435298  1017 tablet_bootstrap.cc:654] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.448387   942 leader_election.cc:304] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.453923  1024 raft_consensus.cc:515] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.454823  1018 ts_tablet_manager.cc:1434] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.102s	user 0.016s	sys 0.012s
I20260709 13:17:06.456449  1024 leader_election.cc:290] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.456704   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.456087  1045 raft_consensus.cc:2804] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.457175   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.457898  1018 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.458284   976 raft_consensus.cc:2393] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f30dd7659dd84ca7bb677ab9804924af in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:06.459038  1045 raft_consensus.cc:493] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.459441  1045 raft_consensus.cc:3060] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.462419   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:06.463148   827 raft_consensus.cc:3060] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.463022   902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.464706   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
W20260709 13:17:06.467056   942 leader_election.cc:343] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:06.468115   942 leader_election.cc:343] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.451306  1021 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.468537   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:06.469125  1021 ts_tablet_manager.cc:1403] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.122s	user 0.021s	sys 0.010s
I20260709 13:17:06.469393  1045 raft_consensus.cc:515] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.470671   794 leader_election.cc:304] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f30dd7659dd84ca7bb677ab9804924af; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.471663  1045 leader_election.cc:290] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.472436   977 raft_consensus.cc:2393] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f30dd7659dd84ca7bb677ab9804924af in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:06.472648  1021 raft_consensus.cc:359] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.473552  1021 raft_consensus.cc:385] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.473889  1021 raft_consensus.cc:740] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.475333   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
W20260709 13:17:06.475497   942 leader_election.cc:343] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.476019   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:06.476543   794 leader_election.cc:304] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f30dd7659dd84ca7bb677ab9804924af; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.476045   828 raft_consensus.cc:3060] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:06.477483   942 leader_election.cc:343] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.478849  1018 tablet_bootstrap.cc:654] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.474763  1021 consensus_queue.cc:260] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.485122   828 raft_consensus.cc:2468] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 1.
I20260709 13:17:06.485734  1025 raft_consensus.cc:2749] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:06.488477  1021 ts_tablet_manager.cc:1434] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 13:17:06.489969  1021 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.491266   942 leader_election.cc:304] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.491977  1018 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.492571  1018 ts_tablet_manager.cc:1403] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.003s
I20260709 13:17:06.492734  1045 raft_consensus.cc:2804] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:06.493378  1045 raft_consensus.cc:697] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:06.494592  1045 consensus_queue.cc:237] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.496433  1017 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.497048  1017 ts_tablet_manager.cc:1403] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.000s
I20260709 13:17:06.497917  1021 tablet_bootstrap.cc:654] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.500700  1017 raft_consensus.cc:359] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.501749  1017 raft_consensus.cc:385] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.502211  1017 raft_consensus.cc:740] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.503594  1017 consensus_queue.cc:260] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.511916  1017 ts_tablet_manager.cc:1434] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 13:17:06.515758  1025 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.516482  1025 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.517854  1018 raft_consensus.cc:359] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.519065  1018 raft_consensus.cc:385] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.519511  1018 raft_consensus.cc:740] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.520937  1025 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.522038   827 raft_consensus.cc:2468] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 1.
I20260709 13:17:06.523087  1040 raft_consensus.cc:2749] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:06.524228  1023 raft_consensus.cc:493] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.524132   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.525043   977 raft_consensus.cc:2468] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30dd7659dd84ca7bb677ab9804924af in term 0.
I20260709 13:17:06.524891  1023 raft_consensus.cc:515] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.526309   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.528028  1017 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
W20260709 13:17:06.528193   793 leader_election.cc:343] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.528797   942 leader_election.cc:304] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.529964  1044 raft_consensus.cc:2804] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:06.530791   794 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.531857  1048 raft_consensus.cc:2804] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.532230  1048 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.532548  1048 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.533860   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.534185   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68dfa173666343c7acbc683137e46389" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.534754  1023 leader_election.cc:290] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.535501   827 raft_consensus.cc:2393] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdbc0a613a499e831042aca13d367e in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:06.520669  1018 consensus_queue.cc:260] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.539795  1023 raft_consensus.cc:493] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.540728  1024 raft_consensus.cc:493] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.540683  1023 raft_consensus.cc:515] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.541471  1024 raft_consensus.cc:515] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.543190  1018 ts_tablet_manager.cc:1434] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.003s
I20260709 13:17:06.544559  1018 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.547194  1045 raft_consensus.cc:493] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.547468   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.547922  1045 raft_consensus.cc:515] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.548591   827 raft_consensus.cc:2468] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 0.
I20260709 13:17:06.550716  1045 leader_election.cc:290] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.550532   868 leader_election.cc:304] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.554958  1018 tablet_bootstrap.cc:654] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.555169   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26359eb1650e41ee99a2f5e88a6d3a20" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.556113   901 raft_consensus.cc:2468] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 0.
I20260709 13:17:06.560472   942 leader_election.cc:304] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: 
I20260709 13:17:06.561894  1023 leader_election.cc:290] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.558295   724 catalog_manager.cc:5697] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.562757  1023 raft_consensus.cc:2804] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.563122   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.563197  1023 raft_consensus.cc:493] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.563859  1023 raft_consensus.cc:3060] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.563930   976 raft_consensus.cc:2468] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 0.
I20260709 13:17:06.565218  1021 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.560806   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.558815  1044 raft_consensus.cc:697] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:06.561013  1024 leader_election.cc:290] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789)
I20260709 13:17:06.566459  1045 raft_consensus.cc:2804] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.568001  1045 raft_consensus.cc:493] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.568424  1045 raft_consensus.cc:3060] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.568369  1021 ts_tablet_manager.cc:1403] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.079s	user 0.024s	sys 0.027s
I20260709 13:17:06.568333  1044 consensus_queue.cc:237] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.570518   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26359eb1650e41ee99a2f5e88a6d3a20" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.573089   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
W20260709 13:17:06.573652   942 leader_election.cc:343] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.573870   828 raft_consensus.cc:2468] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 0.
I20260709 13:17:06.575493   942 leader_election.cc:304] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.577095  1051 raft_consensus.cc:2804] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.577653  1051 raft_consensus.cc:493] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.578078  1051 raft_consensus.cc:3060] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.581722  1048 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.586519   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
W20260709 13:17:06.588769   793 leader_election.cc:343] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:06.590303  1049 log.cc:927] Time spent T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:17:06.597783   869 leader_election.cc:304] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:06.599037  1052 raft_consensus.cc:3060] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.599449  1017 tablet_bootstrap.cc:654] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.605304  1048 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 election: Requested vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.607940  1018 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.608578  1018 ts_tablet_manager.cc:1403] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.016s
I20260709 13:17:06.608625  1051 raft_consensus.cc:515] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.610937   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:06.611616   977 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.613806   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:06.614032  1018 raft_consensus.cc:359] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.614857  1018 raft_consensus.cc:385] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.615255  1018 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.613492  1023 raft_consensus.cc:515] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.616183  1018 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.617285   901 raft_consensus.cc:2393] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0f89b8f359fc4d2a8165bf7bae901e33 in current term 1: Already voted for candidate ccbdbc0a613a499e831042aca13d367e in this term.
I20260709 13:17:06.615432  1021 raft_consensus.cc:359] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.621601  1021 raft_consensus.cc:385] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.622051  1021 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.622582  1023 leader_election.cc:290] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.620004  1018 ts_tablet_manager.cc:1434] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.006s
I20260709 13:17:06.622927  1021 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.623724   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:06.623886   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:06.624257  1018 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.624425   828 raft_consensus.cc:3060] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.624869   976 raft_consensus.cc:2393] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ccbdbc0a613a499e831042aca13d367e in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:06.627697   977 raft_consensus.cc:2468] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30dd7659dd84ca7bb677ab9804924af in term 1.
I20260709 13:17:06.603725  1045 raft_consensus.cc:515] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.632452  1045 leader_election.cc:290] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.632938  1018 tablet_bootstrap.cc:654] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.635018  1017 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.635703  1017 ts_tablet_manager.cc:1403] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.123s	user 0.022s	sys 0.000s
I20260709 13:17:06.636013  1052 raft_consensus.cc:2749] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:06.638559  1021 ts_tablet_manager.cc:1434] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.002s
I20260709 13:17:06.639851  1021 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.639776  1017 raft_consensus.cc:359] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.640916  1017 raft_consensus.cc:385] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.641345  1017 raft_consensus.cc:740] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.642550  1017 consensus_queue.cc:260] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.645454  1017 ts_tablet_manager.cc:1434] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:17:06.646682  1017 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.647686  1051 leader_election.cc:290] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789)
I20260709 13:17:06.651141   828 raft_consensus.cc:2468] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 1.
I20260709 13:17:06.651711  1018 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.652350  1018 ts_tablet_manager.cc:1403] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.014s
I20260709 13:17:06.652943   794 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.655570   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26359eb1650e41ee99a2f5e88a6d3a20" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:06.655923  1048 raft_consensus.cc:2804] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:06.656162  1017 tablet_bootstrap.cc:654] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.656661  1048 raft_consensus.cc:697] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 LEADER]: Becoming Leader. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Running, Role: LEADER
W20260709 13:17:06.657888   942 leader_election.cc:343] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.657853  1048 consensus_queue.cc:237] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.660354  1018 raft_consensus.cc:359] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.661302  1018 raft_consensus.cc:385] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.661716  1018 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.662072   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26359eb1650e41ee99a2f5e88a6d3a20" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:06.662999   901 raft_consensus.cc:3060] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.662750  1018 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.665302  1018 ts_tablet_manager.cc:1434] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.005s
I20260709 13:17:06.666406  1018 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.666867   868 leader_election.cc:304] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 0f89b8f359fc4d2a8165bf7bae901e33
I20260709 13:17:06.668399  1023 raft_consensus.cc:2804] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:06.673784  1017 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.674572  1017 ts_tablet_manager.cc:1403] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.012s
I20260709 13:17:06.678445  1017 raft_consensus.cc:359] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.679320  1017 raft_consensus.cc:385] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.679679  1017 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.684316  1018 tablet_bootstrap.cc:654] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.680655  1017 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.685092   725 catalog_manager.cc:5697] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.688710  1061 raft_consensus.cc:493] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.689363  1061 raft_consensus.cc:515] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.692997   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.693969   827 raft_consensus.cc:2393] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdbc0a613a499e831042aca13d367e in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:06.695675  1023 raft_consensus.cc:697] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:06.696812  1023 consensus_queue.cc:237] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.700496  1021 tablet_bootstrap.cc:654] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.701634   724 catalog_manager.cc:5697] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af reported cstate change: term changed from 0 to 1, leader changed from <none> to f30dd7659dd84ca7bb677ab9804924af (127.21.33.1). New cstate: current_term: 1 leader_uuid: "f30dd7659dd84ca7bb677ab9804924af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.705955  1061 leader_election.cc:290] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.706197   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.708182   869 leader_election.cc:304] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:06.709105  1061 raft_consensus.cc:3060] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.711148   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c1a020d7eea4c1aaf57aaef1f7a2242" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:06.712381   828 raft_consensus.cc:2393] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0f89b8f359fc4d2a8165bf7bae901e33 in current term 1: Already voted for candidate ccbdbc0a613a499e831042aca13d367e in this term.
I20260709 13:17:06.703490   901 raft_consensus.cc:2468] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 1.
I20260709 13:17:06.713766   942 leader_election.cc:304] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33; no voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:06.715302   942 leader_election.cc:304] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:06.716174  1045 raft_consensus.cc:2749] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:06.716572  1051 raft_consensus.cc:2804] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:06.717154  1051 raft_consensus.cc:697] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:06.717996  1063 raft_consensus.cc:493] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.718143  1051 consensus_queue.cc:237] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.718642  1063 raft_consensus.cc:515] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.718204   726 catalog_manager.cc:5697] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.722082   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.722843   828 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 0.
I20260709 13:17:06.723186   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.724200   868 leader_election.cc:304] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.724450   977 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 0.
I20260709 13:17:06.725241  1023 raft_consensus.cc:2804] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.725672  1023 raft_consensus.cc:493] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.726042  1023 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.726495  1063 leader_election.cc:290] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.735288  1023 raft_consensus.cc:515] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.738996   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:06.739768   828 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.740492  1017 ts_tablet_manager.cc:1434] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.066s	user 0.021s	sys 0.012s
I20260709 13:17:06.745775  1017 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.749943  1018 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.750751  1018 ts_tablet_manager.cc:1403] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.006s
I20260709 13:17:06.754081  1021 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.755010  1021 ts_tablet_manager.cc:1403] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.115s	user 0.017s	sys 0.020s
I20260709 13:17:06.755939  1061 raft_consensus.cc:2749] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:06.757331  1023 leader_election.cc:290] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.757491  1017 tablet_bootstrap.cc:654] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.760643   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:06.764555   977 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.764770   828 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 1.
I20260709 13:17:06.766250   868 leader_election.cc:304] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.767179  1023 raft_consensus.cc:2804] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:06.767866  1023 raft_consensus.cc:697] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:06.768883  1023 consensus_queue.cc:237] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.758823  1018 raft_consensus.cc:359] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.771107  1018 raft_consensus.cc:385] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.772884  1018 raft_consensus.cc:740] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.771552   725 catalog_manager.cc:5697] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.774386  1018 consensus_queue.cc:260] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.763407  1021 raft_consensus.cc:359] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.776497  1021 raft_consensus.cc:385] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.776926  1021 raft_consensus.cc:740] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.780589  1018 ts_tablet_manager.cc:1434] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 13:17:06.782346  1021 consensus_queue.cc:260] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.783198   977 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 1.
I20260709 13:17:06.784809  1018 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.799508  1021 ts_tablet_manager.cc:1434] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.044s	user 0.010s	sys 0.000s
I20260709 13:17:06.806514  1021 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.811125  1018 tablet_bootstrap.cc:654] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.813804  1021 tablet_bootstrap.cc:654] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.815162  1017 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.815706  1017 ts_tablet_manager.cc:1403] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.032s
I20260709 13:17:06.815233   726 catalog_manager.cc:5697] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.818866  1017 raft_consensus.cc:359] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.819474  1017 raft_consensus.cc:385] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.819773  1017 raft_consensus.cc:740] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.820513  1017 consensus_queue.cc:260] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.822785  1017 ts_tablet_manager.cc:1434] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:06.823889  1017 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.832305  1017 tablet_bootstrap.cc:654] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.835856  1021 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.835912  1018 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.836496  1021 ts_tablet_manager.cc:1403] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.014s
I20260709 13:17:06.836961  1018 ts_tablet_manager.cc:1403] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.003s
I20260709 13:17:06.840226  1021 raft_consensus.cc:359] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.841024  1021 raft_consensus.cc:385] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.841360  1021 raft_consensus.cc:740] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.842186  1021 consensus_queue.cc:260] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.841837  1018 raft_consensus.cc:359] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.843070  1018 raft_consensus.cc:385] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.843495  1018 raft_consensus.cc:740] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.844555  1021 ts_tablet_manager.cc:1434] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:17:06.844511  1018 consensus_queue.cc:260] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.845975  1021 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.847940  1018 ts_tablet_manager.cc:1434] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:17:06.849534  1018 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33: Bootstrap starting.
I20260709 13:17:06.850203  1048 raft_consensus.cc:493] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.850811  1048 raft_consensus.cc:515] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.855329   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26359eb1650e41ee99a2f5e88a6d3a20" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.856326   901 raft_consensus.cc:2393] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f30dd7659dd84ca7bb677ab9804924af in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:06.856894  1018 tablet_bootstrap.cc:654] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.856801  1048 leader_election.cc:290] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.855347   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26359eb1650e41ee99a2f5e88a6d3a20" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.859560  1017 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.860247  1021 tablet_bootstrap.cc:654] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.860085   794 leader_election.cc:304] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f30dd7659dd84ca7bb677ab9804924af; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.862814  1048 raft_consensus.cc:3060] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.860178  1017 ts_tablet_manager.cc:1403] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.004s
I20260709 13:17:06.866381  1017 raft_consensus.cc:359] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.867074  1017 raft_consensus.cc:385] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.867374  1017 raft_consensus.cc:740] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.868109  1017 consensus_queue.cc:260] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.870651  1017 ts_tablet_manager.cc:1434] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:06.871711  1021 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.871726  1018 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33: No bootstrap required, opened a new log
I20260709 13:17:06.872210  1017 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.872548  1018 ts_tablet_manager.cc:1403] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.009s
I20260709 13:17:06.872548  1021 ts_tablet_manager.cc:1403] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 13:17:06.876883  1021 raft_consensus.cc:359] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.877699  1021 raft_consensus.cc:385] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.877986  1021 raft_consensus.cc:740] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.878737  1021 consensus_queue.cc:260] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.880340  1017 tablet_bootstrap.cc:654] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.881084  1021 ts_tablet_manager.cc:1434] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:17:06.882362  1021 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.886500  1048 raft_consensus.cc:2749] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:06.887442  1051 consensus_queue.cc:1048] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:06.889169  1021 tablet_bootstrap.cc:654] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.894691  1018 raft_consensus.cc:359] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.895972  1018 raft_consensus.cc:385] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.896360  1018 raft_consensus.cc:740] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.897780  1018 consensus_queue.cc:260] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.906021  1018 ts_tablet_manager.cc:1434] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:17:06.914613  1048 raft_consensus.cc:493] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.917853  1048 raft_consensus.cc:515] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.922592  1048 leader_election.cc:290] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:06.930296   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:06.931224   977 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30dd7659dd84ca7bb677ab9804924af in term 0.
I20260709 13:17:06.932865   794 leader_election.cc:304] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:06.933609   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.935052   901 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30dd7659dd84ca7bb677ab9804924af in term 0.
I20260709 13:17:06.937616  1044 raft_consensus.cc:493] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.938508  1044 raft_consensus.cc:515] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.941380  1021 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
I20260709 13:17:06.942461  1021 ts_tablet_manager.cc:1403] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.021s
I20260709 13:17:06.942811  1017 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:06.943328  1017 ts_tablet_manager.cc:1403] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.071s	user 0.024s	sys 0.036s
I20260709 13:17:06.943926  1048 raft_consensus.cc:2804] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:06.944387  1048 raft_consensus.cc:493] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.945098  1048 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.946499  1017 raft_consensus.cc:359] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.947623  1017 raft_consensus.cc:385] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.947983  1017 raft_consensus.cc:740] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:06.947050  1021 raft_consensus.cc:359] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.948606  1044 leader_election.cc:290] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:06.949054  1017 consensus_queue.cc:260] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.951566   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:06.951836  1017 ts_tablet_manager.cc:1434] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:17:06.952958  1017 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: Bootstrap starting.
I20260709 13:17:06.948630  1021 raft_consensus.cc:385] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:06.957733  1021 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
W20260709 13:17:06.960843   942 leader_election.cc:343] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:06.961803  1021 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.963810  1021 ts_tablet_manager.cc:1434] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:17:06.968607   828 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:06.969399   828 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 0.
I20260709 13:17:06.974519  1021 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:06.976491   942 leader_election.cc:304] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:06.983253  1021 tablet_bootstrap.cc:654] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:06.994601  1048 raft_consensus.cc:515] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:06.999742   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:07.000463   901 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:06.989315  1044 raft_consensus.cc:2804] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:07.001349  1044 raft_consensus.cc:493] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.001853  1044 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.003543  1045 consensus_queue.cc:1048] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.009452  1017 tablet_bootstrap.cc:654] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:07.010150  1048 leader_election.cc:290] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 election: Requested vote from peers ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:07.015823   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "f30dd7659dd84ca7bb677ab9804924af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:07.016891   977 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.020993   901 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30dd7659dd84ca7bb677ab9804924af in term 1.
I20260709 13:17:07.023744   793 leader_election.cc:304] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:07.024788  1048 raft_consensus.cc:2804] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:07.025741  1048 raft_consensus.cc:697] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 LEADER]: Becoming Leader. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Running, Role: LEADER
I20260709 13:17:07.026917  1048 consensus_queue.cc:237] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.032045  1045 raft_consensus.cc:493] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.032372  1044 raft_consensus.cc:515] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.032712  1045 raft_consensus.cc:515] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.033963   977 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30dd7659dd84ca7bb677ab9804924af in term 1.
I20260709 13:17:07.035480  1045 leader_election.cc:290] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:07.036798  1044 leader_election.cc:290] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:07.037986   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:07.038345   826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:07.038672   902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:07.039002   826 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.040270   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
W20260709 13:17:07.040957   942 leader_election.cc:343] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:07.050500   942 leader_election.cc:343] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:07.041280   901 raft_consensus.cc:2468] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 0.
I20260709 13:17:07.056150   942 leader_election.cc:304] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:07.058602  1044 raft_consensus.cc:2804] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:07.059172  1044 raft_consensus.cc:493] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.059598  1044 raft_consensus.cc:3060] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.062590  1017 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: No bootstrap required, opened a new log
I20260709 13:17:07.066779  1017 ts_tablet_manager.cc:1403] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: Time spent bootstrapping tablet: real 0.114s	user 0.025s	sys 0.007s
I20260709 13:17:07.071130  1017 raft_consensus.cc:359] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.073755  1051 consensus_queue.cc:1048] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.074496  1017 raft_consensus.cc:385] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:07.075139  1044 raft_consensus.cc:515] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.075874  1017 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Initialized, Role: FOLLOWER
I20260709 13:17:07.077441  1017 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.093647  1051 consensus_queue.cc:1048] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.095191   826 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 1.
I20260709 13:17:07.097381  1044 leader_election.cc:290] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:07.102751   826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:07.103468   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:07.104220   903 raft_consensus.cc:3060] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.106456   942 leader_election.cc:304] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:07.108387  1044 raft_consensus.cc:2804] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:07.107129   723 catalog_manager.cc:5697] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af reported cstate change: term changed from 0 to 1, leader changed from <none> to f30dd7659dd84ca7bb677ab9804924af (127.21.33.1). New cstate: current_term: 1 leader_uuid: "f30dd7659dd84ca7bb677ab9804924af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.109045  1044 raft_consensus.cc:697] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:07.110245  1044 consensus_queue.cc:237] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.115423  1021 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: No bootstrap required, opened a new log
W20260709 13:17:07.116470   942 leader_election.cc:343] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:07.117278   903 raft_consensus.cc:2468] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 1.
I20260709 13:17:07.117360  1021 ts_tablet_manager.cc:1403] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.143s	user 0.019s	sys 0.006s
I20260709 13:17:07.126263  1021 raft_consensus.cc:359] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.127329  1017 ts_tablet_manager.cc:1434] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:17:07.129850  1021 raft_consensus.cc:385] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:07.130297  1021 raft_consensus.cc:740] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:07.122745   942 leader_election.cc:304] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:07.131186  1044 raft_consensus.cc:2804] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:07.131774  1044 raft_consensus.cc:697] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:07.131289  1021 consensus_queue.cc:260] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.132771  1044 consensus_queue.cc:237] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.134060  1021 ts_tablet_manager.cc:1434] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:17:07.137709  1051 consensus_queue.cc:1048] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.140486  1048 consensus_queue.cc:1048] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:07.143851   725 catalog_manager.cc:5697] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:07.149204  1023 consensus_queue.cc:1048] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.159377  1044 consensus_queue.cc:1048] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.179415  1023 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:07.190701   903 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.201746  1063 consensus_queue.cc:1048] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.203483  1040 consensus_queue.cc:1048] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.201870  1064 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.204948  1064 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
W20260709 13:17:07.210391   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:07.223014  1052 raft_consensus.cc:493] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.223961  1052 raft_consensus.cc:515] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.227566  1052 leader_election.cc:290] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:07.215631   725 catalog_manager.cc:5697] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:07.230235   974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:07.227895   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:07.220274  1064 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:07.245886   826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:07.245978  1064 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:07.247677   868 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:07.247996   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:07.249639   827 raft_consensus.cc:2393] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdbc0a613a499e831042aca13d367e in current term 1: Already voted for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in this term.
I20260709 13:17:07.249744  1064 raft_consensus.cc:2749] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:07.255570   868 leader_election.cc:304] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e; no voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:07.258699  1064 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.268873  1064 raft_consensus.cc:2749] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:07.306524  1064 raft_consensus.cc:493] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.307127  1064 raft_consensus.cc:515] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.309289  1064 leader_election.cc:290] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:07.310230   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0c817782084112a28fa822071ccc71" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af" is_pre_election: true
I20260709 13:17:07.310448   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0c817782084112a28fa822071ccc71" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:07.311021   827 raft_consensus.cc:2468] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 0.
I20260709 13:17:07.311141   976 raft_consensus.cc:2468] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 0.
I20260709 13:17:07.312279   868 leader_election.cc:304] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:07.313014  1064 raft_consensus.cc:2804] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:07.313331  1064 raft_consensus.cc:493] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.313634  1064 raft_consensus.cc:3060] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.319419  1064 raft_consensus.cc:515] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.321305  1064 leader_election.cc:290] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:07.322480   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0c817782084112a28fa822071ccc71" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:07.322351   827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0c817782084112a28fa822071ccc71" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:07.323256   976 raft_consensus.cc:3060] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.323307   827 raft_consensus.cc:3060] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.331712   976 raft_consensus.cc:2468] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 1.
I20260709 13:17:07.331712   827 raft_consensus.cc:2468] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 1.
I20260709 13:17:07.333360   868 leader_election.cc:304] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:07.334616  1064 raft_consensus.cc:2804] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:07.335143  1064 raft_consensus.cc:697] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:07.335949  1064 consensus_queue.cc:237] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:07.344591   725 catalog_manager.cc:5697] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.404232 21636 ts_manager.cc:295] Set tserver state for f30dd7659dd84ca7bb677ab9804924af to MAINTENANCE_MODE
I20260709 13:17:07.405125 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
I20260709 13:17:07.435925 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.436844 21636 tablet_replica.cc:333] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.437688 21636 raft_consensus.cc:2243] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.438190 21636 raft_consensus.cc:2272] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.440356 21636 tablet_replica.cc:333] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.441169 21636 raft_consensus.cc:2243] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.441738 21636 raft_consensus.cc:2272] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.444636 21636 tablet_replica.cc:333] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.445380 21636 raft_consensus.cc:2243] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.446229 21636 pending_rounds.cc:70] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Trying to abort 1 pending ops.
I20260709 13:17:07.446476 21636 pending_rounds.cc:77] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305637998718726144 op_type: NO_OP noop_request { }
I20260709 13:17:07.446807 21636 raft_consensus.cc:2889] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:07.447151 21636 raft_consensus.cc:2272] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.449919 21636 tablet_replica.cc:333] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.450610 21636 raft_consensus.cc:2243] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.451066 21636 raft_consensus.cc:2272] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.454039 21636 tablet_replica.cc:333] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.454994 21636 raft_consensus.cc:2243] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.455690 21636 raft_consensus.cc:2272] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.459653 21636 tablet_replica.cc:333] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.460682 21636 raft_consensus.cc:2243] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.462451 21636 raft_consensus.cc:2272] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.466310 21636 tablet_replica.cc:333] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.467033 21636 raft_consensus.cc:2243] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.467519 21636 raft_consensus.cc:2272] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.470551 21636 tablet_replica.cc:333] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.471262 21636 raft_consensus.cc:2243] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.471990 21636 raft_consensus.cc:2272] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.475301 21636 tablet_replica.cc:333] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.475908 21636 raft_consensus.cc:2243] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.476382 21636 raft_consensus.cc:2272] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.479256 21636 tablet_replica.cc:333] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: stopping tablet replica
I20260709 13:17:07.479827 21636 raft_consensus.cc:2243] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.480319 21636 raft_consensus.cc:2272] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.516629 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
W20260709 13:17:07.525880   942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111) [suppressed 247 similar messages]
W20260709 13:17:07.531061   942 consensus_peers.cc:597] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.562152  1045 consensus_queue.cc:1048] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:07.583643   942 consensus_peers.cc:597] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.584220   942 consensus_peers.cc:597] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.627475   942 consensus_peers.cc:597] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.648396  1044 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:07.668046   868 consensus_peers.cc:597] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.691246   942 consensus_peers.cc:597] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.703974   868 consensus_peers.cc:597] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.726020   868 consensus_peers.cc:597] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.896054  1107 consensus_queue.cc:1048] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:08.237722   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:08.692296  1109 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f30dd7659dd84ca7bb677ab9804924af)
I20260709 13:17:08.693071  1109 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:08.695781  1109 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:08.697892   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
W20260709 13:17:08.699998   942 leader_election.cc:336] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111)
I20260709 13:17:08.700414   942 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33; no voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:08.701066  1109 raft_consensus.cc:2749] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:08.898160  1052 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f30dd7659dd84ca7bb677ab9804924af)
I20260709 13:17:08.898746  1052 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:08.900676  1052 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:08.901939   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" is_pre_election: true
I20260709 13:17:08.902729   976 raft_consensus.cc:2468] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 1.
I20260709 13:17:08.903996   869 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: 
W20260709 13:17:08.904750   868 leader_election.cc:336] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111)
I20260709 13:17:08.904801  1052 raft_consensus.cc:2804] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:08.905417  1052 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f30dd7659dd84ca7bb677ab9804924af)
I20260709 13:17:08.905881  1052 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:08.912153  1052 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:08.914263  1052 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:08.915697   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:08.916601   976 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:08.918787   868 leader_election.cc:336] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111)
I20260709 13:17:08.924055   976 raft_consensus.cc:2468] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:08.925307   869 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:08.926301  1052 raft_consensus.cc:2804] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:08.927537  1052 raft_consensus.cc:697] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:08.928354  1052 consensus_queue.cc:237] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:08.936518   725 catalog_manager.cc:5697] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 1 to 2, leader changed from f30dd7659dd84ca7bb677ab9804924af (127.21.33.1) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:08.980587  1109 raft_consensus.cc:493] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.981208  1109 raft_consensus.cc:515] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:08.983510  1109 leader_election.cc:290] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:08.985699   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e" is_pre_election: true
I20260709 13:17:08.986505   901 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 1.
I20260709 13:17:08.987835   942 leader_election.cc:304] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: 
I20260709 13:17:08.988920  1109 raft_consensus.cc:2804] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:17:08.989001   942 leader_election.cc:336] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111)
I20260709 13:17:08.989372  1109 raft_consensus.cc:493] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.989769  1109 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:08.996204  1109 raft_consensus.cc:515] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:08.998365  1109 leader_election.cc:290] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:09.000277   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:09.001042   901 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:09.003067   942 leader_election.cc:336] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111)
I20260709 13:17:09.008531   901 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 2.
I20260709 13:17:09.009840   942 leader_election.cc:304] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: f30dd7659dd84ca7bb677ab9804924af
I20260709 13:17:09.010588  1109 raft_consensus.cc:2804] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:09.011025  1109 raft_consensus.cc:697] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:09.011792  1109 consensus_queue.cc:237] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:09.020819   725 catalog_manager.cc:5697] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 1 to 2, leader changed from f30dd7659dd84ca7bb677ab9804924af (127.21.33.1) to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:09.239094   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:09.398155   868 consensus_peers.cc:597] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:09.425268   901 raft_consensus.cc:1275] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:09.427330  1109 consensus_queue.cc:1048] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.432986   976 raft_consensus.cc:1275] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.434549  1110 consensus_queue.cc:1048] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:09.445839   942 consensus_peers.cc:597] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:09.867457   942 consensus_peers.cc:597] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.068945   942 consensus_peers.cc:597] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.121243   942 consensus_peers.cc:597] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.151520   942 consensus_peers.cc:597] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.240401   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:10.246956   868 consensus_peers.cc:597] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.291499   868 consensus_peers.cc:597] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.293937   942 consensus_peers.cc:597] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.450929   868 consensus_peers.cc:597] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:11.241815   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:11.866603   942 consensus_peers.cc:597] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:11.876430   868 consensus_peers.cc:597] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:12.201946   942 consensus_peers.cc:597] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.243000   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:12.550606 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:12.557987  1125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.560431  1128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.560930  1126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:12.564298 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:12.568032 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:12.568269 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:12.568493 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603032568473 us; error 0 us; skew 500 ppm
I20260709 13:17:12.569141 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:12.572083 21636 webserver.cc:533] Webserver started at http://127.21.33.1:38945/ using document root <none> and password file <none>
I20260709 13:17:12.572638 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:12.572856 21636 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:17:12.578384   942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111) [suppressed 99 similar messages]
I20260709 13:17:12.579056 21636 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.001s	sys 0.004s
W20260709 13:17:12.581701   942 consensus_peers.cc:597] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:12.584416  1133 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.585722 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:17:12.586258 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "f30dd7659dd84ca7bb677ab9804924af"
format_stamp: "Formatted at 2026-07-09 13:17:05 on dist-test-slave-p8rg"
I20260709 13:17:12.586782 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:17:12.591305   942 consensus_peers.cc:597] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:12.600945 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:12.602499 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:12.604254 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:12.608618  1140 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:17:12.648712   868 consensus_peers.cc:597] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.674319   942 consensus_peers.cc:597] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:12.680364 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:17:12.680888 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.073s	user 0.003s	sys 0.000s
I20260709 13:17:12.681206 21636 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:17:12.687331  1140 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.708012  1140 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.709270  1140 tablet_bootstrap.cc:492] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.710086  1140 ts_tablet_manager.cc:1403] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.023s	user 0.022s	sys 0.000s
I20260709 13:17:12.713207  1140 raft_consensus.cc:359] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.714067  1140 raft_consensus.cc:740] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.714957  1140 consensus_queue.cc:260] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.718237  1140 ts_tablet_manager.cc:1434] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:12.719228  1140 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.735463  1140 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.736361  1140 tablet_bootstrap.cc:492] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.737154  1140 ts_tablet_manager.cc:1403] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 13:17:12.739358  1140 raft_consensus.cc:359] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.739905  1140 raft_consensus.cc:740] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.740569  1140 consensus_queue.cc:260] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.742291  1140 ts_tablet_manager.cc:1434] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.008s
I20260709 13:17:12.743021  1140 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.746860 21636 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:17:12.747327 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.066s	user 0.059s	sys 0.007s
I20260709 13:17:12.759935  1140 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.761059  1140 tablet_bootstrap.cc:492] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.761855  1140 ts_tablet_manager.cc:1403] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.019s	user 0.018s	sys 0.000s
I20260709 13:17:12.765363  1140 raft_consensus.cc:359] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.766804  1140 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.767936  1140 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.770016  1140 ts_tablet_manager.cc:1434] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:12.770990  1140 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.784641  1140 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.785579  1140 tablet_bootstrap.cc:492] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.786525  1140 ts_tablet_manager.cc:1403] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:17:12.789608  1140 raft_consensus.cc:359] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.790467  1140 raft_consensus.cc:740] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.791484  1140 consensus_queue.cc:260] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.793898  1140 ts_tablet_manager.cc:1434] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:12.794977  1140 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.810129  1140 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.811231  1140 tablet_bootstrap.cc:492] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.812170  1140 ts_tablet_manager.cc:1403] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:17:12.814644  1140 raft_consensus.cc:359] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.815383  1140 raft_consensus.cc:740] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.816138  1140 consensus_queue.cc:260] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.818073  1140 ts_tablet_manager.cc:1434] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:17:12.818823  1140 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.822854 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:46789
I20260709 13:17:12.822966  1202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:46789 every 8 connection(s)
I20260709 13:17:12.828012 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:12.869298  1140 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:17:12.871225  1140 tablet_bootstrap.cc:492] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.872028  1140 ts_tablet_manager.cc:1403] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.028s
I20260709 13:17:12.874840  1203 heartbeater.cc:344] Connected to a master server at 127.21.33.62:34643
I20260709 13:17:12.875362  1203 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:12.874881  1140 raft_consensus.cc:359] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.880472  1140 raft_consensus.cc:740] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.880753  1203 heartbeater.cc:507] Master 127.21.33.62:34643 requested a full tablet report, sending...
I20260709 13:17:12.881379  1140 consensus_queue.cc:260] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.882273   976 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
mode: GRACEFUL
new_leader_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:41576
I20260709 13:17:12.884241  1140 ts_tablet_manager.cc:1434] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:17:12.884541   976 raft_consensus.cc:606] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Received request to transfer leadership to TS ccbdbc0a613a499e831042aca13d367e
W20260709 13:17:12.885039   868 consensus_peers.cc:597] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:12.885335  1140 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.888208   976 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38c50490ed8f45b394bba5138382e651"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
mode: GRACEFUL
new_leader_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:41576
I20260709 13:17:12.889003   976 raft_consensus.cc:606] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Received request to transfer leadership to TS ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:12.889900   725 ts_manager.cc:194] Re-registered known tserver with Master: f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789)
I20260709 13:17:12.890833 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:12.891812 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:12.892872 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:12.897481  1177 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.906041   725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43574
I20260709 13:17:12.911834  1203 heartbeater.cc:499] Master 127.21.33.62:34643 was elected leader, sending a full tablet report...
I20260709 13:17:12.913570  1177 pending_rounds.cc:85] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Aborting all ops after (but not including) 0
I20260709 13:17:12.914044  1177 pending_rounds.cc:107] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:12.914314  1177 raft_consensus.cc:2889] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.915012  1140 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.915803  1140 tablet_bootstrap.cc:492] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.916390  1140 ts_tablet_manager.cc:1403] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.012s
I20260709 13:17:12.919031  1140 raft_consensus.cc:359] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.919811  1140 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.920665  1140 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.922381  1140 ts_tablet_manager.cc:1434] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:17:12.923107  1140 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.937628  1140 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.939011  1140 tablet_bootstrap.cc:492] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.940008  1140 ts_tablet_manager.cc:1403] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
W20260709 13:17:12.941689   942 consensus_peers.cc:597] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:12.942744  1140 raft_consensus.cc:359] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.943521  1140 raft_consensus.cc:740] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.944283  1140 consensus_queue.cc:260] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.946399  1140 ts_tablet_manager.cc:1434] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:17:12.947232  1140 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.965262  1140 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.966118  1140 tablet_bootstrap.cc:492] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.966774  1140 ts_tablet_manager.cc:1403] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:17:12.968959  1140 raft_consensus.cc:359] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.969655  1140 raft_consensus.cc:385] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:12.970012  1140 raft_consensus.cc:740] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.970752  1140 consensus_queue.cc:260] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.973009  1140 ts_tablet_manager.cc:1434] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:17:12.973992  1140 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap starting.
I20260709 13:17:12.988013  1140 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:12.988957  1140 tablet_bootstrap.cc:492] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Bootstrap complete.
I20260709 13:17:12.989723  1140 ts_tablet_manager.cc:1403] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:17:12.991974  1140 raft_consensus.cc:359] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.992501  1140 raft_consensus.cc:740] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30dd7659dd84ca7bb677ab9804924af, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.993206  1140 consensus_queue.cc:260] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:12.994962  1140 ts_tablet_manager.cc:1434] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:13.036751  1024 raft_consensus.cc:993] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: : Instructing follower ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:13.037269  1219 raft_consensus.cc:1081] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Signalling peer ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:13.038954   901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38c50490ed8f45b394bba5138382e651"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:17:13.039475   901 raft_consensus.cc:493] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.039731   901 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.045398   901 raft_consensus.cc:515] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.047289   901 leader_election.cc:290] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:13.048236  1177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:13.048434   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:13.049157  1177 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.049222   976 raft_consensus.cc:3055] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:13.049688   976 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:13.050418   976 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.051455   976 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.057591  1177 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:13.058323   976 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:13.059303   868 leader_election.cc:304] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:13.060333  1110 raft_consensus.cc:2804] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:13.061570  1110 raft_consensus.cc:697] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:13.062477  1110 consensus_queue.cc:237] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.070827   725 catalog_manager.cc:5697] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 1 to 2, leader changed from 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:13.244307   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:13.308661  1219 raft_consensus.cc:993] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: : Instructing follower ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:13.309345  1219 raft_consensus.cc:1081] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Signalling peer ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:13.311187   901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:17:13.311863   901 raft_consensus.cc:493] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.312283   901 raft_consensus.cc:3060] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.318472   901 raft_consensus.cc:515] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.320196   901 leader_election.cc:290] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:13.321296  1177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:13.321393   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20e4db9ddb954998a5233fe1682e94cb" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:13.322098  1177 raft_consensus.cc:3060] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.322178   976 raft_consensus.cc:3055] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:13.322633   976 raft_consensus.cc:740] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:13.323525   976 consensus_queue.cc:260] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.324668   976 raft_consensus.cc:3060] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.328024  1176 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.328330  1175 raft_consensus.cc:3060] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.330142  1177 raft_consensus.cc:2468] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:13.332269   868 leader_election.cc:304] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:13.333284  1110 raft_consensus.cc:2804] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:13.334074  1110 raft_consensus.cc:697] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:13.334986  1110 consensus_queue.cc:237] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.340620   976 raft_consensus.cc:2468] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:13.347841   725 catalog_manager.cc:5697] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 1 to 2, leader changed from 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.503309  1176 raft_consensus.cc:1275] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.505179  1110 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:13.518046   976 raft_consensus.cc:1275] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.520901  1110 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:13.742419   976 raft_consensus.cc:1275] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.744385  1110 consensus_queue.cc:1048] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:13.760174  1175 raft_consensus.cc:1275] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.762631  1220 consensus_queue.cc:1048] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:13.908102 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:13.934681   901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
mode: GRACEFUL
new_leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:51588
I20260709 13:17:13.935483   901 raft_consensus.cc:606] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Received request to transfer leadership to TS 0f89b8f359fc4d2a8165bf7bae901e33
I20260709 13:17:13.938594   901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
mode: GRACEFUL
new_leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:51588
I20260709 13:17:13.939158   901 raft_consensus.cc:606] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Received request to transfer leadership to TS 0f89b8f359fc4d2a8165bf7bae901e33
I20260709 13:17:13.940321 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:13.940915 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:13.942059 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:13.969657  1110 raft_consensus.cc:993] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: : Instructing follower 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:13.970345  1220 raft_consensus.cc:1081] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Signalling peer 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:13.972219   976 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:44976
I20260709 13:17:13.972895   976 raft_consensus.cc:493] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.973227   976 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.983906   976 raft_consensus.cc:515] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.986090   976 leader_election.cc:290] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 3 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:13.987250  1175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:13.987377   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc01e2a1c574e12a1288e9ce08da385" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:13.988190  1175 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.988222   901 raft_consensus.cc:3055] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.988826   901 raft_consensus.cc:740] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:13.989902   901 consensus_queue.cc:260] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:13.991155   901 raft_consensus.cc:3060] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.997803  1175 raft_consensus.cc:2468] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 3.
I20260709 13:17:13.999388   942 leader_election.cc:304] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:14.000301   901 raft_consensus.cc:2468] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 3.
I20260709 13:17:14.000524  1239 raft_consensus.cc:2804] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:14.001190  1239 raft_consensus.cc:697] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:14.002660  1239 consensus_queue.cc:237] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:14.011565   725 catalog_manager.cc:5697] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 2 to 3, leader changed from ccbdbc0a613a499e831042aca13d367e (127.21.33.2) to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:14.157816  1220 raft_consensus.cc:993] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: : Instructing follower 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:14.158324  1110 raft_consensus.cc:1081] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Signalling peer 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:14.160048   977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:44976
I20260709 13:17:14.160723   977 raft_consensus.cc:493] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.161089   977 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.169446   977 raft_consensus.cc:515] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:14.171670   977 leader_election.cc:290] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:14.172961   901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:14.172729  1175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:14.173538   901 raft_consensus.cc:3055] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.173700  1175 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.173956   901 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:14.175029   901 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:14.176712   901 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.183029  1175 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 2.
I20260709 13:17:14.185199   942 leader_election.cc:304] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:14.185899   901 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 2.
I20260709 13:17:14.186234  1239 raft_consensus.cc:2804] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.186883  1239 raft_consensus.cc:697] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:14.188036  1239 consensus_queue.cc:237] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:14.196554   725 catalog_manager.cc:5697] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 1 to 2, leader changed from ccbdbc0a613a499e831042aca13d367e (127.21.33.2) to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:14.245790   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:14.404697  1175 raft_consensus.cc:1275] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.406570  1239 consensus_queue.cc:1048] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:14.418939   901 raft_consensus.cc:1275] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.420895  1239 consensus_queue.cc:1048] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:14.586658  1175 raft_consensus.cc:1275] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.588574  1239 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.600033   901 raft_consensus.cc:1275] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.602496  1239 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.948745 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:14.956398   977 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
mode: GRACEFUL
new_leader_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:41576
I20260709 13:17:14.957042   977 raft_consensus.cc:606] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Received request to transfer leadership to TS ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:14.959910   977 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
mode: GRACEFUL
new_leader_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:41576
I20260709 13:17:14.960481   977 raft_consensus.cc:606] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Received request to transfer leadership to TS ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:14.961858 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:14.962419 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:14.963063 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:15.090827  1250 raft_consensus.cc:993] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: : Instructing follower ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:15.091396  1239 raft_consensus.cc:1081] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Signalling peer ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:15.093221   901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:17:15.093811   901 raft_consensus.cc:493] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:15.094139   901 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.102833   901 raft_consensus.cc:515] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:15.104808   901 leader_election.cc:290] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 3 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:15.105677  1176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:15.106117   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:15.106410  1176 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.106766   977 raft_consensus.cc:3055] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:15.107069   977 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:15.107859   977 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:15.109126   977 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.114338  1176 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 3.
I20260709 13:17:15.114802   977 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 3.
I20260709 13:17:15.115787   868 leader_election.cc:304] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:15.116902  1110 raft_consensus.cc:2804] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:15.118000  1110 raft_consensus.cc:697] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:15.118870  1110 consensus_queue.cc:237] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:15.125743   724 catalog_manager.cc:5697] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 2 to 3, leader changed from 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 3 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.144635  1250 raft_consensus.cc:993] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: : Instructing follower ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:15.145220  1239 raft_consensus.cc:1081] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Signalling peer ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:15.146733   901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:17:15.147305   901 raft_consensus.cc:493] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:15.147660   901 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.154290   901 raft_consensus.cc:515] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:15.156327   901 leader_election.cc:290] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 3 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:15.157451  1176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:15.158020   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5dbb685cfd941618e5faf493d1824d7" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:15.158160  1176 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.158591   977 raft_consensus.cc:3055] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:15.158895   977 raft_consensus.cc:740] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:15.159790   977 consensus_queue.cc:260] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:15.160965   977 raft_consensus.cc:3060] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:15.165290  1176 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 3.
I20260709 13:17:15.166693   868 leader_election.cc:304] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:15.167526  1110 raft_consensus.cc:2804] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:15.167912   977 raft_consensus.cc:2468] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 3.
I20260709 13:17:15.168092  1110 raft_consensus.cc:697] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:15.169478  1110 consensus_queue.cc:237] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:15.178740   724 catalog_manager.cc:5697] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 2 to 3, leader changed from 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 3 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:15.246985   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:15.580927   977 raft_consensus.cc:1275] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:15.582576  1110 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:15.593432  1175 raft_consensus.cc:1275] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:15.595157  1110 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:15.625277   977 raft_consensus.cc:1275] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:15.626605  1110 consensus_queue.cc:1048] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:15.640686  1175 raft_consensus.cc:1275] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:15.643138  1263 consensus_queue.cc:1048] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:15.971659 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:15.980610   901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
mode: GRACEFUL
new_leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:51588
I20260709 13:17:15.981341   901 raft_consensus.cc:606] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Received request to transfer leadership to TS 0f89b8f359fc4d2a8165bf7bae901e33
I20260709 13:17:15.984556   901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38c50490ed8f45b394bba5138382e651"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
mode: GRACEFUL
new_leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:51588
I20260709 13:17:15.985172   901 raft_consensus.cc:606] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Received request to transfer leadership to TS 0f89b8f359fc4d2a8165bf7bae901e33
I20260709 13:17:15.986454 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:15.987263 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:15.988088 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:16.134513  1263 raft_consensus.cc:993] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: : Instructing follower 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:16.134964  1110 raft_consensus.cc:1081] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Signalling peer 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:16.136489   977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:44976
I20260709 13:17:16.137076   977 raft_consensus.cc:493] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.137409   977 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.143651   977 raft_consensus.cc:515] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:16.145836   977 leader_election.cc:290] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 4 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:16.147090   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:16.147421  1175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1734ca1fafaa47f28f09883752366ef3" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:16.147902   903 raft_consensus.cc:3055] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:16.148020  1175 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.148280   903 raft_consensus.cc:740] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:16.149354   903 consensus_queue.cc:260] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:16.150890   903 raft_consensus.cc:3060] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.157534  1175 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 4.
I20260709 13:17:16.159256   942 leader_election.cc:304] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:16.160065  1239 raft_consensus.cc:2804] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:16.160133   903 raft_consensus.cc:2468] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 4.
I20260709 13:17:16.160668  1239 raft_consensus.cc:697] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:16.161648  1239 consensus_queue.cc:237] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:16.168534   724 catalog_manager.cc:5697] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 3 to 4, leader changed from ccbdbc0a613a499e831042aca13d367e (127.21.33.2) to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 4 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:16.248360   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:16.314281  1263 raft_consensus.cc:993] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: : Instructing follower 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:16.314935  1110 raft_consensus.cc:1081] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Signalling peer 0f89b8f359fc4d2a8165bf7bae901e33 to start an election
I20260709 13:17:16.316759   977 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38c50490ed8f45b394bba5138382e651"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
 from {username='slave'} at 127.0.0.1:44976
I20260709 13:17:16.317414   977 raft_consensus.cc:493] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.317816   977 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:16.323907   977 raft_consensus.cc:515] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:16.325748   977 leader_election.cc:290] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 3 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721)
I20260709 13:17:16.326891   903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
I20260709 13:17:16.327250  1175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:16.327626   903 raft_consensus.cc:3055] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:16.327900  1175 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:16.328007   903 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:16.329105   903 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:16.330545   903 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:16.336194  1175 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 3.
I20260709 13:17:16.337600   942 leader_election.cc:304] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:16.338407   903 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f89b8f359fc4d2a8165bf7bae901e33 in term 3.
I20260709 13:17:16.338446  1239 raft_consensus.cc:2804] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:16.339175  1239 raft_consensus.cc:697] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Becoming Leader. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:16.340256  1239 consensus_queue.cc:237] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:16.347553   724 catalog_manager.cc:5697] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 reported cstate change: term changed from 2 to 3, leader changed from ccbdbc0a613a499e831042aca13d367e (127.21.33.2) to 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:16.568463   903 raft_consensus.cc:1275] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:16.569934  1250 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:16.585044  1175 raft_consensus.cc:1275] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:16.586757  1250 consensus_queue.cc:1048] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:16.770030   903 raft_consensus.cc:1275] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:16.771739  1250 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:16.781574  1175 raft_consensus.cc:1275] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Refusing update from remote peer 0f89b8f359fc4d2a8165bf7bae901e33: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:16.783587  1250 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:16.994959 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:17.003389   977 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38c50490ed8f45b394bba5138382e651"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
mode: GRACEFUL
new_leader_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:41576
I20260709 13:17:17.003974   977 raft_consensus.cc:606] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Received request to transfer leadership to TS ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:17.006615   977 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff"
dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
mode: GRACEFUL
new_leader_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:41576
I20260709 13:17:17.007083   977 raft_consensus.cc:606] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Received request to transfer leadership to TS ccbdbc0a613a499e831042aca13d367e
I20260709 13:17:17.008302 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:17.008822 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:17.009397 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:17.249759   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:17.306372  1239 raft_consensus.cc:993] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: : Instructing follower ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:17.307119  1275 raft_consensus.cc:1081] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Signalling peer ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:17.309027  1239 raft_consensus.cc:993] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: : Instructing follower ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:17.309006   901 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:17:17.309728  1239 raft_consensus.cc:1081] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Signalling peer ccbdbc0a613a499e831042aca13d367e to start an election
I20260709 13:17:17.309733   901 raft_consensus.cc:493] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.310348   901 raft_consensus.cc:3060] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:17.311702   903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38c50490ed8f45b394bba5138382e651"
dest_uuid: "ccbdbc0a613a499e831042aca13d367e"
 from {username='slave'} at 127.0.0.1:39020
I20260709 13:17:17.312288   903 raft_consensus.cc:493] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.312662   903 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.317987   901 raft_consensus.cc:515] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:17.319623   903 raft_consensus.cc:515] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:17.320369   901 leader_election.cc:290] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:17.320776  1176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:17.321002   977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a917d956d957469eacc1bacf690a53ff" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:17.321446  1176 raft_consensus.cc:3060] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:17.321600   977 raft_consensus.cc:3055] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:17.322031   977 raft_consensus.cc:740] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:17.323064   977 consensus_queue.cc:260] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:17.323491   903 leader_election.cc:290] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 4 election: Requested vote from peers f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789), 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3:42819)
I20260709 13:17:17.324220   976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0f89b8f359fc4d2a8165bf7bae901e33"
I20260709 13:17:17.324633   977 raft_consensus.cc:3060] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:17.324764   976 raft_consensus.cc:3055] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:17.325321   976 raft_consensus.cc:740] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0f89b8f359fc4d2a8165bf7bae901e33, State: Running, Role: LEADER
I20260709 13:17:17.326124  1175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38c50490ed8f45b394bba5138382e651" candidate_uuid: "ccbdbc0a613a499e831042aca13d367e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f30dd7659dd84ca7bb677ab9804924af"
I20260709 13:17:17.326800  1175 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.326351   976 consensus_queue.cc:260] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:17.327802   976 raft_consensus.cc:3060] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.330253  1176 raft_consensus.cc:2468] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:17.331995   868 leader_election.cc:304] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdbc0a613a499e831042aca13d367e, f30dd7659dd84ca7bb677ab9804924af; no voters: 
I20260709 13:17:17.332986  1263 raft_consensus.cc:2804] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:17.333451   977 raft_consensus.cc:2468] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 2.
I20260709 13:17:17.336213   976 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 4.
I20260709 13:17:17.335215  1175 raft_consensus.cc:2468] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdbc0a613a499e831042aca13d367e in term 4.
I20260709 13:17:17.336205  1263 raft_consensus.cc:697] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:17.337914   869 leader_election.cc:304] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f89b8f359fc4d2a8165bf7bae901e33, ccbdbc0a613a499e831042aca13d367e; no voters: 
I20260709 13:17:17.338248  1263 consensus_queue.cc:237] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:17.339010  1280 raft_consensus.cc:2804] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:17.340210  1280 raft_consensus.cc:697] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 4 LEADER]: Becoming Leader. State: Replica: ccbdbc0a613a499e831042aca13d367e, State: Running, Role: LEADER
I20260709 13:17:17.341045  1280 consensus_queue.cc:237] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } }
I20260709 13:17:17.347168   724 catalog_manager.cc:5697] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 1 to 2, leader changed from 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:17.364578   724 catalog_manager.cc:5697] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e reported cstate change: term changed from 3 to 4, leader changed from 0f89b8f359fc4d2a8165bf7bae901e33 (127.21.33.3) to ccbdbc0a613a499e831042aca13d367e (127.21.33.2). New cstate: current_term: 4 leader_uuid: "ccbdbc0a613a499e831042aca13d367e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdbc0a613a499e831042aca13d367e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:17.850292  1175 raft_consensus.cc:1275] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.852219  1280 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:17.868795   976 raft_consensus.cc:1275] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.871263  1288 consensus_queue.cc:1048] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:17.880888   977 raft_consensus.cc:1275] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.884402  1289 consensus_queue.cc:1048] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f89b8f359fc4d2a8165bf7bae901e33" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:17.901690  1175 raft_consensus.cc:1275] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Refusing update from remote peer ccbdbc0a613a499e831042aca13d367e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.904772  1288 consensus_queue.cc:1048] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30dd7659dd84ca7bb677ab9804924af" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 46789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.015669 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:18.047197 21636 tablet_server.cc:179] TabletServer@127.21.33.1:46789 shutting down...
I20260709 13:17:18.087445 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:18.088699 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.089972 21636 raft_consensus.cc:2243] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.090704 21636 raft_consensus.cc:2272] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.094295 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.095636 21636 raft_consensus.cc:2243] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.096261 21636 raft_consensus.cc:2272] T 38c50490ed8f45b394bba5138382e651 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.100162 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.100937 21636 raft_consensus.cc:2243] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.101469 21636 raft_consensus.cc:2272] T d5dbb685cfd941618e5faf493d1824d7 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.104166 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.104879 21636 raft_consensus.cc:2243] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.105408 21636 raft_consensus.cc:2272] T 26359eb1650e41ee99a2f5e88a6d3a20 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.108203 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.108952 21636 raft_consensus.cc:2243] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.109751 21636 raft_consensus.cc:2272] T 1d0c817782084112a28fa822071ccc71 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.112398 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.113042 21636 raft_consensus.cc:2243] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.113579 21636 raft_consensus.cc:2272] T a917d956d957469eacc1bacf690a53ff P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.117012 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.117758 21636 raft_consensus.cc:2243] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.118284 21636 raft_consensus.cc:2272] T 1734ca1fafaa47f28f09883752366ef3 P f30dd7659dd84ca7bb677ab9804924af [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.120824 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.121459 21636 raft_consensus.cc:2243] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.122010 21636 raft_consensus.cc:2272] T fdc01e2a1c574e12a1288e9ce08da385 P f30dd7659dd84ca7bb677ab9804924af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.124418 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.125005 21636 raft_consensus.cc:2243] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.125484 21636 raft_consensus.cc:2272] T 68dfa173666343c7acbc683137e46389 P f30dd7659dd84ca7bb677ab9804924af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.128351 21636 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:18.128932 21636 raft_consensus.cc:2243] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.129416 21636 raft_consensus.cc:2272] T 20e4db9ddb954998a5233fe1682e94cb P f30dd7659dd84ca7bb677ab9804924af [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.149381   868 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 13:17:18.156481   868 consensus_peers.cc:597] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:18.163589   942 consensus_peers.cc:597] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.169938 21636 tablet_server.cc:196] TabletServer@127.21.33.1:46789 shutdown complete.
W20260709 13:17:18.200649   942 consensus_peers.cc:597] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.207876 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
W20260709 13:17:18.219022   942 consensus_peers.cc:597] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer f30dd7659dd84ca7bb677ab9804924af (127.21.33.1:46789): Couldn't send request to peer f30dd7659dd84ca7bb677ab9804924af. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:46789: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.243619 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:18.244700 21636 tablet_replica.cc:333] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.246119 21636 raft_consensus.cc:2243] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.246793 21636 raft_consensus.cc:2272] T fdc01e2a1c574e12a1288e9ce08da385 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.250079 21636 tablet_replica.cc:333] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
W20260709 13:17:18.250950   774 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:18.251169 21636 raft_consensus.cc:2243] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.252244 21636 raft_consensus.cc:2272] T a917d956d957469eacc1bacf690a53ff P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.255131 21636 tablet_replica.cc:333] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.255728 21636 raft_consensus.cc:2243] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.256158 21636 raft_consensus.cc:2272] T 1734ca1fafaa47f28f09883752366ef3 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.258495 21636 tablet_replica.cc:333] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.259114 21636 raft_consensus.cc:2243] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.260372 21636 raft_consensus.cc:2272] T 1d0c817782084112a28fa822071ccc71 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.263350 21636 tablet_replica.cc:333] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.263983 21636 raft_consensus.cc:2243] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.264434 21636 raft_consensus.cc:2272] T 26359eb1650e41ee99a2f5e88a6d3a20 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.267060 21636 tablet_replica.cc:333] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.267680 21636 raft_consensus.cc:2243] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.268608 21636 raft_consensus.cc:2272] T 20e4db9ddb954998a5233fe1682e94cb P ccbdbc0a613a499e831042aca13d367e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.271728 21636 tablet_replica.cc:333] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.272346 21636 raft_consensus.cc:2243] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.272863 21636 raft_consensus.cc:2272] T 68dfa173666343c7acbc683137e46389 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.277205 21636 tablet_replica.cc:333] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.278105 21636 raft_consensus.cc:2243] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 4 LEADER]: Raft consensus shutting down.
W20260709 13:17:18.279001   942 consensus_peers.cc:597] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 -> Peer ccbdbc0a613a499e831042aca13d367e (127.21.33.2:41721): Couldn't send request to peer ccbdbc0a613a499e831042aca13d367e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.279426 21636 raft_consensus.cc:2272] T 38c50490ed8f45b394bba5138382e651 P ccbdbc0a613a499e831042aca13d367e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.283366 21636 tablet_replica.cc:333] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.284063 21636 raft_consensus.cc:2243] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.285127 21636 raft_consensus.cc:2272] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P ccbdbc0a613a499e831042aca13d367e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.288153 21636 tablet_replica.cc:333] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e: stopping tablet replica
I20260709 13:17:18.288731 21636 raft_consensus.cc:2243] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.289647 21636 raft_consensus.cc:2272] T d5dbb685cfd941618e5faf493d1824d7 P ccbdbc0a613a499e831042aca13d367e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.320355 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:17:18.349236 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
I20260709 13:17:18.388815 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:18.389680 21636 tablet_replica.cc:333] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.390578 21636 raft_consensus.cc:2243] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.391196 21636 raft_consensus.cc:2272] T d5dbb685cfd941618e5faf493d1824d7 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.393895 21636 tablet_replica.cc:333] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.394582 21636 raft_consensus.cc:2243] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.395056 21636 raft_consensus.cc:2272] T a917d956d957469eacc1bacf690a53ff P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.398623 21636 tablet_replica.cc:333] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.399328 21636 raft_consensus.cc:2243] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.400231 21636 raft_consensus.cc:2272] T 1734ca1fafaa47f28f09883752366ef3 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.402745 21636 tablet_replica.cc:333] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.403354 21636 raft_consensus.cc:2243] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.404229 21636 raft_consensus.cc:2272] T fdc01e2a1c574e12a1288e9ce08da385 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.407276 21636 tablet_replica.cc:333] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.408062 21636 raft_consensus.cc:2243] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.408587 21636 raft_consensus.cc:2272] T 38c50490ed8f45b394bba5138382e651 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.411662 21636 tablet_replica.cc:333] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.412320 21636 raft_consensus.cc:2243] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.412848 21636 raft_consensus.cc:2272] T 8c1a020d7eea4c1aaf57aaef1f7a2242 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.415431 21636 tablet_replica.cc:333] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.416007 21636 raft_consensus.cc:2243] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.416968 21636 raft_consensus.cc:2272] T 68dfa173666343c7acbc683137e46389 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.419584 21636 tablet_replica.cc:333] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.420176 21636 raft_consensus.cc:2243] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.420660 21636 raft_consensus.cc:2272] T 20e4db9ddb954998a5233fe1682e94cb P 0f89b8f359fc4d2a8165bf7bae901e33 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.423072 21636 tablet_replica.cc:333] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.423606 21636 raft_consensus.cc:2243] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.424106 21636 raft_consensus.cc:2272] T 1d0c817782084112a28fa822071ccc71 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.426890 21636 tablet_replica.cc:333] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33: stopping tablet replica
I20260709 13:17:18.427475 21636 raft_consensus.cc:2243] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.428527 21636 raft_consensus.cc:2272] T 26359eb1650e41ee99a2f5e88a6d3a20 P 0f89b8f359fc4d2a8165bf7bae901e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.466948 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:17:18.498500 21636 master.cc:562] Master@127.21.33.62:34643 shutting down...
I20260709 13:17:18.524329 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.525017 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.525388 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 77b1bea634234737a3ee2938c35732de: stopping tablet replica
I20260709 13:17:18.548832 21636 master.cc:584] Master@127.21.33.62:34643 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13584 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:17:18.602744 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:36745
I20260709 13:17:18.604036 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.611251  1302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.611395  1303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.612366  1305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.613200 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:18.614634 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.614925 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.615159 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603038615136 us; error 0 us; skew 500 ppm
I20260709 13:17:18.615795 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.619668 21636 webserver.cc:533] Webserver started at http://127.21.33.62:33845/ using document root <none> and password file <none>
I20260709 13:17:18.620281 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.620494 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.620820 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.622184 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "6a1ad7b2afeb44d3b27a6e3480021fef"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-p8rg"
I20260709 13:17:18.627424 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:17:18.631772  1310 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.632838 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:17:18.633112 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "6a1ad7b2afeb44d3b27a6e3480021fef"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-p8rg"
I20260709 13:17:18.633400 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.650579 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.651957 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.699163 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:36745
I20260709 13:17:18.699280  1361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:36745 every 8 connection(s)
I20260709 13:17:18.703962  1362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:18.717792  1362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef: Bootstrap starting.
I20260709 13:17:18.723356  1362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:18.728186  1362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef: No bootstrap required, opened a new log
I20260709 13:17:18.730679  1362 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" member_type: VOTER }
I20260709 13:17:18.731113  1362 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:18.731364  1362 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a1ad7b2afeb44d3b27a6e3480021fef, State: Initialized, Role: FOLLOWER
I20260709 13:17:18.732059  1362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" member_type: VOTER }
I20260709 13:17:18.732542  1362 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:18.732774  1362 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:18.733053  1362 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:18.739001  1362 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" member_type: VOTER }
I20260709 13:17:18.739557  1362 leader_election.cc:304] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a1ad7b2afeb44d3b27a6e3480021fef; no voters: 
I20260709 13:17:18.740881  1362 leader_election.cc:290] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:18.741411  1365 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:18.743043  1365 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 1 LEADER]: Becoming Leader. State: Replica: 6a1ad7b2afeb44d3b27a6e3480021fef, State: Running, Role: LEADER
I20260709 13:17:18.743809  1365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" member_type: VOTER }
I20260709 13:17:18.744299  1362 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:18.748615  1366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" member_type: VOTER } }
I20260709 13:17:18.748615  1367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a1ad7b2afeb44d3b27a6e3480021fef. Latest consensus state: current_term: 1 leader_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1ad7b2afeb44d3b27a6e3480021fef" member_type: VOTER } }
I20260709 13:17:18.749609  1366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:18.749708  1367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:18.756245  1372 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:18.764665  1372 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:18.765910 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:18.775359  1372 catalog_manager.cc:1383] Generated new cluster ID: 1a5365b1ed5a487abb21d2c89c6fe12e
I20260709 13:17:18.775740  1372 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:18.803848  1372 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:18.805634  1372 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:18.821759  1372 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef: Generated new TSK 0
I20260709 13:17:18.822453  1372 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:18.832893 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.839905  1383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.841115  1384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.842927  1386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.843432 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:18.844367 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.844547 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.844749 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603038844711 us; error 0 us; skew 500 ppm
I20260709 13:17:18.845335 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.848120 21636 webserver.cc:533] Webserver started at http://127.21.33.1:41437/ using document root <none> and password file <none>
I20260709 13:17:18.848650 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.848827 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.849100 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.850430 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "d79a71b7633445f4ac092687ee0cf69a"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-p8rg"
I20260709 13:17:18.855340 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:17:18.859211  1391 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.860038 21636 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:17:18.860307 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "d79a71b7633445f4ac092687ee0cf69a"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-p8rg"
I20260709 13:17:18.860570 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.872720 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.874104 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.875790 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:18.878532 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:18.878727 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.879047 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:18.879212 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.930421 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:44695
I20260709 13:17:18.930547  1453 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:44695 every 8 connection(s)
I20260709 13:17:18.937778 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.944617  1458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.945278  1459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.949026 21636 server_base.cc:1061] running on GCE node
W20260709 13:17:18.949447  1462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.950680  1454 heartbeater.cc:344] Connected to a master server at 127.21.33.62:36745
I20260709 13:17:18.950723 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.951179 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.951215  1454 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:18.951390 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603038951360 us; error 0 us; skew 500 ppm
I20260709 13:17:18.952168  1454 heartbeater.cc:507] Master 127.21.33.62:36745 requested a full tablet report, sending...
I20260709 13:17:18.952340 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.955009  1327 ts_manager.cc:194] Registered new tserver with Master: d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695)
I20260709 13:17:18.955600 21636 webserver.cc:533] Webserver started at http://127.21.33.2:42883/ using document root <none> and password file <none>
I20260709 13:17:18.956426 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.956707 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.957018 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.957815  1327 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44356
I20260709 13:17:18.958647 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "1e80485e9f984a51a43e3055f85bb19d"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-p8rg"
I20260709 13:17:18.964071 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.005s
I20260709 13:17:18.968078  1466 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.969471 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:17:18.969956 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "1e80485e9f984a51a43e3055f85bb19d"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-p8rg"
I20260709 13:17:18.970327 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.989270 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.990825 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.992612 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:18.995527 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:18.995759 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.996050 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:18.996202 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:19.048559 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:40137
I20260709 13:17:19.048677  1528 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:40137 every 8 connection(s)
I20260709 13:17:19.054204 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:19.064177  1532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:19.068806  1529 heartbeater.cc:344] Connected to a master server at 127.21.33.62:36745
I20260709 13:17:19.069440  1529 heartbeater.cc:461] Registering TS with master...
W20260709 13:17:19.069739  1533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:19.070814  1529 heartbeater.cc:507] Master 127.21.33.62:36745 requested a full tablet report, sending...
W20260709 13:17:19.071385  1535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:19.072292 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:19.073764 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:17:19.073750  1327 ts_manager.cc:194] Registered new tserver with Master: 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137)
W20260709 13:17:19.074352 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:19.074590 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603039074569 us; error 0 us; skew 500 ppm
I20260709 13:17:19.075527 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:19.076128  1327 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44364
I20260709 13:17:19.078876 21636 webserver.cc:533] Webserver started at http://127.21.33.3:32773/ using document root <none> and password file <none>
I20260709 13:17:19.079550 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:19.079846 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:19.080142 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:19.081456 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "519294df0dc748d98a8e500cb3417403"
format_stamp: "Formatted at 2026-07-09 13:17:19 on dist-test-slave-p8rg"
I20260709 13:17:19.086908 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:19.091032  1540 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:19.092077 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:17:19.092362 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "519294df0dc748d98a8e500cb3417403"
format_stamp: "Formatted at 2026-07-09 13:17:19 on dist-test-slave-p8rg"
I20260709 13:17:19.092623 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:19.127936 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:19.129343 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:19.131229 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:19.133716 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:19.133929 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:19.134312 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:19.134517 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:19.187929 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:42451
I20260709 13:17:19.188046  1602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:42451 every 8 connection(s)
I20260709 13:17:19.205230  1603 heartbeater.cc:344] Connected to a master server at 127.21.33.62:36745
I20260709 13:17:19.205906  1603 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:19.206784  1603 heartbeater.cc:507] Master 127.21.33.62:36745 requested a full tablet report, sending...
I20260709 13:17:19.209410  1327 ts_manager.cc:194] Registered new tserver with Master: 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:19.209832 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018200378s
I20260709 13:17:19.211817  1327 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44378
I20260709 13:17:19.242009  1327 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44382:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:19.244406  1327 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:19.295477  1568 tablet_service.cc:1511] Processing CreateTablet for tablet aecdf4444f4e4221aa01a6e894bf285a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=aea34d7c6efd4a8c899c52a99e975477]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:19.297290  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aecdf4444f4e4221aa01a6e894bf285a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.310005  1567 tablet_service.cc:1511] Processing CreateTablet for tablet 6370e807c6e34f289bfc57d82de7ee44 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=aea34d7c6efd4a8c899c52a99e975477]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:19.311829  1567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6370e807c6e34f289bfc57d82de7ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.319293  1494 tablet_service.cc:1511] Processing CreateTablet for tablet aecdf4444f4e4221aa01a6e894bf285a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=aea34d7c6efd4a8c899c52a99e975477]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:19.320752  1494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aecdf4444f4e4221aa01a6e894bf285a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.322826  1493 tablet_service.cc:1511] Processing CreateTablet for tablet 6370e807c6e34f289bfc57d82de7ee44 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=aea34d7c6efd4a8c899c52a99e975477]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:19.323474  1419 tablet_service.cc:1511] Processing CreateTablet for tablet aecdf4444f4e4221aa01a6e894bf285a (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=aea34d7c6efd4a8c899c52a99e975477]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:19.324841  1419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aecdf4444f4e4221aa01a6e894bf285a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.325376  1493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6370e807c6e34f289bfc57d82de7ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.329957  1418 tablet_service.cc:1511] Processing CreateTablet for tablet 6370e807c6e34f289bfc57d82de7ee44 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=aea34d7c6efd4a8c899c52a99e975477]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:19.331351  1418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6370e807c6e34f289bfc57d82de7ee44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.364022  1618 tablet_bootstrap.cc:492] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403: Bootstrap starting.
I20260709 13:17:19.375849  1618 tablet_bootstrap.cc:654] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.376996  1619 tablet_bootstrap.cc:492] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a: Bootstrap starting.
I20260709 13:17:19.378652  1620 tablet_bootstrap.cc:492] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: Bootstrap starting.
I20260709 13:17:19.384958  1618 tablet_bootstrap.cc:492] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403: No bootstrap required, opened a new log
I20260709 13:17:19.385702  1618 ts_tablet_manager.cc:1403] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:17:19.385766  1620 tablet_bootstrap.cc:654] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.385810  1619 tablet_bootstrap.cc:654] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.388729  1618 raft_consensus.cc:359] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.389370  1618 raft_consensus.cc:385] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.389703  1618 raft_consensus.cc:740] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.391057  1618 consensus_queue.cc:260] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.395861  1619 tablet_bootstrap.cc:492] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a: No bootstrap required, opened a new log
I20260709 13:17:19.396374  1620 tablet_bootstrap.cc:492] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: No bootstrap required, opened a new log
I20260709 13:17:19.396431  1619 ts_tablet_manager.cc:1403] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 13:17:19.397651  1603 heartbeater.cc:499] Master 127.21.33.62:36745 was elected leader, sending a full tablet report...
I20260709 13:17:19.398208  1620 ts_tablet_manager.cc:1403] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:17:19.398284  1618 ts_tablet_manager.cc:1434] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:17:19.399904  1618 tablet_bootstrap.cc:492] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: Bootstrap starting.
I20260709 13:17:19.400543  1619 raft_consensus.cc:359] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.401434  1619 raft_consensus.cc:385] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.401820  1619 raft_consensus.cc:740] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.401824  1620 raft_consensus.cc:359] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.402951  1620 raft_consensus.cc:385] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.403347  1620 raft_consensus.cc:740] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.403062  1619 consensus_queue.cc:260] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.404268  1620 consensus_queue.cc:260] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.409256  1618 tablet_bootstrap.cc:654] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.416697  1454 heartbeater.cc:499] Master 127.21.33.62:36745 was elected leader, sending a full tablet report...
I20260709 13:17:19.417717  1529 heartbeater.cc:499] Master 127.21.33.62:36745 was elected leader, sending a full tablet report...
I20260709 13:17:19.418195  1620 ts_tablet_manager.cc:1434] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: Time spent starting tablet: real 0.019s	user 0.019s	sys 0.000s
I20260709 13:17:19.419351  1620 tablet_bootstrap.cc:492] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d: Bootstrap starting.
I20260709 13:17:19.420269  1619 ts_tablet_manager.cc:1434] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a: Time spent starting tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:17:19.421751  1619 tablet_bootstrap.cc:492] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a: Bootstrap starting.
I20260709 13:17:19.422610  1618 tablet_bootstrap.cc:492] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: No bootstrap required, opened a new log
I20260709 13:17:19.423295  1618 ts_tablet_manager.cc:1403] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 13:17:19.427289  1620 tablet_bootstrap.cc:654] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.427325  1618 raft_consensus.cc:359] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.428402  1618 raft_consensus.cc:385] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.428733  1618 raft_consensus.cc:740] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.429440  1619 tablet_bootstrap.cc:654] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.429637  1618 consensus_queue.cc:260] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.432543  1618 ts_tablet_manager.cc:1434] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:17:19.439525  1620 tablet_bootstrap.cc:492] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d: No bootstrap required, opened a new log
I20260709 13:17:19.439859  1619 tablet_bootstrap.cc:492] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a: No bootstrap required, opened a new log
I20260709 13:17:19.440305  1620 ts_tablet_manager.cc:1403] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:17:19.440497  1619 ts_tablet_manager.cc:1403] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.009s
I20260709 13:17:19.443945  1620 raft_consensus.cc:359] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.444337  1619 raft_consensus.cc:359] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.444871  1620 raft_consensus.cc:385] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.445248  1619 raft_consensus.cc:385] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.445362  1620 raft_consensus.cc:740] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.446039  1619 raft_consensus.cc:740] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.447165  1619 consensus_queue.cc:260] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.447010  1620 consensus_queue.cc:260] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.450085  1619 ts_tablet_manager.cc:1434] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:17:19.450356  1620 ts_tablet_manager.cc:1434] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:19.486111  1624 raft_consensus.cc:493] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.486871  1624 raft_consensus.cc:515] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.489694  1624 leader_election.cc:290] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695), 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137)
I20260709 13:17:19.499877  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44" candidate_uuid: "519294df0dc748d98a8e500cb3417403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79a71b7633445f4ac092687ee0cf69a" is_pre_election: true
I20260709 13:17:19.500721  1429 raft_consensus.cc:2468] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 519294df0dc748d98a8e500cb3417403 in term 0.
I20260709 13:17:19.501344  1504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44" candidate_uuid: "519294df0dc748d98a8e500cb3417403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80485e9f984a51a43e3055f85bb19d" is_pre_election: true
I20260709 13:17:19.502050  1541 leader_election.cc:304] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 519294df0dc748d98a8e500cb3417403, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:19.502177  1504 raft_consensus.cc:2468] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 519294df0dc748d98a8e500cb3417403 in term 0.
I20260709 13:17:19.503122  1624 raft_consensus.cc:2804] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:19.503508  1624 raft_consensus.cc:493] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.503846  1624 raft_consensus.cc:3060] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.510107  1624 raft_consensus.cc:515] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.511830  1624 leader_election.cc:290] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [CANDIDATE]: Term 1 election: Requested vote from peers d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695), 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137)
I20260709 13:17:19.512881  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44" candidate_uuid: "519294df0dc748d98a8e500cb3417403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
I20260709 13:17:19.512924  1504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44" candidate_uuid: "519294df0dc748d98a8e500cb3417403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
I20260709 13:17:19.513559  1429 raft_consensus.cc:3060] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.513700  1504 raft_consensus.cc:3060] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.517228  1626 raft_consensus.cc:493] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.517876  1626 raft_consensus.cc:515] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.520368  1429 raft_consensus.cc:2468] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 519294df0dc748d98a8e500cb3417403 in term 1.
I20260709 13:17:19.522284  1504 raft_consensus.cc:2468] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 519294df0dc748d98a8e500cb3417403 in term 1.
I20260709 13:17:19.522516  1626 leader_election.cc:290] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451), d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695)
I20260709 13:17:19.523420  1541 leader_election.cc:304] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 519294df0dc748d98a8e500cb3417403, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:19.524286  1624 raft_consensus.cc:2804] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:19.528462  1624 raft_consensus.cc:697] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 LEADER]: Becoming Leader. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Running, Role: LEADER
I20260709 13:17:19.529305  1624 consensus_queue.cc:237] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.534421  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519294df0dc748d98a8e500cb3417403" is_pre_election: true
I20260709 13:17:19.535367  1578 raft_consensus.cc:2468] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 0.
I20260709 13:17:19.537266  1470 leader_election.cc:304] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, 519294df0dc748d98a8e500cb3417403; no voters: 
I20260709 13:17:19.538203  1626 raft_consensus.cc:2804] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:19.538622  1626 raft_consensus.cc:493] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.538928  1626 raft_consensus.cc:3060] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.539006  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79a71b7633445f4ac092687ee0cf69a" is_pre_election: true
I20260709 13:17:19.539742  1429 raft_consensus.cc:2468] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 0.
I20260709 13:17:19.543531  1327 catalog_manager.cc:5697] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 reported cstate change: term changed from 0 to 1, leader changed from <none> to 519294df0dc748d98a8e500cb3417403 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "519294df0dc748d98a8e500cb3417403" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:19.546634  1626 raft_consensus.cc:515] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.549592  1626 leader_election.cc:290] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 election: Requested vote from peers 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451), d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695)
I20260709 13:17:19.550009  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519294df0dc748d98a8e500cb3417403"
I20260709 13:17:19.550310  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
I20260709 13:17:19.550729  1578 raft_consensus.cc:3060] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.550927  1429 raft_consensus.cc:3060] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.558591  1578 raft_consensus.cc:2468] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 1.
I20260709 13:17:19.559348  1429 raft_consensus.cc:2468] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 1.
I20260709 13:17:19.560055  1470 leader_election.cc:304] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, 519294df0dc748d98a8e500cb3417403; no voters: 
I20260709 13:17:19.561023  1626 raft_consensus.cc:2804] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:19.562387  1626 raft_consensus.cc:697] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Becoming Leader. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Running, Role: LEADER
I20260709 13:17:19.563191  1626 consensus_queue.cc:237] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:19.573321  1327 catalog_manager.cc:5697] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2). New cstate: current_term: 1 leader_uuid: "1e80485e9f984a51a43e3055f85bb19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.664199  1327 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44396:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:19.666407  1327 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:19.691777  1418 tablet_service.cc:1511] Processing CreateTablet for tablet 166563391f764edf83386363ae83522f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f320e62379e443a39731db9d74160491]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:19.693222  1418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 166563391f764edf83386363ae83522f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.693020  1494 tablet_service.cc:1511] Processing CreateTablet for tablet 166563391f764edf83386363ae83522f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f320e62379e443a39731db9d74160491]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:19.694335  1494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 166563391f764edf83386363ae83522f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.694377  1568 tablet_service.cc:1511] Processing CreateTablet for tablet 166563391f764edf83386363ae83522f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f320e62379e443a39731db9d74160491]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:19.695742  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 166563391f764edf83386363ae83522f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.696554  1419 tablet_service.cc:1511] Processing CreateTablet for tablet 5703a577f1664d0d97b82ebae4d8cc6e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f320e62379e443a39731db9d74160491]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:19.697922  1419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5703a577f1664d0d97b82ebae4d8cc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.712553  1619 tablet_bootstrap.cc:492] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a: Bootstrap starting.
I20260709 13:17:19.712553  1620 tablet_bootstrap.cc:492] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d: Bootstrap starting.
I20260709 13:17:19.715571  1493 tablet_service.cc:1511] Processing CreateTablet for tablet 5703a577f1664d0d97b82ebae4d8cc6e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f320e62379e443a39731db9d74160491]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:19.717023  1493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5703a577f1664d0d97b82ebae4d8cc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.717386  1618 tablet_bootstrap.cc:492] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403: Bootstrap starting.
I20260709 13:17:19.720471  1568 tablet_service.cc:1511] Processing CreateTablet for tablet 5703a577f1664d0d97b82ebae4d8cc6e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f320e62379e443a39731db9d74160491]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:19.721484  1620 tablet_bootstrap.cc:654] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.722021  1568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5703a577f1664d0d97b82ebae4d8cc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.725282  1618 tablet_bootstrap.cc:654] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.729199  1619 tablet_bootstrap.cc:654] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.735994  1618 tablet_bootstrap.cc:492] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403: No bootstrap required, opened a new log
I20260709 13:17:19.736613  1618 ts_tablet_manager.cc:1403] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 13:17:19.735443  1620 tablet_bootstrap.cc:492] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d: No bootstrap required, opened a new log
I20260709 13:17:19.744940  1618 raft_consensus.cc:359] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.745987  1620 ts_tablet_manager.cc:1403] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.012s
I20260709 13:17:19.748925  1619 tablet_bootstrap.cc:492] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a: No bootstrap required, opened a new log
I20260709 13:17:19.746202  1618 raft_consensus.cc:385] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.751190  1618 raft_consensus.cc:740] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.750232  1620 raft_consensus.cc:359] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.751972  1620 raft_consensus.cc:385] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.752243  1618 consensus_queue.cc:260] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.752943  1619 ts_tablet_manager.cc:1403] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.009s
I20260709 13:17:19.752319  1620 raft_consensus.cc:740] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.754817  1618 ts_tablet_manager.cc:1434] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:17:19.755714  1620 consensus_queue.cc:260] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.756487  1618 tablet_bootstrap.cc:492] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: Bootstrap starting.
W20260709 13:17:19.758066  1375 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:19.758277  1620 ts_tablet_manager.cc:1434] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:17:19.759495  1620 tablet_bootstrap.cc:492] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: Bootstrap starting.
I20260709 13:17:19.765354  1619 raft_consensus.cc:359] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.766393  1619 raft_consensus.cc:385] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.766819  1619 raft_consensus.cc:740] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.767414  1620 tablet_bootstrap.cc:654] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.767995  1619 consensus_queue.cc:260] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.770152  1618 tablet_bootstrap.cc:654] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.778386  1620 tablet_bootstrap.cc:492] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: No bootstrap required, opened a new log
I20260709 13:17:19.779075  1620 ts_tablet_manager.cc:1403] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:17:19.782477  1619 ts_tablet_manager.cc:1434] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:17:19.782660  1620 raft_consensus.cc:359] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.783607  1620 raft_consensus.cc:385] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.783965  1620 raft_consensus.cc:740] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.784965  1620 consensus_queue.cc:260] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.788053  1620 ts_tablet_manager.cc:1434] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:17:19.788493  1619 tablet_bootstrap.cc:492] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: Bootstrap starting.
I20260709 13:17:19.792766  1618 tablet_bootstrap.cc:492] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: No bootstrap required, opened a new log
I20260709 13:17:19.793591  1618 ts_tablet_manager.cc:1403] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.019s
I20260709 13:17:19.797575  1619 tablet_bootstrap.cc:654] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.797210  1618 raft_consensus.cc:359] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.798081  1618 raft_consensus.cc:385] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.798411  1618 raft_consensus.cc:740] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.799417  1618 consensus_queue.cc:260] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.802973  1618 ts_tablet_manager.cc:1434] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:19.807199  1626 raft_consensus.cc:493] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.807933  1626 raft_consensus.cc:515] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.807493  1619 tablet_bootstrap.cc:492] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: No bootstrap required, opened a new log
I20260709 13:17:19.809301  1619 ts_tablet_manager.cc:1403] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:17:19.810608  1626 leader_election.cc:290] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695), 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:19.811849  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519294df0dc748d98a8e500cb3417403" is_pre_election: true
I20260709 13:17:19.812610  1578 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 0.
I20260709 13:17:19.812503  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79a71b7633445f4ac092687ee0cf69a" is_pre_election: true
I20260709 13:17:19.813284  1619 raft_consensus.cc:359] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.813869  1470 leader_election.cc:304] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, 519294df0dc748d98a8e500cb3417403; no voters: 
I20260709 13:17:19.814733  1626 raft_consensus.cc:2804] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:19.814272  1619 raft_consensus.cc:385] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.815181  1626 raft_consensus.cc:493] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.815320  1619 raft_consensus.cc:740] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.815626  1626 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.816735  1619 consensus_queue.cc:260] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.818465  1429 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 0.
I20260709 13:17:19.820240  1619 ts_tablet_manager.cc:1434] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:17:19.825829  1626 raft_consensus.cc:515] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.828231  1626 leader_election.cc:290] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 election: Requested vote from peers d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695), 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:19.829137  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
I20260709 13:17:19.829404  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519294df0dc748d98a8e500cb3417403"
I20260709 13:17:19.829914  1429 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.829998  1578 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.839723  1578 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 1.
I20260709 13:17:19.839723  1429 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 1.
I20260709 13:17:19.841606  1467 leader_election.cc:304] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:19.842609  1626 raft_consensus.cc:2804] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:19.843144  1626 raft_consensus.cc:697] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Becoming Leader. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Running, Role: LEADER
I20260709 13:17:19.844053  1626 consensus_queue.cc:237] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.852634  1324 catalog_manager.cc:5697] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2). New cstate: current_term: 1 leader_uuid: "1e80485e9f984a51a43e3055f85bb19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.887506  1625 raft_consensus.cc:493] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.888146  1625 raft_consensus.cc:515] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.891347  1625 leader_election.cc:290] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137), 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:19.904546  1504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "166563391f764edf83386363ae83522f" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80485e9f984a51a43e3055f85bb19d" is_pre_election: true
I20260709 13:17:19.905390  1504 raft_consensus.cc:2468] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 0.
I20260709 13:17:19.906455  1395 leader_election.cc:304] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:19.906646  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "166563391f764edf83386363ae83522f" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519294df0dc748d98a8e500cb3417403" is_pre_election: true
I20260709 13:17:19.907336  1578 raft_consensus.cc:2468] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 0.
I20260709 13:17:19.907353  1625 raft_consensus.cc:2804] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:19.907912  1625 raft_consensus.cc:493] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.908362  1625 raft_consensus.cc:3060] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.919376  1625 raft_consensus.cc:515] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.921900  1625 leader_election.cc:290] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 1 election: Requested vote from peers 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137), 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:19.922988  1504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "166563391f764edf83386363ae83522f" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
I20260709 13:17:19.923547  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "166563391f764edf83386363ae83522f" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519294df0dc748d98a8e500cb3417403"
I20260709 13:17:19.923772  1504 raft_consensus.cc:3060] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.924214  1578 raft_consensus.cc:3060] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:19.930900  1504 raft_consensus.cc:2468] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 1.
I20260709 13:17:19.932420  1578 raft_consensus.cc:2468] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 1.
I20260709 13:17:19.932485  1395 leader_election.cc:304] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:19.934115  1625 raft_consensus.cc:2804] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:19.935608  1625 raft_consensus.cc:697] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 1 LEADER]: Becoming Leader. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Running, Role: LEADER
I20260709 13:17:19.936514  1625 consensus_queue.cc:237] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:19.945375  1324 catalog_manager.cc:5697] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a reported cstate change: term changed from 0 to 1, leader changed from <none> to d79a71b7633445f4ac092687ee0cf69a (127.21.33.1). New cstate: current_term: 1 leader_uuid: "d79a71b7633445f4ac092687ee0cf69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:20.007479  1504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a"
dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
mode: GRACEFUL
new_leader_uuid: "d79a71b7633445f4ac092687ee0cf69a"
 from {username='slave'} at 127.0.0.1:57420
I20260709 13:17:20.008292  1504 raft_consensus.cc:606] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Received request to transfer leadership to TS d79a71b7633445f4ac092687ee0cf69a
I20260709 13:17:20.025854  1578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44"
dest_uuid: "519294df0dc748d98a8e500cb3417403"
mode: GRACEFUL
new_leader_uuid: "1e80485e9f984a51a43e3055f85bb19d"
 from {username='slave'} at 127.0.0.1:44234
I20260709 13:17:20.026490  1578 raft_consensus.cc:606] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 LEADER]: Received request to transfer leadership to TS 1e80485e9f984a51a43e3055f85bb19d
I20260709 13:17:20.037891  1504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e"
dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
mode: GRACEFUL
new_leader_uuid: "d79a71b7633445f4ac092687ee0cf69a"
 from {username='slave'} at 127.0.0.1:57420
I20260709 13:17:20.038578  1504 raft_consensus.cc:606] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Received request to transfer leadership to TS d79a71b7633445f4ac092687ee0cf69a
I20260709 13:17:20.064910  1624 consensus_queue.cc:1048] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:20.075866  1654 raft_consensus.cc:993] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: : Instructing follower 1e80485e9f984a51a43e3055f85bb19d to start an election
I20260709 13:17:20.077637  1624 raft_consensus.cc:1081] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 LEADER]: Signalling peer 1e80485e9f984a51a43e3055f85bb19d to start an election
I20260709 13:17:20.080572  1503 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44"
dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
 from {username='slave'} at 127.0.0.1:57398
I20260709 13:17:20.084764  1503 raft_consensus.cc:493] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.085282  1503 raft_consensus.cc:3060] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.088011  1624 consensus_queue.cc:1048] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.097576  1503 raft_consensus.cc:515] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.105376  1503 leader_election.cc:290] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 2 election: Requested vote from peers d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695), 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:20.107261  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
I20260709 13:17:20.108224  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6370e807c6e34f289bfc57d82de7ee44" candidate_uuid: "1e80485e9f984a51a43e3055f85bb19d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "519294df0dc748d98a8e500cb3417403"
I20260709 13:17:20.108482  1429 raft_consensus.cc:3060] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.109223  1578 raft_consensus.cc:3055] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:20.109670  1578 raft_consensus.cc:740] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Running, Role: LEADER
I20260709 13:17:20.110543  1578 consensus_queue.cc:260] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.111749  1578 raft_consensus.cc:3060] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.116719  1429 raft_consensus.cc:2468] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 2.
I20260709 13:17:20.118641  1467 leader_election.cc:304] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:20.119663  1626 raft_consensus.cc:2804] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:20.123207  1578 raft_consensus.cc:2468] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e80485e9f984a51a43e3055f85bb19d in term 2.
I20260709 13:17:20.125919  1626 raft_consensus.cc:697] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 2 LEADER]: Becoming Leader. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Running, Role: LEADER
I20260709 13:17:20.126720  1660 consensus_queue.cc:1048] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.128300  1626 consensus_queue.cc:237] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.138394  1327 catalog_manager.cc:5697] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d reported cstate change: term changed from 1 to 2, leader changed from 519294df0dc748d98a8e500cb3417403 (127.21.33.3) to 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2). New cstate: current_term: 2 leader_uuid: "1e80485e9f984a51a43e3055f85bb19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:20.143240  1634 raft_consensus.cc:993] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: : Instructing follower d79a71b7633445f4ac092687ee0cf69a to start an election
I20260709 13:17:20.143920  1634 raft_consensus.cc:1081] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Signalling peer d79a71b7633445f4ac092687ee0cf69a to start an election
I20260709 13:17:20.146271  1428 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a"
dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
 from {username='slave'} at 127.0.0.1:52702
I20260709 13:17:20.146915  1428 raft_consensus.cc:493] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.147459  1428 raft_consensus.cc:3060] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.150314  1626 consensus_queue.cc:1048] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.173168  1428 raft_consensus.cc:515] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:20.177840  1577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "519294df0dc748d98a8e500cb3417403"
I20260709 13:17:20.178620  1577 raft_consensus.cc:3060] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.184922  1503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecdf4444f4e4221aa01a6e894bf285a" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
I20260709 13:17:20.185642  1503 raft_consensus.cc:3055] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:20.182147  1428 leader_election.cc:290] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 2 election: Requested vote from peers 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451), 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137)
I20260709 13:17:20.185999  1503 raft_consensus.cc:740] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Running, Role: LEADER
I20260709 13:17:20.186965  1503 consensus_queue.cc:260] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:20.188199  1577 raft_consensus.cc:2468] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 2.
I20260709 13:17:20.188696  1503 raft_consensus.cc:3060] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.189714  1395 leader_election.cc:304] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 519294df0dc748d98a8e500cb3417403, d79a71b7633445f4ac092687ee0cf69a; no voters: 
I20260709 13:17:20.190714  1625 raft_consensus.cc:2804] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:20.191186  1625 raft_consensus.cc:697] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Becoming Leader. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Running, Role: LEADER
I20260709 13:17:20.192029  1625 consensus_queue.cc:237] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } }
I20260709 13:17:20.202881  1323 catalog_manager.cc:5697] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a reported cstate change: term changed from 1 to 2, leader changed from 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2) to d79a71b7633445f4ac092687ee0cf69a (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d79a71b7633445f4ac092687ee0cf69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:20.206907  1503 raft_consensus.cc:2468] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 2.
I20260709 13:17:20.304965  1662 consensus_queue.cc:1048] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:20.314425  1626 raft_consensus.cc:993] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: : Instructing follower d79a71b7633445f4ac092687ee0cf69a to start an election
I20260709 13:17:20.315346  1626 raft_consensus.cc:1081] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Signalling peer d79a71b7633445f4ac092687ee0cf69a to start an election
I20260709 13:17:20.319005  1429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e"
dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
 from {username='slave'} at 127.0.0.1:52702
I20260709 13:17:20.319782  1429 raft_consensus.cc:493] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.320247  1429 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.321851  1626 consensus_queue.cc:1048] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.330152  1429 raft_consensus.cc:515] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.333786  1503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
I20260709 13:17:20.334273  1578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "d79a71b7633445f4ac092687ee0cf69a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "519294df0dc748d98a8e500cb3417403"
I20260709 13:17:20.334508  1503 raft_consensus.cc:3055] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:20.335083  1503 raft_consensus.cc:740] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1e80485e9f984a51a43e3055f85bb19d, State: Running, Role: LEADER
I20260709 13:17:20.336047  1503 consensus_queue.cc:260] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.336695  1429 leader_election.cc:290] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 2 election: Requested vote from peers 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137), 519294df0dc748d98a8e500cb3417403 (127.21.33.3:42451)
I20260709 13:17:20.337057  1503 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.347774  1503 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d79a71b7633445f4ac092687ee0cf69a in term 2.
I20260709 13:17:20.349475  1395 leader_election.cc:304] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, d79a71b7633445f4ac092687ee0cf69a; no voters: 519294df0dc748d98a8e500cb3417403
I20260709 13:17:20.350898  1625 raft_consensus.cc:2804] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:20.351421  1625 raft_consensus.cc:697] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Becoming Leader. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Running, Role: LEADER
I20260709 13:17:20.352239  1625 consensus_queue.cc:237] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.360870  1327 catalog_manager.cc:5697] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a reported cstate change: term changed from 1 to 2, leader changed from 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2) to d79a71b7633445f4ac092687ee0cf69a (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d79a71b7633445f4ac092687ee0cf69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:20.393383  1625 consensus_queue.cc:1048] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.408309  1625 consensus_queue.cc:1048] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.444139 21636 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:17:20.446170 21636 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:17:20.446674 21636 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:17:20.463275  1429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e"
dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
mode: GRACEFUL
new_leader_uuid: "519294df0dc748d98a8e500cb3417403"
 from {username='slave'} at 127.0.0.1:52708
I20260709 13:17:20.463989  1429 raft_consensus.cc:606] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Received request to transfer leadership to TS 519294df0dc748d98a8e500cb3417403
I20260709 13:17:20.465392 21636 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:17:20.593945  1577 raft_consensus.cc:1275] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Refusing update from remote peer d79a71b7633445f4ac092687ee0cf69a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.595722  1667 consensus_queue.cc:1048] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.599576  1503 raft_consensus.cc:1275] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Refusing update from remote peer d79a71b7633445f4ac092687ee0cf69a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.601301  1667 consensus_queue.cc:1048] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.633121  1577 raft_consensus.cc:1275] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Refusing update from remote peer 1e80485e9f984a51a43e3055f85bb19d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.634940  1626 consensus_queue.cc:1048] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.644263  1429 raft_consensus.cc:1275] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Refusing update from remote peer 1e80485e9f984a51a43e3055f85bb19d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.646081  1626 consensus_queue.cc:1048] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:20.872036  1577 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:20.878328  1577 raft_consensus.cc:1275] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Refusing update from remote peer d79a71b7633445f4ac092687ee0cf69a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.879908  1650 consensus_queue.cc:1048] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.886897  1667 raft_consensus.cc:993] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: : Instructing follower 519294df0dc748d98a8e500cb3417403 to start an election
I20260709 13:17:20.887483  1667 raft_consensus.cc:1081] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Signalling peer 519294df0dc748d98a8e500cb3417403 to start an election
I20260709 13:17:20.889297  1578 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e"
dest_uuid: "519294df0dc748d98a8e500cb3417403"
 from {username='slave'} at 127.0.0.1:44224
I20260709 13:17:20.891829  1503 raft_consensus.cc:1275] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Refusing update from remote peer d79a71b7633445f4ac092687ee0cf69a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.891860  1578 raft_consensus.cc:493] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.893352  1578 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.893761  1667 consensus_queue.cc:1048] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.902942  1578 raft_consensus.cc:515] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.907102  1429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "519294df0dc748d98a8e500cb3417403" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d79a71b7633445f4ac092687ee0cf69a"
I20260709 13:17:20.907747  1578 leader_election.cc:290] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [CANDIDATE]: Term 3 election: Requested vote from peers d79a71b7633445f4ac092687ee0cf69a (127.21.33.1:44695), 1e80485e9f984a51a43e3055f85bb19d (127.21.33.2:40137)
I20260709 13:17:20.908030  1429 raft_consensus.cc:3055] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:20.908552  1429 raft_consensus.cc:740] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d79a71b7633445f4ac092687ee0cf69a, State: Running, Role: LEADER
I20260709 13:17:20.909693  1503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5703a577f1664d0d97b82ebae4d8cc6e" candidate_uuid: "519294df0dc748d98a8e500cb3417403" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e80485e9f984a51a43e3055f85bb19d"
I20260709 13:17:20.910300  1503 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.910961  1429 consensus_queue.cc:260] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.912606  1429 raft_consensus.cc:3060] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.920543  1503 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 519294df0dc748d98a8e500cb3417403 in term 3.
I20260709 13:17:20.921103  1429 raft_consensus.cc:2468] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 519294df0dc748d98a8e500cb3417403 in term 3.
I20260709 13:17:20.922274  1544 leader_election.cc:304] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e80485e9f984a51a43e3055f85bb19d, 519294df0dc748d98a8e500cb3417403; no voters: 
I20260709 13:17:20.923146  1624 raft_consensus.cc:2804] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:20.924158  1624 raft_consensus.cc:697] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 3 LEADER]: Becoming Leader. State: Replica: 519294df0dc748d98a8e500cb3417403, State: Running, Role: LEADER
I20260709 13:17:20.925014  1624 consensus_queue.cc:237] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } }
I20260709 13:17:20.932085  1324 catalog_manager.cc:5697] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 reported cstate change: term changed from 2 to 3, leader changed from d79a71b7633445f4ac092687ee0cf69a (127.21.33.1) to 519294df0dc748d98a8e500cb3417403 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "519294df0dc748d98a8e500cb3417403" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d79a71b7633445f4ac092687ee0cf69a" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e80485e9f984a51a43e3055f85bb19d" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "519294df0dc748d98a8e500cb3417403" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 42451 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.116168 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
I20260709 13:17:21.142334 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:21.143332 21636 tablet_replica.cc:333] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a: stopping tablet replica
I20260709 13:17:21.143968 21636 raft_consensus.cc:2243] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.144418 21636 raft_consensus.cc:2272] T 5703a577f1664d0d97b82ebae4d8cc6e P d79a71b7633445f4ac092687ee0cf69a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.147470 21636 tablet_replica.cc:333] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a: stopping tablet replica
I20260709 13:17:21.148032 21636 raft_consensus.cc:2243] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.148856 21636 raft_consensus.cc:2272] T 166563391f764edf83386363ae83522f P d79a71b7633445f4ac092687ee0cf69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.151427 21636 tablet_replica.cc:333] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a: stopping tablet replica
I20260709 13:17:21.151983 21636 raft_consensus.cc:2243] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.152827 21636 raft_consensus.cc:2272] T aecdf4444f4e4221aa01a6e894bf285a P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.155507 21636 tablet_replica.cc:333] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a: stopping tablet replica
I20260709 13:17:21.156021 21636 raft_consensus.cc:2243] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.156450 21636 raft_consensus.cc:2272] T 6370e807c6e34f289bfc57d82de7ee44 P d79a71b7633445f4ac092687ee0cf69a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.184137 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
I20260709 13:17:21.200438 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
I20260709 13:17:21.225029 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:21.225883 21636 tablet_replica.cc:333] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d: stopping tablet replica
I20260709 13:17:21.226524 21636 raft_consensus.cc:2243] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.227077 21636 raft_consensus.cc:2272] T 5703a577f1664d0d97b82ebae4d8cc6e P 1e80485e9f984a51a43e3055f85bb19d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.229802 21636 tablet_replica.cc:333] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d: stopping tablet replica
I20260709 13:17:21.230521 21636 raft_consensus.cc:2243] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.231112 21636 raft_consensus.cc:2272] T 166563391f764edf83386363ae83522f P 1e80485e9f984a51a43e3055f85bb19d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.233836 21636 tablet_replica.cc:333] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d: stopping tablet replica
I20260709 13:17:21.234433 21636 raft_consensus.cc:2243] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.235411 21636 raft_consensus.cc:2272] T 6370e807c6e34f289bfc57d82de7ee44 P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.237890 21636 tablet_replica.cc:333] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d: stopping tablet replica
I20260709 13:17:21.238422 21636 raft_consensus.cc:2243] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.238890 21636 raft_consensus.cc:2272] T aecdf4444f4e4221aa01a6e894bf285a P 1e80485e9f984a51a43e3055f85bb19d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.254114 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:17:21.269013 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
I20260709 13:17:21.293598 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:21.294740 21636 tablet_replica.cc:333] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: stopping tablet replica
I20260709 13:17:21.295504 21636 raft_consensus.cc:2243] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.296267 21636 pending_rounds.cc:70] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: Trying to abort 1 pending ops.
I20260709 13:17:21.296496 21636 pending_rounds.cc:77] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305638055637221376 op_type: NO_OP noop_request { }
I20260709 13:17:21.296837 21636 raft_consensus.cc:2889] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:21.297153 21636 raft_consensus.cc:2272] T 5703a577f1664d0d97b82ebae4d8cc6e P 519294df0dc748d98a8e500cb3417403 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.300079 21636 tablet_replica.cc:333] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403: stopping tablet replica
I20260709 13:17:21.300746 21636 raft_consensus.cc:2243] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.301242 21636 raft_consensus.cc:2272] T 166563391f764edf83386363ae83522f P 519294df0dc748d98a8e500cb3417403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.305979 21636 tablet_replica.cc:333] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403: stopping tablet replica
I20260709 13:17:21.306816 21636 raft_consensus.cc:2243] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.307395 21636 raft_consensus.cc:2272] T 6370e807c6e34f289bfc57d82de7ee44 P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.310038 21636 tablet_replica.cc:333] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403: stopping tablet replica
I20260709 13:17:21.310678 21636 raft_consensus.cc:2243] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.311501 21636 raft_consensus.cc:2272] T aecdf4444f4e4221aa01a6e894bf285a P 519294df0dc748d98a8e500cb3417403 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.328099 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:17:21.342715 21636 master.cc:562] Master@127.21.33.62:36745 shutting down...
I20260709 13:17:21.366783 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.367513 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.367887 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6a1ad7b2afeb44d3b27a6e3480021fef: stopping tablet replica
I20260709 13:17:21.388954 21636 master.cc:584] Master@127.21.33.62:36745 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2820 ms)
[----------] 6 tests from LeaderRebalancerTest (411400 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:17:21.423146 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:36851
I20260709 13:17:21.424350 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:21.430307  1675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.430970  1676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.431982  1678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:21.432559 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:21.433457 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:21.433704 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:21.433917 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603041433879 us; error 0 us; skew 500 ppm
I20260709 13:17:21.434481 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:21.438385 21636 webserver.cc:533] Webserver started at http://127.21.33.62:36945/ using document root <none> and password file <none>
I20260709 13:17:21.439420 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:21.439751 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:21.440150 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:21.442303 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "4f686b4594bf4d02a2983c96c17af8be"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.452922 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.000s	sys 0.008s
I20260709 13:17:21.458922  1683 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.460170 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:17:21.460649 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "4f686b4594bf4d02a2983c96c17af8be"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.461097 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:21.480325 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:21.482271 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:21.538790 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:36851
I20260709 13:17:21.538939  1734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:36851 every 8 connection(s)
I20260709 13:17:21.544040  1735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:21.558873  1735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be: Bootstrap starting.
I20260709 13:17:21.564857  1735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.570142  1735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be: No bootstrap required, opened a new log
I20260709 13:17:21.572796  1735 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f686b4594bf4d02a2983c96c17af8be" member_type: VOTER }
I20260709 13:17:21.573271  1735 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.573689  1735 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f686b4594bf4d02a2983c96c17af8be, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.574380  1735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f686b4594bf4d02a2983c96c17af8be" member_type: VOTER }
I20260709 13:17:21.574990  1735 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:21.575232  1735 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:21.575565  1735 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.581936  1735 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f686b4594bf4d02a2983c96c17af8be" member_type: VOTER }
I20260709 13:17:21.582612  1735 leader_election.cc:304] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f686b4594bf4d02a2983c96c17af8be; no voters: 
I20260709 13:17:21.584141  1735 leader_election.cc:290] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:21.584746  1738 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.586825  1738 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 1 LEADER]: Becoming Leader. State: Replica: 4f686b4594bf4d02a2983c96c17af8be, State: Running, Role: LEADER
I20260709 13:17:21.587756  1738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f686b4594bf4d02a2983c96c17af8be" member_type: VOTER }
I20260709 13:17:21.588294  1735 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:21.592941  1740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f686b4594bf4d02a2983c96c17af8be. Latest consensus state: current_term: 1 leader_uuid: "4f686b4594bf4d02a2983c96c17af8be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f686b4594bf4d02a2983c96c17af8be" member_type: VOTER } }
I20260709 13:17:21.593863  1740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:21.592896  1739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f686b4594bf4d02a2983c96c17af8be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f686b4594bf4d02a2983c96c17af8be" member_type: VOTER } }
I20260709 13:17:21.594595  1739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:21.599337  1744 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:21.605914  1744 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:21.607931 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:21.617020  1744 catalog_manager.cc:1383] Generated new cluster ID: a43739d3de5a4066adb046c3e804f170
I20260709 13:17:21.617384  1744 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:21.644898  1744 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:21.646435  1744 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:21.662550  1744 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be: Generated new TSK 0
I20260709 13:17:21.663702  1744 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:21.675595 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:21.684154  1756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.686131  1757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.687201  1759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:21.688202 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:21.689324 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:21.689640 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:21.689821 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603041689799 us; error 0 us; skew 500 ppm
I20260709 13:17:21.690433 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:21.694056 21636 webserver.cc:533] Webserver started at http://127.21.33.1:37777/ using document root <none> and password file <none>
I20260709 13:17:21.695048 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:21.695307 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:21.695653 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:21.696997 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "d4d2de778a80423c8ea7230963568a1c"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.702525 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:21.706895  1764 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.708211 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:21.708630 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "d4d2de778a80423c8ea7230963568a1c"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.708976 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:21.737174 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:21.738607 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:21.740203 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:21.743139 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:21.743386 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.743690 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:21.743877 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.797945 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:38529
I20260709 13:17:21.798072  1826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:38529 every 8 connection(s)
I20260709 13:17:21.803608 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:21.813202  1831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.813603  1832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:21.818689 21636 server_base.cc:1061] running on GCE node
W20260709 13:17:21.818948  1834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:21.820758  1827 heartbeater.cc:344] Connected to a master server at 127.21.33.62:36851
I20260709 13:17:21.821224  1827 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:21.821254 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:21.821663 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:21.821852 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603041821836 us; error 0 us; skew 500 ppm
I20260709 13:17:21.822248  1827 heartbeater.cc:507] Master 127.21.33.62:36851 requested a full tablet report, sending...
I20260709 13:17:21.822553 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:21.824790  1700 ts_manager.cc:194] Registered new tserver with Master: d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529)
I20260709 13:17:21.825662 21636 webserver.cc:533] Webserver started at http://127.21.33.2:37279/ using document root <none> and password file <none>
I20260709 13:17:21.826246 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:21.826465 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:21.826746 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:21.827293  1700 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41802
I20260709 13:17:21.828023 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "e91ba25a6a344f70aef0384659b3fd73"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.834686 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:17:21.838693  1839 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.839587 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:21.839907 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "e91ba25a6a344f70aef0384659b3fd73"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.840248 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:21.854844 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:21.856194 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:21.857963 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:21.861090 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:21.861354 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.861653 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:21.861835 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.908571 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:40885
I20260709 13:17:21.908748  1901 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:40885 every 8 connection(s)
I20260709 13:17:21.914249 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:21.925451  1905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.925801  1906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:21.929217  1909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:21.929401 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:21.930980 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:21.931285 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:21.931495 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603041931478 us; error 0 us; skew 500 ppm
I20260709 13:17:21.932199  1902 heartbeater.cc:344] Connected to a master server at 127.21.33.62:36851
I20260709 13:17:21.932232 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:21.932797  1902 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:21.933810  1902 heartbeater.cc:507] Master 127.21.33.62:36851 requested a full tablet report, sending...
I20260709 13:17:21.935527 21636 webserver.cc:533] Webserver started at http://127.21.33.3:38457/ using document root <none> and password file <none>
I20260709 13:17:21.936293 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:21.936509 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:21.936393  1700 ts_manager.cc:194] Registered new tserver with Master: e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:21.936985 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:21.938236  1700 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41810
I20260709 13:17:21.938282 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "f9e8deca048d4039851f55fe03299952"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.943565 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:17:21.947345  1913 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.948705 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:17:21.949080 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "f9e8deca048d4039851f55fe03299952"
format_stamp: "Formatted at 2026-07-09 13:17:21 on dist-test-slave-p8rg"
I20260709 13:17:21.949591 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:21.975116 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:21.976466 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:21.977965 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:21.980744 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:21.981010 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:21.981276 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:21.981452 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.034762 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:41281
I20260709 13:17:22.034874  1975 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:41281 every 8 connection(s)
I20260709 13:17:22.051502  1976 heartbeater.cc:344] Connected to a master server at 127.21.33.62:36851
I20260709 13:17:22.052068  1976 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:22.053191  1976 heartbeater.cc:507] Master 127.21.33.62:36851 requested a full tablet report, sending...
I20260709 13:17:22.056008  1700 ts_manager.cc:194] Registered new tserver with Master: f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.056684 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017988464s
I20260709 13:17:22.057983  1700 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41824
I20260709 13:17:22.088999  1700 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41840:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:22.092073  1700 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:22.194073  1867 tablet_service.cc:1511] Processing CreateTablet for tablet 5d190db5d02f418d80ee52a83ea60119 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:22.195688  1867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d190db5d02f418d80ee52a83ea60119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.195817  1866 tablet_service.cc:1511] Processing CreateTablet for tablet 7563b7d4737943d6987c8a3451d85a75 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:22.197481  1866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7563b7d4737943d6987c8a3451d85a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.198870  1865 tablet_service.cc:1511] Processing CreateTablet for tablet 484797364d6049a68a06966fb4660242 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:22.211048  1861 tablet_service.cc:1511] Processing CreateTablet for tablet 3dce46c69af941cc81ab6459e9e3424f (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:22.211658  1865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 484797364d6049a68a06966fb4660242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.212416  1861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dce46c69af941cc81ab6459e9e3424f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.200299  1940 tablet_service.cc:1511] Processing CreateTablet for tablet 7563b7d4737943d6987c8a3451d85a75 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:22.213703  1940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7563b7d4737943d6987c8a3451d85a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.213418  1792 tablet_service.cc:1511] Processing CreateTablet for tablet 5d190db5d02f418d80ee52a83ea60119 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:22.214848  1792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d190db5d02f418d80ee52a83ea60119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.200760  1864 tablet_service.cc:1511] Processing CreateTablet for tablet 025ec36b5ae54e11a43c4f78d870c2fc (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:22.218649  1864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025ec36b5ae54e11a43c4f78d870c2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.202565  1863 tablet_service.cc:1511] Processing CreateTablet for tablet 0fbb6f2fcba04b6b9f78d2db58f523a8 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:22.221596  1863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fbb6f2fcba04b6b9f78d2db58f523a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.223119  1991 tablet_bootstrap.cc:492] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.202797  1939 tablet_service.cc:1511] Processing CreateTablet for tablet 484797364d6049a68a06966fb4660242 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:22.226434  1939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 484797364d6049a68a06966fb4660242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.204308  1862 tablet_service.cc:1511] Processing CreateTablet for tablet e1c65f83f85d44d1b7525a643c8fe3e8 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:22.230620  1862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c65f83f85d44d1b7525a643c8fe3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.231796  1991 tablet_bootstrap.cc:654] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.206164  1941 tablet_service.cc:1511] Processing CreateTablet for tablet 5d190db5d02f418d80ee52a83ea60119 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:22.234601  1941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d190db5d02f418d80ee52a83ea60119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.206439  1938 tablet_service.cc:1511] Processing CreateTablet for tablet 025ec36b5ae54e11a43c4f78d870c2fc (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:22.238713  1938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025ec36b5ae54e11a43c4f78d870c2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.207660  1935 tablet_service.cc:1511] Processing CreateTablet for tablet 3dce46c69af941cc81ab6459e9e3424f (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:22.242705  1935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dce46c69af941cc81ab6459e9e3424f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.244139  1991 tablet_bootstrap.cc:492] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.244797  1991 ts_tablet_manager.cc:1403] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.012s
I20260709 13:17:22.208621  1860 tablet_service.cc:1511] Processing CreateTablet for tablet ff66ac876d0546a19d3f5002d1f77848 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:22.248335  1991 raft_consensus.cc:359] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.208793  1937 tablet_service.cc:1511] Processing CreateTablet for tablet 0fbb6f2fcba04b6b9f78d2db58f523a8 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:22.249717  1991 raft_consensus.cc:385] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.250092  1991 raft_consensus.cc:740] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.250788  1937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fbb6f2fcba04b6b9f78d2db58f523a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.251098  1991 consensus_queue.cc:260] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.206164  1936 tablet_service.cc:1511] Processing CreateTablet for tablet e1c65f83f85d44d1b7525a643c8fe3e8 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:22.254839  1936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c65f83f85d44d1b7525a643c8fe3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.265982  1790 tablet_service.cc:1511] Processing CreateTablet for tablet 484797364d6049a68a06966fb4660242 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:22.267810  1790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 484797364d6049a68a06966fb4660242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.271786  1991 ts_tablet_manager.cc:1434] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.027s	user 0.012s	sys 0.006s
I20260709 13:17:22.273339  1902 heartbeater.cc:499] Master 127.21.33.62:36851 was elected leader, sending a full tablet report...
I20260709 13:17:22.271243  1791 tablet_service.cc:1511] Processing CreateTablet for tablet 7563b7d4737943d6987c8a3451d85a75 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:22.275490  1791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7563b7d4737943d6987c8a3451d85a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.275703  1787 tablet_service.cc:1511] Processing CreateTablet for tablet e1c65f83f85d44d1b7525a643c8fe3e8 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:22.277189  1787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c65f83f85d44d1b7525a643c8fe3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.276772  1788 tablet_service.cc:1511] Processing CreateTablet for tablet 0fbb6f2fcba04b6b9f78d2db58f523a8 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:22.279033  1788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fbb6f2fcba04b6b9f78d2db58f523a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.279182  1786 tablet_service.cc:1511] Processing CreateTablet for tablet 3dce46c69af941cc81ab6459e9e3424f (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:22.280682  1786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dce46c69af941cc81ab6459e9e3424f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.281893  1785 tablet_service.cc:1511] Processing CreateTablet for tablet ff66ac876d0546a19d3f5002d1f77848 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:22.283398  1785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff66ac876d0546a19d3f5002d1f77848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.249061  1860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff66ac876d0546a19d3f5002d1f77848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.271243  1789 tablet_service.cc:1511] Processing CreateTablet for tablet 025ec36b5ae54e11a43c4f78d870c2fc (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:22.298105  1789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025ec36b5ae54e11a43c4f78d870c2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.305573  1991 tablet_bootstrap.cc:492] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.329203  1940 tablet_service.cc:1511] Processing CreateTablet for tablet ff66ac876d0546a19d3f5002d1f77848 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:22.330588  1995 tablet_bootstrap.cc:492] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.330814  1940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff66ac876d0546a19d3f5002d1f77848. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.334789  1784 tablet_service.cc:1511] Processing CreateTablet for tablet 011ac8a199d24d52a117549237f0d442 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:22.336328  1784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011ac8a199d24d52a117549237f0d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.343524  1865 tablet_service.cc:1511] Processing CreateTablet for tablet 011ac8a199d24d52a117549237f0d442 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:22.345062  1865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011ac8a199d24d52a117549237f0d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.358675  1935 tablet_service.cc:1511] Processing CreateTablet for tablet 011ac8a199d24d52a117549237f0d442 (DEFAULT_TABLE table=test-workload [id=84654db1e39342e8ae97222c375abe4b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:22.361488  1935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 011ac8a199d24d52a117549237f0d442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.379714  1991 tablet_bootstrap.cc:654] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.388608  1992 tablet_bootstrap.cc:492] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.404201  1995 tablet_bootstrap.cc:654] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.419703  1992 tablet_bootstrap.cc:654] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.449874  1995 tablet_bootstrap.cc:492] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.450130  1991 tablet_bootstrap.cc:492] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.450623  1995 ts_tablet_manager.cc:1403] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.005s
I20260709 13:17:22.450852  1991 ts_tablet_manager.cc:1403] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.161s	user 0.014s	sys 0.031s
I20260709 13:17:22.453130  1992 tablet_bootstrap.cc:492] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.454289  1991 raft_consensus.cc:359] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.454535  1995 raft_consensus.cc:359] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.455262  1991 raft_consensus.cc:385] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.455356  1995 raft_consensus.cc:385] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.455765  1991 raft_consensus.cc:740] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.455834  1995 raft_consensus.cc:740] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.456611  1992 ts_tablet_manager.cc:1403] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.007s
I20260709 13:17:22.456668  1991 consensus_queue.cc:260] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.457154  1995 consensus_queue.cc:260] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.459321  1991 ts_tablet_manager.cc:1434] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:22.460500  1991 tablet_bootstrap.cc:492] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.467388  1991 tablet_bootstrap.cc:654] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.471256  1976 heartbeater.cc:499] Master 127.21.33.62:36851 was elected leader, sending a full tablet report...
I20260709 13:17:22.471748  1995 ts_tablet_manager.cc:1434] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.010s
I20260709 13:17:22.472934  1995 tablet_bootstrap.cc:492] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.474537  1992 raft_consensus.cc:359] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.475515  1992 raft_consensus.cc:385] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.475867  1992 raft_consensus.cc:740] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.476768  1992 consensus_queue.cc:260] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.480289  1995 tablet_bootstrap.cc:654] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.486366  1992 ts_tablet_manager.cc:1434] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.006s
I20260709 13:17:22.487411  1827 heartbeater.cc:499] Master 127.21.33.62:36851 was elected leader, sending a full tablet report...
I20260709 13:17:22.488060  1992 tablet_bootstrap.cc:492] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.489308  1991 tablet_bootstrap.cc:492] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.489916  1991 ts_tablet_manager.cc:1403] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.004s
I20260709 13:17:22.493602  1991 raft_consensus.cc:359] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.494608  1991 raft_consensus.cc:385] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.495054  1991 raft_consensus.cc:740] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.495818  1992 tablet_bootstrap.cc:654] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.496729  1991 consensus_queue.cc:260] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.503047  1991 ts_tablet_manager.cc:1434] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:17:22.504161  1991 tablet_bootstrap.cc:492] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.505296  1992 tablet_bootstrap.cc:492] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.506049  1992 ts_tablet_manager.cc:1403] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.003s
I20260709 13:17:22.509302  1992 raft_consensus.cc:359] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.510140  1992 raft_consensus.cc:385] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.510488  1992 raft_consensus.cc:740] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.511556  1992 consensus_queue.cc:260] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.513422  1999 raft_consensus.cc:493] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.514024  1999 raft_consensus.cc:515] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.514869  1992 ts_tablet_manager.cc:1434] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:22.516031  1992 tablet_bootstrap.cc:492] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.517424  1991 tablet_bootstrap.cc:654] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.521379  1999 leader_election.cc:290] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:22.537978  1992 tablet_bootstrap.cc:654] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.538259  1995 tablet_bootstrap.cc:492] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.538991  1995 ts_tablet_manager.cc:1403] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.015s
I20260709 13:17:22.543606  1995 raft_consensus.cc:359] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.544746  1995 raft_consensus.cc:385] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.545089  1995 raft_consensus.cc:740] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.546155  1995 consensus_queue.cc:260] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.549458  1995 ts_tablet_manager.cc:1434] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 13:17:22.550711  1995 tablet_bootstrap.cc:492] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.556985  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:22.559229  1995 tablet_bootstrap.cc:654] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:22.559991  1915 leader_election.cc:343] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.566082  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
W20260709 13:17:22.568881  1915 leader_election.cc:343] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.569633  1915 leader_election.cc:304] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:22.570996  1999 raft_consensus.cc:2749] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:22.593282  1991 tablet_bootstrap.cc:492] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.594324  1991 ts_tablet_manager.cc:1403] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.090s	user 0.046s	sys 0.033s
I20260709 13:17:22.598541  1991 raft_consensus.cc:359] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.599421  1991 raft_consensus.cc:385] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.599712  1991 raft_consensus.cc:740] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.600607  1992 tablet_bootstrap.cc:492] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.600545  1991 consensus_queue.cc:260] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.601393  1992 ts_tablet_manager.cc:1403] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.086s	user 0.018s	sys 0.001s
W20260709 13:17:22.601464  1748 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:22.603047  1991 ts_tablet_manager.cc:1434] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:17:22.604862  1991 tablet_bootstrap.cc:492] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.605397  1992 raft_consensus.cc:359] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.606268  1992 raft_consensus.cc:385] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.606618  1992 raft_consensus.cc:740] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.607100  1995 tablet_bootstrap.cc:492] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.607657  1995 ts_tablet_manager.cc:1403] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.006s
I20260709 13:17:22.607676  1992 consensus_queue.cc:260] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.611785  1995 raft_consensus.cc:359] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.612605  1995 raft_consensus.cc:385] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.612768  1991 tablet_bootstrap.cc:654] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.612986  1995 raft_consensus.cc:740] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.613960  1995 consensus_queue.cc:260] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.614519  1992 ts_tablet_manager.cc:1434] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:17:22.615851  1992 tablet_bootstrap.cc:492] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.616593  1995 ts_tablet_manager.cc:1434] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:17:22.620342  1995 tablet_bootstrap.cc:492] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.625058  1991 tablet_bootstrap.cc:492] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.625839  1991 ts_tablet_manager.cc:1403] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.003s
I20260709 13:17:22.628821  1995 tablet_bootstrap.cc:654] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.629431  1991 raft_consensus.cc:359] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.630244  1991 raft_consensus.cc:385] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.630505  1991 raft_consensus.cc:740] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.631397  1994 raft_consensus.cc:493] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.631517  1991 consensus_queue.cc:260] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.631979  1994 raft_consensus.cc:515] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.640480  1994 leader_election.cc:290] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281), d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529)
I20260709 13:17:22.640990  1992 tablet_bootstrap.cc:654] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.669229  1994 raft_consensus.cc:493] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.670173  1994 raft_consensus.cc:515] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.673249  1994 leader_election.cc:290] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.692327  1995 tablet_bootstrap.cc:492] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.692847  1991 ts_tablet_manager.cc:1434] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.067s	user 0.032s	sys 0.031s
I20260709 13:17:22.693238  1995 ts_tablet_manager.cc:1403] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.049s
I20260709 13:17:22.694209  1992 tablet_bootstrap.cc:492] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.694877  1992 ts_tablet_manager.cc:1403] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.079s	user 0.020s	sys 0.031s
I20260709 13:17:22.696759  1995 raft_consensus.cc:359] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.697762  1995 raft_consensus.cc:385] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.698109  1995 raft_consensus.cc:740] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.698330  1992 raft_consensus.cc:359] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.699198  1992 raft_consensus.cc:385] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.699126  1995 consensus_queue.cc:260] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.699680  1992 raft_consensus.cc:740] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.700732  1992 consensus_queue.cc:260] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.702081  1995 ts_tablet_manager.cc:1434] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:17:22.702198  1991 tablet_bootstrap.cc:492] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.703239  1995 tablet_bootstrap.cc:492] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.710054  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:22.710865  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c65f83f85d44d1b7525a643c8fe3e8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
W20260709 13:17:22.712061  1841 leader_election.cc:343] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:22.717424  1841 leader_election.cc:343] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.719208  1992 ts_tablet_manager.cc:1434] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:17:22.720214  1991 tablet_bootstrap.cc:654] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.722448  1995 tablet_bootstrap.cc:654] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.725019  1992 tablet_bootstrap.cc:492] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.731873  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c65f83f85d44d1b7525a643c8fe3e8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:22.731957  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:22.732839  1801 raft_consensus.cc:2468] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 0.
I20260709 13:17:22.733086  1802 raft_consensus.cc:2468] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 0.
I20260709 13:17:22.734382  1841 leader_election.cc:304] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.735632  2010 raft_consensus.cc:2804] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.736289  2010 raft_consensus.cc:493] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.736006  1841 leader_election.cc:304] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.736881  2010 raft_consensus.cc:3060] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.737735  1994 raft_consensus.cc:2804] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.738271  1994 raft_consensus.cc:493] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.738725  1994 raft_consensus.cc:3060] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.740639  1992 tablet_bootstrap.cc:654] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.748462  1991 tablet_bootstrap.cc:492] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.749096  1991 ts_tablet_manager.cc:1403] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.047s	user 0.026s	sys 0.017s
I20260709 13:17:22.749459  1999 raft_consensus.cc:493] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.750252  1994 raft_consensus.cc:515] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.750309  1999 raft_consensus.cc:515] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.755949  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:22.756744  1801 raft_consensus.cc:3060] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.756933  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
W20260709 13:17:22.760059  1915 leader_election.cc:343] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.755825  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:22.762276  1999 leader_election.cc:290] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
W20260709 13:17:22.764334  1841 leader_election.cc:343] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.770582  1994 leader_election.cc:290] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Requested vote from peers f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281), d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529)
I20260709 13:17:22.771221  2017 raft_consensus.cc:493] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.771379  1994 raft_consensus.cc:493] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.771821  2017 raft_consensus.cc:515] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.772010  1994 raft_consensus.cc:515] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.774559  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:22.775092  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:22.775265  1877 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:22.775818  1802 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 0.
I20260709 13:17:22.776592  1915 leader_election.cc:304] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:22.769567  1801 raft_consensus.cc:2468] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 1.
I20260709 13:17:22.777220  1841 leader_election.cc:304] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:22.777197  1995 tablet_bootstrap.cc:492] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.778196  1999 raft_consensus.cc:2804] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.778558  1995 ts_tablet_manager.cc:1403] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.076s	user 0.020s	sys 0.011s
I20260709 13:17:22.778635  1999 raft_consensus.cc:493] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.779178  1999 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.779655  1841 leader_election.cc:304] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.780565  1992 tablet_bootstrap.cc:492] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.781138  1992 ts_tablet_manager.cc:1403] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.029s
I20260709 13:17:22.781737  2018 raft_consensus.cc:2804] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.782229  2018 raft_consensus.cc:493] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.782554  2018 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.783465  1994 leader_election.cc:290] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.784471  1992 raft_consensus.cc:359] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.785001  1994 raft_consensus.cc:2804] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.785637  1992 raft_consensus.cc:385] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.786603  1992 raft_consensus.cc:740] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.786723  1994 raft_consensus.cc:697] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:22.787621  1992 consensus_queue.cc:260] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.790700  1992 ts_tablet_manager.cc:1434] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.005s
I20260709 13:17:22.790548  1991 raft_consensus.cc:359] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.791538  1991 raft_consensus.cc:385] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.790549  1995 raft_consensus.cc:359] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.791913  1991 raft_consensus.cc:740] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.792002  1992 tablet_bootstrap.cc:492] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.792271  1995 raft_consensus.cc:385] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.792815  1991 consensus_queue.cc:260] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.792856  1995 raft_consensus.cc:740] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.795363  1991 ts_tablet_manager.cc:1434] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.003s
I20260709 13:17:22.794993  1995 consensus_queue.cc:260] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.796551  1991 tablet_bootstrap.cc:492] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.798985  1999 raft_consensus.cc:515] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.799007  2018 raft_consensus.cc:515] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.800665  1992 tablet_bootstrap.cc:654] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.801587  2018 leader_election.cc:290] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.802394  2017 leader_election.cc:290] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.765363  2010 raft_consensus.cc:515] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.808048  1995 ts_tablet_manager.cc:1434] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.001s
I20260709 13:17:22.809639  2010 leader_election.cc:290] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.812126  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:22.812831  1877 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.812845  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:22.813643  1950 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 0.
I20260709 13:17:22.813903  1992 tablet_bootstrap.cc:492] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.814670  1992 ts_tablet_manager.cc:1403] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 13:17:22.815544  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:22.817435  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:22.820204  1802 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.821858  1801 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.827219  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:22.828989  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c65f83f85d44d1b7525a643c8fe3e8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
W20260709 13:17:22.829205  1841 leader_election.cc:343] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.829841  1799 raft_consensus.cc:3060] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.831259  1991 tablet_bootstrap.cc:654] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.832158  1801 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:22.818270  1992 raft_consensus.cc:359] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.819074  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:22.833878  1992 raft_consensus.cc:385] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.823163  2010 raft_consensus.cc:493] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.823375  1999 leader_election.cc:290] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:22.834609  1992 raft_consensus.cc:740] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.817947  1995 tablet_bootstrap.cc:492] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.834749  1915 leader_election.cc:304] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:22.835865  1992 consensus_queue.cc:260] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.837327  1999 raft_consensus.cc:2804] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.837909  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c65f83f85d44d1b7525a643c8fe3e8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:22.838981  1992 ts_tablet_manager.cc:1434] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
W20260709 13:17:22.839958  1841 leader_election.cc:343] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.840220  1992 tablet_bootstrap.cc:492] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.838147  1999 raft_consensus.cc:697] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:22.841812  1999 consensus_queue.cc:237] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.822861  1877 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:22.838758  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:22.834031  1950 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.848884  1992 tablet_bootstrap.cc:654] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.856097  1995 tablet_bootstrap.cc:654] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.833400  1994 consensus_queue.cc:237] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
W20260709 13:17:22.855949  1841 leader_election.cc:343] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.861837  1841 leader_election.cc:304] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.862226  1992 tablet_bootstrap.cc:492] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.834734  2010 raft_consensus.cc:515] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.863030  2017 raft_consensus.cc:2749] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:22.863171  1992 ts_tablet_manager.cc:1403] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.010s
I20260709 13:17:22.865216  2010 leader_election.cc:290] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.867839  1992 raft_consensus.cc:359] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.872658  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011ac8a199d24d52a117549237f0d442" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:22.879110  2001 raft_consensus.cc:493] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.875715  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011ac8a199d24d52a117549237f0d442" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:22.879647  2001 raft_consensus.cc:515] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
W20260709 13:17:22.881069  1841 leader_election.cc:343] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.881422  1992 raft_consensus.cc:385] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.881966  1992 raft_consensus.cc:740] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
W20260709 13:17:22.882267  1841 leader_election.cc:343] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.882905  1992 consensus_queue.cc:260] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.884436  1802 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 1.
I20260709 13:17:22.878232  1699 catalog_manager.cc:5697] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 0 to 1, leader changed from <none> to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:22.892511  1700 catalog_manager.cc:5697] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:22.894332  1992 ts_tablet_manager.cc:1434] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.007s
I20260709 13:17:22.882954  1841 leader_election.cc:304] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.895902  1992 tablet_bootstrap.cc:492] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:22.897004  1994 raft_consensus.cc:2749] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:22.894235  2001 leader_election.cc:290] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.904878  1841 leader_election.cc:304] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:22.906085  1995 tablet_bootstrap.cc:492] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.906688  1995 ts_tablet_manager.cc:1403] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.089s	user 0.016s	sys 0.006s
I20260709 13:17:22.908605  1994 raft_consensus.cc:2804] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.909394  1994 raft_consensus.cc:697] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:22.910364  1950 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 1.
I20260709 13:17:22.911494  1995 raft_consensus.cc:359] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.911304  1994 consensus_queue.cc:237] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.912276  1995 raft_consensus.cc:385] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.912679  1995 raft_consensus.cc:740] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.913411  1995 consensus_queue.cc:260] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.898725  1799 raft_consensus.cc:2468] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 1.
I20260709 13:17:22.922842  1995 ts_tablet_manager.cc:1434] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.005s
I20260709 13:17:22.924458  1995 tablet_bootstrap.cc:492] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.934947  1991 tablet_bootstrap.cc:492] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.935765  1991 ts_tablet_manager.cc:1403] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.139s	user 0.042s	sys 0.029s
I20260709 13:17:22.935552  1841 leader_election.cc:304] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.939320  1995 tablet_bootstrap.cc:654] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.944679  1994 raft_consensus.cc:2804] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.946439  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:22.945309  1991 raft_consensus.cc:359] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.947395  1994 raft_consensus.cc:697] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:22.948345  1991 raft_consensus.cc:385] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.950011  1991 raft_consensus.cc:740] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
W20260709 13:17:22.952095  1766 leader_election.cc:343] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.952477  1991 consensus_queue.cc:260] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.955621  1991 ts_tablet_manager.cc:1434] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:17:22.956864  1991 tablet_bootstrap.cc:492] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:22.951066  1994 consensus_queue.cc:237] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.964264  1992 tablet_bootstrap.cc:654] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.966109  1991 tablet_bootstrap.cc:654] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.969808  1995 tablet_bootstrap.cc:492] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:22.970595  1995 ts_tablet_manager.cc:1403] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.047s	user 0.001s	sys 0.015s
I20260709 13:17:22.974514  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:22.975481  2028 raft_consensus.cc:493] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.976224  1766 leader_election.cc:343] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.976053  2028 raft_consensus.cc:515] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.976922  1766 leader_election.cc:304] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c; no voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952
I20260709 13:17:22.976258  1995 raft_consensus.cc:359] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.978073  1995 raft_consensus.cc:385] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.978356  1995 raft_consensus.cc:740] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.978595  2001 raft_consensus.cc:2749] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:22.979153  2028 leader_election.cc:290] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:22.979297  1995 consensus_queue.cc:260] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.982117  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:22.983335  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:22.984061  1950 raft_consensus.cc:2468] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:22.985154  1995 ts_tablet_manager.cc:1434] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.008s
I20260709 13:17:22.985919  1999 raft_consensus.cc:493] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:22.986114  1766 leader_election.cc:343] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.986380  1995 tablet_bootstrap.cc:492] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:22.987242  1992 tablet_bootstrap.cc:492] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:22.986490  1999 raft_consensus.cc:515] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:22.988405  1992 ts_tablet_manager.cc:1403] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.093s	user 0.015s	sys 0.001s
I20260709 13:17:22.988286  1766 leader_election.cc:304] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:22.990175  2020 raft_consensus.cc:493] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.990618  1999 leader_election.cc:290] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529)
I20260709 13:17:22.990931  2020 raft_consensus.cc:515] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.993338  1992 raft_consensus.cc:359] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.993956  1991 tablet_bootstrap.cc:492] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:22.994270  1992 raft_consensus.cc:385] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.994520  1991 ts_tablet_manager.cc:1403] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.009s
I20260709 13:17:22.994896  1992 raft_consensus.cc:740] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.994431  2020 leader_election.cc:290] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:22.996892  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:22.997926  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:22.998796  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c65f83f85d44d1b7525a643c8fe3e8" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:22.997958  1992 consensus_queue.cc:260] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.002681  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1c65f83f85d44d1b7525a643c8fe3e8" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:23.003541  1802 raft_consensus.cc:2393] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9e8deca048d4039851f55fe03299952 in current term 1: Already voted for candidate e91ba25a6a344f70aef0384659b3fd73 in this term.
I20260709 13:17:23.004876  1992 ts_tablet_manager.cc:1434] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:17:23.005172  2020 raft_consensus.cc:493] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.006119  1992 tablet_bootstrap.cc:492] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:23.005774  2020 raft_consensus.cc:515] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.009950  1999 raft_consensus.cc:493] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.010699  1999 raft_consensus.cc:515] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:22.996929  2028 raft_consensus.cc:2804] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.012483  2028 raft_consensus.cc:493] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.013072  2028 raft_consensus.cc:3060] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.012799  1915 leader_election.cc:304] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:23.013554  1992 tablet_bootstrap.cc:654] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.002005  1799 raft_consensus.cc:2393] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9e8deca048d4039851f55fe03299952 in current term 1: Already voted for candidate e91ba25a6a344f70aef0384659b3fd73 in this term.
I20260709 13:17:23.021453  1991 raft_consensus.cc:359] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.022567  1991 raft_consensus.cc:385] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.022827  1991 raft_consensus.cc:740] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.021726  1700 catalog_manager.cc:5697] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 0 to 1, leader changed from <none> to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.024310  1991 consensus_queue.cc:260] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.025204  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:23.025981  1799 raft_consensus.cc:2468] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.025543  1700 catalog_manager.cc:5697] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 0 to 1, leader changed from <none> to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.026656  1991 ts_tablet_manager.cc:1434] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:17:23.027730  1991 tablet_bootstrap.cc:492] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:23.033108  2028 raft_consensus.cc:515] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.033919  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:23.040261  2028 leader_election.cc:290] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.037885  2039 raft_consensus.cc:3060] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.041594  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.037962  2020 leader_election.cc:290] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.042950  1995 tablet_bootstrap.cc:654] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:23.043390  1766 leader_election.cc:343] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.038039  1991 tablet_bootstrap.cc:654] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.053835  1999 leader_election.cc:290] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.056775  1915 leader_election.cc:304] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:23.065613  1995 tablet_bootstrap.cc:492] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:23.062570  2039 raft_consensus.cc:2749] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.066241  1995 ts_tablet_manager.cc:1403] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.080s	user 0.004s	sys 0.016s
I20260709 13:17:23.068740  1999 raft_consensus.cc:3060] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.072943  1995 raft_consensus.cc:359] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.073699  1995 raft_consensus.cc:385] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.074057  1995 raft_consensus.cc:740] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.075307  1995 consensus_queue.cc:260] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.078636  1915 leader_election.cc:304] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.082280  1995 ts_tablet_manager.cc:1434] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:17:23.083315  2020 raft_consensus.cc:2804] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.083539  1995 tablet_bootstrap.cc:492] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:23.083998  2020 raft_consensus.cc:493] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.085227  2020 raft_consensus.cc:3060] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.092329  1995 tablet_bootstrap.cc:654] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.094496  1992 tablet_bootstrap.cc:492] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:23.099574  1995 tablet_bootstrap.cc:492] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
W20260709 13:17:23.084563  1915 leader_election.cc:343] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.100672  1995 ts_tablet_manager.cc:1403] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.005s
I20260709 13:17:23.101215  1999 raft_consensus.cc:2749] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.102582  1992 ts_tablet_manager.cc:1403] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.097s	user 0.028s	sys 0.011s
I20260709 13:17:23.102666  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:23.103609  1876 raft_consensus.cc:2468] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.104043  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:23.105051  1877 raft_consensus.cc:2468] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.105644  1995 raft_consensus.cc:359] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.106446  1995 raft_consensus.cc:385] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.106328  1992 raft_consensus.cc:359] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.106786  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.106956  1995 raft_consensus.cc:740] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.107574  1950 raft_consensus.cc:3060] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.107349  1915 leader_election.cc:304] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:23.108484  2020 raft_consensus.cc:515] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.109685  1999 raft_consensus.cc:2804] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.110085  1999 raft_consensus.cc:493] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.110437  1999 raft_consensus.cc:3060] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.108307  1995 consensus_queue.cc:260] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.107239  1992 raft_consensus.cc:385] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.111931  1992 raft_consensus.cc:740] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.112601  2020 leader_election.cc:290] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.113016  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:23.113835  1799 raft_consensus.cc:3060] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.113994  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.114864  1877 raft_consensus.cc:3060] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.112691  1992 consensus_queue.cc:260] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.122635  1877 raft_consensus.cc:2468] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.122926  1799 raft_consensus.cc:2468] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.124565  1915 leader_election.cc:304] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.125162  1992 ts_tablet_manager.cc:1434] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 13:17:23.125762  2020 raft_consensus.cc:2804] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.126313  2020 raft_consensus.cc:697] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.126561  1995 ts_tablet_manager.cc:1434] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:17:23.127444  1991 tablet_bootstrap.cc:492] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:23.127226  2020 consensus_queue.cc:237] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.127970  1991 ts_tablet_manager.cc:1403] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.100s	user 0.031s	sys 0.022s
I20260709 13:17:23.130260  2028 raft_consensus.cc:493] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.130966  2028 raft_consensus.cc:515] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.131669  1991 raft_consensus.cc:359] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.132504  1991 raft_consensus.cc:385] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.132804  1991 raft_consensus.cc:740] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.133062  2028 leader_election.cc:290] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.135316  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:23.135578  1950 raft_consensus.cc:2468] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:23.135969  1876 raft_consensus.cc:2468] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:23.136718  1991 consensus_queue.cc:260] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.137465  1766 leader_election.cc:304] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:23.139418  1991 ts_tablet_manager.cc:1434] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:23.140215  2028 raft_consensus.cc:2804] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.140708  2028 raft_consensus.cc:493] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.141031  2028 raft_consensus.cc:3060] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.141333  1999 raft_consensus.cc:515] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.140241  1699 catalog_manager.cc:5697] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:23.143396  1999 leader_election.cc:290] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.145041  1766 leader_election.cc:304] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:23.146133  2001 raft_consensus.cc:2804] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.146440  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.147226  1876 raft_consensus.cc:3060] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.147323  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:23.148645  1950 raft_consensus.cc:2393] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d2de778a80423c8ea7230963568a1c in current term 1: Already voted for candidate f9e8deca048d4039851f55fe03299952 in this term.
I20260709 13:17:23.150413  1994 raft_consensus.cc:493] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.151016  1994 raft_consensus.cc:515] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.153296  1994 leader_election.cc:290] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.154790  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:23.156653  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:23.177742  1876 raft_consensus.cc:2468] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.178589  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7563b7d4737943d6987c8a3451d85a75" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:23.179471  1950 raft_consensus.cc:2393] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e91ba25a6a344f70aef0384659b3fd73 in current term 1: Already voted for candidate d4d2de778a80423c8ea7230963568a1c in this term.
I20260709 13:17:23.180557  2028 raft_consensus.cc:515] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.181640  1915 leader_election.cc:304] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.182044  1799 raft_consensus.cc:2393] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f9e8deca048d4039851f55fe03299952 in current term 1: Already voted for candidate d4d2de778a80423c8ea7230963568a1c in this term.
I20260709 13:17:23.182536  1999 raft_consensus.cc:2804] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.182976  2028 leader_election.cc:290] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.183174  1999 raft_consensus.cc:697] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.184515  2001 raft_consensus.cc:697] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:23.184176  1999 consensus_queue.cc:237] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.185788  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.184258  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.188318  1876 raft_consensus.cc:2393] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d2de778a80423c8ea7230963568a1c in current term 1: Already voted for candidate f9e8deca048d4039851f55fe03299952 in this term.
I20260709 13:17:23.189659  1766 leader_election.cc:304] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c; no voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952
I20260709 13:17:23.191608  2045 raft_consensus.cc:2749] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.192868  2001 consensus_queue.cc:237] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.196365  1700 catalog_manager.cc:5697] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:23.200866  1841 leader_election.cc:304] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:23.202332  1994 raft_consensus.cc:3060] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.205120  1699 catalog_manager.cc:5697] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 1 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.210919  1994 raft_consensus.cc:2749] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.306932  1999 raft_consensus.cc:493] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.307590  1999 raft_consensus.cc:515] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.309473  1999 leader_election.cc:290] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.310880  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011ac8a199d24d52a117549237f0d442" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:23.311023  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011ac8a199d24d52a117549237f0d442" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:23.311895  1802 raft_consensus.cc:2468] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.312071  1876 raft_consensus.cc:2468] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.313562  1915 leader_election.cc:304] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.314450  1999 raft_consensus.cc:2804] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.314888  1999 raft_consensus.cc:493] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.315212  1999 raft_consensus.cc:3060] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.322664  1999 raft_consensus.cc:515] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.324684  1999 leader_election.cc:290] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.324707  1994 consensus_queue.cc:1048] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.325738  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011ac8a199d24d52a117549237f0d442" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:23.325987  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "011ac8a199d24d52a117549237f0d442" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.326380  1802 raft_consensus.cc:3060] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.326653  1876 raft_consensus.cc:3060] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.335018  1876 raft_consensus.cc:2468] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.336102  1802 raft_consensus.cc:2468] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.337615  1915 leader_election.cc:304] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.339128  1999 raft_consensus.cc:2804] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.339648  1999 raft_consensus.cc:697] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.340618  1999 consensus_queue.cc:237] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.342514  2039 raft_consensus.cc:493] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.342694  1994 consensus_queue.cc:1048] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.343201  2039 raft_consensus.cc:515] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.345480  2039 leader_election.cc:290] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.348367  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:23.349227  1799 raft_consensus.cc:2468] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.349942  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:23.350678  1876 raft_consensus.cc:2468] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:23.350592  1915 leader_election.cc:304] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.352360  1999 raft_consensus.cc:2804] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.352864  1999 raft_consensus.cc:493] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.353329  1999 raft_consensus.cc:3060] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.358604  1700 catalog_manager.cc:5697] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:23.362614  1999 raft_consensus.cc:515] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.363711  1994 consensus_queue.cc:1048] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.364951  1999 leader_election.cc:290] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:23.366240  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.366446  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:23.366940  1876 raft_consensus.cc:3060] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.366976  1799 raft_consensus.cc:3060] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.375706  1876 raft_consensus.cc:2468] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.376240  1799 raft_consensus.cc:2468] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:23.377182  1915 leader_election.cc:304] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:23.378216  1999 raft_consensus.cc:2804] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.379040  1999 raft_consensus.cc:697] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.380002  1999 consensus_queue.cc:237] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.387745  1994 consensus_queue.cc:1048] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.394043  1698 catalog_manager.cc:5697] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:23.414821  1999 consensus_queue.cc:1048] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.433992  2039 consensus_queue.cc:1048] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.454933  1876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
mode: GRACEFUL
new_leader_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59108
I20260709 13:17:23.455556  1876 raft_consensus.cc:606] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Received request to transfer leadership to TS d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:23.465665  1876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "484797364d6049a68a06966fb4660242"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
mode: GRACEFUL
new_leader_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59108
I20260709 13:17:23.466276  1876 raft_consensus.cc:606] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Received request to transfer leadership to TS d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:23.482148  1994 consensus_queue.cc:1048] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.483516  1951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:23.484323  1951 raft_consensus.cc:606] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:23.490360  2017 raft_consensus.cc:993] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: : Instructing follower d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:23.490916  2017 raft_consensus.cc:1081] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Signalling peer d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:23.491159  1951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:23.492038  1951 raft_consensus.cc:606] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:23.494701  1799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "484797364d6049a68a06966fb4660242"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59654
I20260709 13:17:23.496616  1799 raft_consensus.cc:493] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.497093  1799 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.503288  2017 consensus_queue.cc:1048] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.505914  1799 raft_consensus.cc:515] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.509261  1950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:23.510286  1950 raft_consensus.cc:606] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:23.511236  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.511883  1876 raft_consensus.cc:3055] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:23.512210  1876 raft_consensus.cc:740] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:23.513069  1876 consensus_queue.cc:260] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.514111  1876 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.519841  2039 consensus_queue.cc:1048] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.519943  1950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:23.520972  1950 raft_consensus.cc:606] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:23.522753  1876 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:23.524061  1766 leader_election.cc:304] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:23.524938  2001 raft_consensus.cc:2804] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:23.525601  2001 raft_consensus.cc:697] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:23.526508  2001 consensus_queue.cc:237] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.531344  2039 raft_consensus.cc:993] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:23.531903  2039 raft_consensus.cc:1081] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:23.532415  1799 leader_election.cc:290] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.535792  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.536180  1875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59076
I20260709 13:17:23.536465  1951 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.536895  1875 raft_consensus.cc:493] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.537288  1875 raft_consensus.cc:3060] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.536618  1699 catalog_manager.cc:5697] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 1 to 2, leader changed from e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2) to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.540834  2039 consensus_queue.cc:1048] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.545010  1951 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:23.550232  1875 raft_consensus.cc:515] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.551295  1876 raft_consensus.cc:1240] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Rejecting Update request from peer f9e8deca048d4039851f55fe03299952 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:23.552646  1875 leader_election.cc:290] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.554559  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.555016  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fbb6f2fcba04b6b9f78d2db58f523a8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:23.555207  1951 raft_consensus.cc:3055] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:23.555600  1951 raft_consensus.cc:740] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.555682  1799 raft_consensus.cc:3060] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.555815  2039 consensus_queue.cc:1059] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.559466  1951 consensus_queue.cc:260] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.562258  1951 raft_consensus.cc:3060] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.563743  1799 raft_consensus.cc:2468] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:23.566479  1841 leader_election.cc:304] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:23.567479  2017 raft_consensus.cc:2804] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:23.567991  2017 raft_consensus.cc:697] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:23.568923  2017 consensus_queue.cc:237] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.574362  1951 raft_consensus.cc:2468] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
W20260709 13:17:23.575176  1999 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:17:23.578876  1700 catalog_manager.cc:5697] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.655268  2001 consensus_queue.cc:1048] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.669646  2045 consensus_queue.cc:1048] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.775264  1999 consensus_queue.cc:1048] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.798497  2020 consensus_queue.cc:1048] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.813107  1999 raft_consensus.cc:993] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: : Instructing follower d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:23.813273  2039 consensus_queue.cc:1048] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.813805  1999 raft_consensus.cc:1081] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:23.816211  1802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59644
I20260709 13:17:23.816987  1802 raft_consensus.cc:493] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.817400  1802 raft_consensus.cc:3060] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.836268  1802 raft_consensus.cc:515] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.838963  1999 consensus_queue.cc:1048] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.843927  1802 leader_election.cc:290] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.846658  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.847719  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025ec36b5ae54e11a43c4f78d870c2fc" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.848240  1875 raft_consensus.cc:3060] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.848589  1951 raft_consensus.cc:3055] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:23.849018  1951 raft_consensus.cc:740] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.850441  1951 consensus_queue.cc:260] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.852170  1951 raft_consensus.cc:3060] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.861811  1875 raft_consensus.cc:2468] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:23.864090  1766 leader_election.cc:304] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:23.865243  2045 raft_consensus.cc:2804] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:23.865783  2045 raft_consensus.cc:697] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:23.866989  2045 consensus_queue.cc:237] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.868553  1951 raft_consensus.cc:2468] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:23.876791  1700 catalog_manager.cc:5697] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.884987  2039 raft_consensus.cc:993] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:23.885874  1999 raft_consensus.cc:1081] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:23.888168  1875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59076
I20260709 13:17:23.888892  1875 raft_consensus.cc:493] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.889438  1875 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.898672  1875 raft_consensus.cc:515] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.901433  1875 leader_election.cc:290] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.903189  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.903090  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:23.904256  1799 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.905705  1951 raft_consensus.cc:3055] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:23.906126  1951 raft_consensus.cc:740] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:23.907390  1951 consensus_queue.cc:260] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.909667  1951 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.916076  1799 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:23.918897  1841 leader_election.cc:304] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:23.921716  1951 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:23.922510  2017 raft_consensus.cc:2804] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:23.923089  2017 raft_consensus.cc:697] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:23.924825  2017 consensus_queue.cc:237] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:23.931937  1875 raft_consensus.cc:1275] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:23.932852  1994 raft_consensus.cc:993] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: : Instructing follower d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:23.935596  1999 consensus_queue.cc:1048] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.935878  2045 consensus_queue.cc:1048] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:23.943907  2017 raft_consensus.cc:1081] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Signalling peer d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:23.948026  1799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59654
I20260709 13:17:23.947021  1700 catalog_manager.cc:5697] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:23.948761  1799 raft_consensus.cc:493] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:23.949246  1799 raft_consensus.cc:3060] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.969693  1999 consensus_queue.cc:1048] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.971267  1799 raft_consensus.cc:515] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:23.976696  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:23.977593  1876 raft_consensus.cc:3055] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:23.978010  1876 raft_consensus.cc:740] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:23.978801  1951 raft_consensus.cc:1275] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:23.979336  1876 consensus_queue.cc:260] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:23.981649  1876 raft_consensus.cc:3060] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:23.981488  2001 consensus_queue.cc:1048] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:23.990315  1799 leader_election.cc:290] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:23.997131  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d190db5d02f418d80ee52a83ea60119" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:23.997982  1876 raft_consensus.cc:2468] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:23.998266  1950 raft_consensus.cc:3060] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.001742  2039 raft_consensus.cc:993] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:24.002471  2039 raft_consensus.cc:1081] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:24.008167  1766 leader_election.cc:304] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:24.010783  1876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59076
I20260709 13:17:24.011729  1876 raft_consensus.cc:493] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:24.012234  1876 raft_consensus.cc:3060] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.012465  2001 raft_consensus.cc:2804] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:24.013652  2001 raft_consensus.cc:697] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:24.014981  2001 consensus_queue.cc:237] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } }
I20260709 13:17:24.022766  1876 raft_consensus.cc:515] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:24.025594  1876 leader_election.cc:290] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:24.026366  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:24.027412  1802 raft_consensus.cc:3060] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.034428  1700 catalog_manager.cc:5697] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 1 to 2, leader changed from e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2) to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:24.037755  1802 raft_consensus.cc:2468] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:24.038651  1799 raft_consensus.cc:1240] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Rejecting Update request from peer f9e8deca048d4039851f55fe03299952 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:24.039529  1841 leader_election.cc:304] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:24.040597  1994 raft_consensus.cc:2804] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:24.041095  1994 raft_consensus.cc:697] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:24.041680  2039 consensus_queue.cc:1059] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.042223  1994 consensus_queue.cc:237] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:24.042979  2039 raft_consensus.cc:3055] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:24.043380  2039 raft_consensus.cc:740] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:24.044780  1950 raft_consensus.cc:2468] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:24.044682  2039 consensus_queue.cc:260] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:24.045137  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff66ac876d0546a19d3f5002d1f77848" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:24.046273  2039 raft_consensus.cc:3060] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.055475  1699 catalog_manager.cc:5697] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.065685  1949 raft_consensus.cc:2468] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:24.089562  1949 raft_consensus.cc:1275] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.091032  1994 consensus_queue.cc:1048] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:24.103119  1799 raft_consensus.cc:1275] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.104842  1994 consensus_queue.cc:1048] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:24.284101  1949 raft_consensus.cc:1275] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.285822  2001 consensus_queue.cc:1048] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:24.297326  1876 raft_consensus.cc:1275] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.299335  2028 consensus_queue.cc:1048] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:24.379220  1949 raft_consensus.cc:1275] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.381755  1994 consensus_queue.cc:1048] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:24.393795  1799 raft_consensus.cc:1275] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.396121  2073 consensus_queue.cc:1048] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:24.426157  1876 raft_consensus.cc:1275] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.428179  2045 consensus_queue.cc:1048] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:24.440197  1949 raft_consensus.cc:1275] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.442261  2045 consensus_queue.cc:1048] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:24.529254  1799 raft_consensus.cc:1275] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.531045  2010 consensus_queue.cc:1048] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.542960  1949 raft_consensus.cc:1275] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:24.544642  2073 consensus_queue.cc:1048] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:26.581274  1699 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41842:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:26.584079  1699 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:17:26.647195  1748 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:26.652218  1789 tablet_service.cc:1511] Processing CreateTablet for tablet ccd0925475c14c1abdcc732703f9fbd6 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:26.653491  1866 tablet_service.cc:1511] Processing CreateTablet for tablet ccd0925475c14c1abdcc732703f9fbd6 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:26.654387  1789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd0925475c14c1abdcc732703f9fbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.654783  1866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd0925475c14c1abdcc732703f9fbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.664609  1940 tablet_service.cc:1511] Processing CreateTablet for tablet ccd0925475c14c1abdcc732703f9fbd6 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:26.665402  1784 tablet_service.cc:1511] Processing CreateTablet for tablet 372e2af2548a46d39bb7577e4d7e56ff (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:26.666651  1940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd0925475c14c1abdcc732703f9fbd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.666985  1784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372e2af2548a46d39bb7577e4d7e56ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.687916  1860 tablet_service.cc:1511] Processing CreateTablet for tablet 372e2af2548a46d39bb7577e4d7e56ff (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:26.689941  1860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372e2af2548a46d39bb7577e4d7e56ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.692104  1935 tablet_service.cc:1511] Processing CreateTablet for tablet 372e2af2548a46d39bb7577e4d7e56ff (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:26.694020  1935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372e2af2548a46d39bb7577e4d7e56ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.693441  1788 tablet_service.cc:1511] Processing CreateTablet for tablet 361936d22bf24421b2af4d6c0510298f (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:26.695485  1788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361936d22bf24421b2af4d6c0510298f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.713122  1864 tablet_service.cc:1511] Processing CreateTablet for tablet 361936d22bf24421b2af4d6c0510298f (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:26.715961  1864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361936d22bf24421b2af4d6c0510298f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.731331  1937 tablet_service.cc:1511] Processing CreateTablet for tablet 361936d22bf24421b2af4d6c0510298f (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:26.733791  1937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361936d22bf24421b2af4d6c0510298f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.735364  2090 tablet_bootstrap.cc:492] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:26.744359  1866 tablet_service.cc:1511] Processing CreateTablet for tablet 3908698cf2654827abcfe6ae5933096c (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:26.744473  1791 tablet_service.cc:1511] Processing CreateTablet for tablet 3908698cf2654827abcfe6ae5933096c (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:26.747979  2092 tablet_bootstrap.cc:492] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:26.748571  1791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3908698cf2654827abcfe6ae5933096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.751534  1941 tablet_service.cc:1511] Processing CreateTablet for tablet 3908698cf2654827abcfe6ae5933096c (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:26.752959  1866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3908698cf2654827abcfe6ae5933096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.753075  1941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3908698cf2654827abcfe6ae5933096c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.752620  1861 tablet_service.cc:1511] Processing CreateTablet for tablet dbce222191f9408397faebf06a41ba57 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:26.753753  1865 tablet_service.cc:1511] Processing CreateTablet for tablet b4bb4416aa6442e09813b3cf9e8237f8 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:26.754413  1861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbce222191f9408397faebf06a41ba57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.755981  1786 tablet_service.cc:1511] Processing CreateTablet for tablet dbce222191f9408397faebf06a41ba57 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:26.756139  1787 tablet_service.cc:1511] Processing CreateTablet for tablet b4bb4416aa6442e09813b3cf9e8237f8 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:26.758087  1865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4bb4416aa6442e09813b3cf9e8237f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.759688  1936 tablet_service.cc:1511] Processing CreateTablet for tablet dbce222191f9408397faebf06a41ba57 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:26.767448  1790 tablet_service.cc:1511] Processing CreateTablet for tablet f63162ba797b434ca275ef7b6299520f (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:26.770525  1862 tablet_service.cc:1511] Processing CreateTablet for tablet 1888a2cd9a9245ef8ae30cebf3fc1401 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:26.772192  1862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1888a2cd9a9245ef8ae30cebf3fc1401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.775013  1867 tablet_service.cc:1511] Processing CreateTablet for tablet c4123a7d6ed643d2a7213ca3f34e7b1d (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:26.776734  1867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4123a7d6ed643d2a7213ca3f34e7b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.781211  1938 tablet_service.cc:1511] Processing CreateTablet for tablet b4bb4416aa6442e09813b3cf9e8237f8 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:26.784849  1939 tablet_service.cc:1511] Processing CreateTablet for tablet f63162ba797b434ca275ef7b6299520f (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:26.787588  1934 tablet_service.cc:1511] Processing CreateTablet for tablet 1888a2cd9a9245ef8ae30cebf3fc1401 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:26.787880  1933 tablet_service.cc:1511] Processing CreateTablet for tablet c4123a7d6ed643d2a7213ca3f34e7b1d (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:26.790828  1785 tablet_service.cc:1511] Processing CreateTablet for tablet c4123a7d6ed643d2a7213ca3f34e7b1d (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:26.792424  1786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbce222191f9408397faebf06a41ba57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.790920  1789 tablet_service.cc:1511] Processing CreateTablet for tablet 1888a2cd9a9245ef8ae30cebf3fc1401 (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:26.798292  1939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63162ba797b434ca275ef7b6299520f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.765213  1863 tablet_service.cc:1511] Processing CreateTablet for tablet f63162ba797b434ca275ef7b6299520f (DEFAULT_TABLE table=soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:26.801920  1936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbce222191f9408397faebf06a41ba57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.810359  1863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63162ba797b434ca275ef7b6299520f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.811573  1938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4bb4416aa6442e09813b3cf9e8237f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.832389  1933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4123a7d6ed643d2a7213ca3f34e7b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.834585  1934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1888a2cd9a9245ef8ae30cebf3fc1401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:26.842123  2090 tablet_bootstrap.cc:654] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.862581  2091 tablet_bootstrap.cc:492] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:26.894418  2092 tablet_bootstrap.cc:654] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.933248  2091 tablet_bootstrap.cc:654] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.935175  2090 tablet_bootstrap.cc:492] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:26.935827  2090 ts_tablet_manager.cc:1403] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.201s	user 0.019s	sys 0.013s
I20260709 13:17:26.943791  2090 raft_consensus.cc:359] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:26.944654  2090 raft_consensus.cc:385] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.944993  2090 raft_consensus.cc:740] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.946008  2090 consensus_queue.cc:260] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:26.949286  2090 ts_tablet_manager.cc:1434] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.006s
I20260709 13:17:26.949573  2091 tablet_bootstrap.cc:492] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:26.950359  2091 ts_tablet_manager.cc:1403] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.088s	user 0.010s	sys 0.009s
I20260709 13:17:26.950569  2090 tablet_bootstrap.cc:492] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:26.949488  2092 tablet_bootstrap.cc:492] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:26.954200  2092 ts_tablet_manager.cc:1403] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.207s	user 0.013s	sys 0.006s
I20260709 13:17:26.958971  2091 raft_consensus.cc:359] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:26.960014  2091 raft_consensus.cc:385] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.960124  2090 tablet_bootstrap.cc:654] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.960372  2091 raft_consensus.cc:740] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.961251  2091 consensus_queue.cc:260] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:26.968513  2091 ts_tablet_manager.cc:1434] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:17:26.969928  2091 tablet_bootstrap.cc:492] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:26.971263  2092 raft_consensus.cc:359] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:26.972623  2092 raft_consensus.cc:385] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.972941  2092 raft_consensus.cc:740] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.975543  2092 consensus_queue.cc:260] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:26.988237  2090 tablet_bootstrap.cc:492] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:26.988982  2090 ts_tablet_manager.cc:1403] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.006s
I20260709 13:17:26.989900  2091 tablet_bootstrap.cc:654] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.000196  2092 ts_tablet_manager.cc:1434] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.045s	user 0.014s	sys 0.000s
I20260709 13:17:27.014334  2090 raft_consensus.cc:359] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.017630  2090 raft_consensus.cc:385] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.018548  2090 raft_consensus.cc:740] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.021238  2090 consensus_queue.cc:260] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.029039  2090 ts_tablet_manager.cc:1434] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.040s	user 0.017s	sys 0.002s
I20260709 13:17:27.031989  2090 tablet_bootstrap.cc:492] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:27.052726  2090 tablet_bootstrap.cc:654] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.079542  2090 tablet_bootstrap.cc:492] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:27.081327  2090 ts_tablet_manager.cc:1403] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.019s
I20260709 13:17:27.091626  2090 raft_consensus.cc:359] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.093910  2090 raft_consensus.cc:385] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.094802  2090 raft_consensus.cc:740] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.098330  2090 consensus_queue.cc:260] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.106616  2090 ts_tablet_manager.cc:1434] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.010s
I20260709 13:17:27.109701  2090 tablet_bootstrap.cc:492] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:27.122370  2039 raft_consensus.cc:493] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.123431  2039 raft_consensus.cc:515] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.128597  2090 tablet_bootstrap.cc:654] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.129642  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:27.130749  1876 raft_consensus.cc:2468] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:27.133185  1915 leader_election.cc:304] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:27.154505  2090 tablet_bootstrap.cc:492] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:27.156708  2090 ts_tablet_manager.cc:1403] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.048s	user 0.022s	sys 0.008s
I20260709 13:17:27.159572  2039 leader_election.cc:290] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:27.160761  2039 raft_consensus.cc:2804] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.161422  2039 raft_consensus.cc:493] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.162019  2039 raft_consensus.cc:3060] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.166349  2090 raft_consensus.cc:359] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.168954  2090 raft_consensus.cc:385] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.171129  2090 raft_consensus.cc:740] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.176914  2090 consensus_queue.cc:260] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.183549  2090 ts_tablet_manager.cc:1434] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.026s	user 0.013s	sys 0.001s
I20260709 13:17:27.185099  2090 tablet_bootstrap.cc:492] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:27.193562  2090 tablet_bootstrap.cc:654] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.207568  2017 raft_consensus.cc:493] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.208293  2017 raft_consensus.cc:515] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.212477  2090 tablet_bootstrap.cc:492] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:27.213307  2090 ts_tablet_manager.cc:1403] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.004s
I20260709 13:17:27.222010  2090 raft_consensus.cc:359] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.223198  2090 raft_consensus.cc:385] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.223595  2090 raft_consensus.cc:740] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.224643  2090 consensus_queue.cc:260] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.226295  2093 raft_consensus.cc:493] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.226835  2093 raft_consensus.cc:515] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.229455  2093 leader_election.cc:290] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:27.265527  2017 leader_election.cc:290] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
W20260709 13:17:27.272032  1915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:40885, user_credentials={real_user=slave}} blocked reactor thread for 139162us
I20260709 13:17:27.476977  2091 tablet_bootstrap.cc:492] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:27.477834  2091 ts_tablet_manager.cc:1403] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.508s	user 0.053s	sys 0.086s
I20260709 13:17:27.479385  2106 raft_consensus.cc:493] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.480366  2106 raft_consensus.cc:515] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.481026  2091 raft_consensus.cc:359] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.481871  2091 raft_consensus.cc:385] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.482194  2091 raft_consensus.cc:740] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.482990  2091 consensus_queue.cc:260] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.485177  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:27.484984  2106 leader_election.cc:290] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:27.486019  1875 raft_consensus.cc:2468] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:27.487829  1766 leader_election.cc:304] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:27.490325  2106 raft_consensus.cc:2804] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.491016  2106 raft_consensus.cc:493] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.491588  2106 raft_consensus.cc:3060] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.488773  2091 ts_tablet_manager.cc:1434] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:17:27.507540  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:27.509188  1789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1888a2cd9a9245ef8ae30cebf3fc1401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:27.884361  2091 tablet_bootstrap.cc:492] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:27.509987  2106 raft_consensus.cc:515] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.885051  1785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4123a7d6ed643d2a7213ca3f34e7b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:27.886399  2106 leader_election.cc:290] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:27.804553  2103 raft_consensus.cc:493] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.887207  2103 raft_consensus.cc:515] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.887902  1787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4bb4416aa6442e09813b3cf9e8237f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:27.811257  2092 tablet_bootstrap.cc:492] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:27.824232  2090 ts_tablet_manager.cc:1434] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.611s	user 0.027s	sys 0.054s
W20260709 13:17:27.889137  2090 ts_tablet_manager.cc:1467] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Tablet startup took 675ms
W20260709 13:17:27.889616  2090 ts_tablet_manager.cc:1469] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Trace:
0709 13:17:26.752050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.752404 (+   354us) service_pool.cc:224] Handling call
0709 13:17:26.754764 (+  2360us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.754906 (+   142us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:26.783123 (+ 28217us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:26.905894 (+122771us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:26.906436 (+   542us) inbound_call.cc:177] Queueing success response
0709 13:17:27.184662 (+278226us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:17:27.213691 (+ 29029us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:17:27.214466 (+   775us) tablet_replica.cc:261] Starting instrumentation
0709 13:17:27.215329 (+   863us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":40596,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"tablet-open.queue_time_us":278342,"thread_start_us":78271,"threads_started":1}
I20260709 13:17:27.892594  2090 tablet_bootstrap.cc:492] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
W20260709 13:17:27.832747  1841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.000s	sys 0.010s
W20260709 13:17:27.894254  1841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.001s	sys 0.011s
I20260709 13:17:27.894595  1799 raft_consensus.cc:2393] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9e8deca048d4039851f55fe03299952 in current term 1: Already voted for candidate d4d2de778a80423c8ea7230963568a1c in this term.
W20260709 13:17:27.894964  1748 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:27.894635  1841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.1:38529, user_credentials={real_user=slave}} blocked reactor thread for 675213us
I20260709 13:17:27.898540  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
W20260709 13:17:27.858240  2039 consensus_meta.cc:287] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: Time spent flushing consensus metadata: real 0.695s	user 0.022s	sys 0.010s
W20260709 13:17:27.861368  1843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.015s	sys 0.015s
W20260709 13:17:27.900722  1843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.015s	sys 0.016s
I20260709 13:17:27.901806  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:27.902621  1875 raft_consensus.cc:3060] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.902886  2039 raft_consensus.cc:515] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.906111  2039 leader_election.cc:290] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:27.907822  1950 raft_consensus.cc:2393] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d2de778a80423c8ea7230963568a1c in current term 1: Already voted for candidate f9e8deca048d4039851f55fe03299952 in this term.
W20260709 13:17:27.869568  1864 consensus_meta.cc:287] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: Time spent flushing consensus metadata: real 0.662s	user 0.020s	sys 0.013s
I20260709 13:17:27.915448  1864 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59072 (request call id 11) took 1204 ms. Trace:
W20260709 13:17:27.911286  1841 leader_election.cc:343] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Not found: Tablet not found: c4123a7d6ed643d2a7213ca3f34e7b1d
I20260709 13:17:27.916448  2091 tablet_bootstrap.cc:654] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.933427  1864 rpcz_store.cc:276] 0709 13:17:26.710761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.711179 (+   418us) service_pool.cc:224] Handling call
0709 13:17:26.715156 (+  3977us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.715302 (+   146us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:27.207178 (+491876us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:27.914835 (+707657us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:27.915366 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.938408  2103 leader_election.cc:290] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
W20260709 13:17:27.943212  1841 leader_election.cc:343] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Not found: Tablet not found: f63162ba797b434ca275ef7b6299520f
I20260709 13:17:27.945896  2091 tablet_bootstrap.cc:492] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:27.946652  2091 ts_tablet_manager.cc:1403] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.000s
I20260709 13:17:27.951753  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:27.953222  1950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:27.954758  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:27.955693  1950 raft_consensus.cc:2393] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d2de778a80423c8ea7230963568a1c in current term 1: Already voted for candidate f9e8deca048d4039851f55fe03299952 in this term.
I20260709 13:17:27.954888  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:27.953338  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccd0925475c14c1abdcc732703f9fbd6" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:27.957216  1802 raft_consensus.cc:2393] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f9e8deca048d4039851f55fe03299952 in current term 1: Already voted for candidate d4d2de778a80423c8ea7230963568a1c in this term.
I20260709 13:17:27.957556  1875 raft_consensus.cc:2468] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:27.958171  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:27.960568  1789 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59630 (request call id 16) took 1171 ms. Trace:
I20260709 13:17:27.961002  1789 rpcz_store.cc:276] 0709 13:17:26.788907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.789983 (+  1076us) service_pool.cc:224] Handling call
0709 13:17:26.794577 (+  4594us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.795645 (+  1068us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:27.932067 (+1136422us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:27.959601 (+ 27534us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:27.960481 (+   880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.960125  1790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63162ba797b434ca275ef7b6299520f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:27.968659  1766 leader_election.cc:304] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:27.969686  2092 tablet_bootstrap.cc:654] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.919384  2090 tablet_bootstrap.cc:654] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.962725  2091 raft_consensus.cc:359] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.972090  1787 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59630 (request call id 14) took 1217 ms. Trace:
W20260709 13:17:27.980293  1841 leader_election.cc:343] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.982527  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:27.985301  1948 raft_consensus.cc:2468] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 0.
W20260709 13:17:27.987396  1841 leader_election.cc:343] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.987944  1841 leader_election.cc:304] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:27.988623  1915 leader_election.cc:304] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:27.988916  2103 raft_consensus.cc:2749] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.983289  1785 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59630 (request call id 17) took 1194 ms. Trace:
I20260709 13:17:27.977999  2045 raft_consensus.cc:2804] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.990342  2045 raft_consensus.cc:697] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:27.990461  2039 raft_consensus.cc:2749] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.982000  1791 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59630 (request call id 12) took 1239 ms. Trace:
I20260709 13:17:27.991699  1791 rpcz_store.cc:276] 0709 13:17:26.742720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.743005 (+   285us) service_pool.cc:224] Handling call
0709 13:17:26.747642 (+  4637us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.747771 (+   129us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:27.941211 (+1193440us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:27.981298 (+ 40087us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:27.981923 (+   625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.983783  1787 rpcz_store.cc:276] 0709 13:17:26.754626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.754855 (+   229us) service_pool.cc:224] Handling call
0709 13:17:26.757169 (+  2314us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.757291 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:27.949411 (+1192120us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:27.971462 (+ 22051us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:27.972012 (+   550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.991400  2045 consensus_queue.cc:237] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:27.990108  1785 rpcz_store.cc:276] 0709 13:17:26.789313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.789637 (+   324us) service_pool.cc:224] Handling call
0709 13:17:26.791807 (+  2170us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.791916 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:27.932335 (+1140419us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:27.982679 (+ 50344us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:27.983218 (+   539us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:27.983196  2091 raft_consensus.cc:385] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.995697  2091 raft_consensus.cc:740] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.996675  2091 consensus_queue.cc:260] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.003860  2091 ts_tablet_manager.cc:1434] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:17:28.004436  2090 tablet_bootstrap.cc:492] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:28.005260  2091 tablet_bootstrap.cc:492] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:28.005287  2090 ts_tablet_manager.cc:1403] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.000s
I20260709 13:17:28.010226  2090 raft_consensus.cc:359] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.011232  2090 raft_consensus.cc:385] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.011546  2090 raft_consensus.cc:740] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.012584  2090 consensus_queue.cc:260] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.014824  2091 tablet_bootstrap.cc:654] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.021873  1790 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59630 (request call id 15) took 1258 ms. Trace:
I20260709 13:17:28.022989  1790 rpcz_store.cc:276] 0709 13:17:26.763178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.765707 (+  2529us) service_pool.cc:224] Handling call
0709 13:17:26.768580 (+  2873us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:26.768724 (+   144us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:27.996321 (+1227597us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:28.021223 (+ 24902us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:28.021802 (+   579us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:28.014914  1841 leader_election.cc:343] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.026274  1841 leader_election.cc:304] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:28.028541  2093 raft_consensus.cc:2749] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.032660  1841 leader_election.cc:304] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:28.033728  2093 raft_consensus.cc:2804] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.034492  2093 raft_consensus.cc:493] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.035043  2093 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.035204  2092 tablet_bootstrap.cc:492] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.036115  2092 ts_tablet_manager.cc:1403] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.225s	user 0.007s	sys 0.009s
I20260709 13:17:28.032694  2090 ts_tablet_manager.cc:1434] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:17:28.040025  2090 tablet_bootstrap.cc:492] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:28.050515  2091 tablet_bootstrap.cc:492] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:28.051260  2091 ts_tablet_manager.cc:1403] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.046s	user 0.031s	sys 0.009s
I20260709 13:17:28.055078  2091 raft_consensus.cc:359] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.056391  2091 raft_consensus.cc:385] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.056726  2091 raft_consensus.cc:740] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.057642  2091 consensus_queue.cc:260] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.063539  2091 ts_tablet_manager.cc:1434] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:17:28.064258  2093 raft_consensus.cc:515] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.060911  2092 raft_consensus.cc:359] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.066331  2092 raft_consensus.cc:385] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.066844  2092 raft_consensus.cc:740] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.067958  2093 leader_election.cc:290] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.067791  2092 consensus_queue.cc:260] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.070537  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:28.071308  1948 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.071810  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
W20260709 13:17:28.073830  1841 leader_election.cc:343] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.075032  1697 catalog_manager.cc:5697] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 1 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.064906  2091 tablet_bootstrap.cc:492] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:28.065450  2090 tablet_bootstrap.cc:654] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.082818  1948 raft_consensus.cc:2468] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 1.
I20260709 13:17:28.084499  1841 leader_election.cc:304] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:28.085711  2093 raft_consensus.cc:2804] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.086256  2093 raft_consensus.cc:697] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:28.087157  2093 consensus_queue.cc:237] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.092906  2092 ts_tablet_manager.cc:1434] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.003s
I20260709 13:17:28.102571  2092 tablet_bootstrap.cc:492] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:28.097684  2091 tablet_bootstrap.cc:654] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.116153  2092 tablet_bootstrap.cc:654] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.115154  1699 catalog_manager.cc:5697] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 0 to 1, leader changed from <none> to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 1 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.124742  2090 tablet_bootstrap.cc:492] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:28.126406  2090 ts_tablet_manager.cc:1403] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.010s
I20260709 13:17:28.125952  2093 raft_consensus.cc:493] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.127707  2093 raft_consensus.cc:515] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.130227  2093 leader_election.cc:290] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.132033  2090 raft_consensus.cc:359] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.134150  2090 raft_consensus.cc:385] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.134640  2090 raft_consensus.cc:740] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.135344  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:28.137210  2090 consensus_queue.cc:260] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.139372  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
W20260709 13:17:28.141592  1841 leader_election.cc:343] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:28.142802  1841 leader_election.cc:343] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.143469  1841 leader_election.cc:304] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:28.144634  2039 raft_consensus.cc:493] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.144874  2093 raft_consensus.cc:2749] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.145314  2039 raft_consensus.cc:515] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.143334  2090 ts_tablet_manager.cc:1434] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.001s
I20260709 13:17:28.147928  2039 leader_election.cc:290] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.148344  2090 tablet_bootstrap.cc:492] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:28.151219  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
W20260709 13:17:28.156172  1915 leader_election.cc:343] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.157570  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:28.158703  1802 raft_consensus.cc:2468] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.160598  1915 leader_election.cc:304] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:28.162364  2039 raft_consensus.cc:2804] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.162837  2039 raft_consensus.cc:493] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.163187  2039 raft_consensus.cc:3060] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.164520  2091 tablet_bootstrap.cc:492] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:28.165207  2091 ts_tablet_manager.cc:1403] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.101s	user 0.002s	sys 0.018s
I20260709 13:17:28.168944  2091 raft_consensus.cc:359] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.170077  2091 raft_consensus.cc:385] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.170459  2091 raft_consensus.cc:740] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.171734  2091 consensus_queue.cc:260] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.177947  2092 tablet_bootstrap.cc:492] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.178769  2092 ts_tablet_manager.cc:1403] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.077s	user 0.033s	sys 0.027s
I20260709 13:17:28.182740  2092 raft_consensus.cc:359] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.183828  2092 raft_consensus.cc:385] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.184232  2092 raft_consensus.cc:740] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.179368  2039 raft_consensus.cc:515] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.185268  2091 ts_tablet_manager.cc:1434] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:17:28.185302  2092 consensus_queue.cc:260] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.196297  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:28.197081  1802 raft_consensus.cc:3060] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.201046  2090 tablet_bootstrap.cc:654] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.196384  2092 ts_tablet_manager.cc:1434] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:17:28.206790  2092 tablet_bootstrap.cc:492] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:28.215106  2092 tablet_bootstrap.cc:654] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.197817  2091 tablet_bootstrap.cc:492] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:28.199743  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
W20260709 13:17:28.219888  1915 leader_election.cc:343] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.226464  2091 tablet_bootstrap.cc:654] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.199957  2039 leader_election.cc:290] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.232456  1802 raft_consensus.cc:2468] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.234812  1915 leader_election.cc:304] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:28.237392  2039 raft_consensus.cc:2804] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.238349  2039 raft_consensus.cc:697] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:28.239318  2039 consensus_queue.cc:237] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.240357  2090 tablet_bootstrap.cc:492] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:28.241273  2090 ts_tablet_manager.cc:1403] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.022s
I20260709 13:17:28.242391  2091 tablet_bootstrap.cc:492] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:28.243113  2091 ts_tablet_manager.cc:1403] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.008s
I20260709 13:17:28.245193  2090 raft_consensus.cc:359] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.246384  2090 raft_consensus.cc:385] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.246763  2090 raft_consensus.cc:740] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.247826  2090 consensus_queue.cc:260] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.254521  2091 raft_consensus.cc:359] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.255529  2091 raft_consensus.cc:385] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.255844  2091 raft_consensus.cc:740] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.260432  2092 tablet_bootstrap.cc:492] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.261164  2092 ts_tablet_manager.cc:1403] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.003s
I20260709 13:17:28.264971  2092 raft_consensus.cc:359] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.265925  2090 ts_tablet_manager.cc:1434] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 13:17:28.266386  2091 consensus_queue.cc:260] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.269598  2091 ts_tablet_manager.cc:1434] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.005s
I20260709 13:17:28.271237  2091 tablet_bootstrap.cc:492] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:28.265919  2092 raft_consensus.cc:385] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.273428  2092 raft_consensus.cc:740] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.274587  2092 consensus_queue.cc:260] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.275694  1699 catalog_manager.cc:5697] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.281301  2092 ts_tablet_manager.cc:1434] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:17:28.287669  2090 tablet_bootstrap.cc:492] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: Bootstrap starting.
I20260709 13:17:28.291185  2092 tablet_bootstrap.cc:492] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:28.299691  2090 tablet_bootstrap.cc:654] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.300935  2091 tablet_bootstrap.cc:654] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.301836  2093 raft_consensus.cc:493] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.302529  2093 raft_consensus.cc:515] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.305243  2093 leader_election.cc:290] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.306041  2092 tablet_bootstrap.cc:654] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.308349  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:28.310359  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
W20260709 13:17:28.310748  1841 leader_election.cc:343] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:28.312762  1841 leader_election.cc:343] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.313824  1841 leader_election.cc:304] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73; no voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952
I20260709 13:17:28.315194  2093 raft_consensus.cc:2749] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.320817  2090 tablet_bootstrap.cc:492] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: No bootstrap required, opened a new log
I20260709 13:17:28.321589  2090 ts_tablet_manager.cc:1403] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.005s
I20260709 13:17:28.322093  2092 tablet_bootstrap.cc:492] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.322754  2092 ts_tablet_manager.cc:1403] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.006s
I20260709 13:17:28.324157  2091 tablet_bootstrap.cc:492] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:28.324887  2091 ts_tablet_manager.cc:1403] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.014s
I20260709 13:17:28.325248  2090 raft_consensus.cc:359] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.326128  2090 raft_consensus.cc:385] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.326431  2090 raft_consensus.cc:740] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.326561  2092 raft_consensus.cc:359] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.327476  2092 raft_consensus.cc:385] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.328044  2092 raft_consensus.cc:740] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.327418  2090 consensus_queue.cc:260] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.328990  2092 consensus_queue.cc:260] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.329531  2091 raft_consensus.cc:359] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.330446  2091 raft_consensus.cc:385] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.330799  2091 raft_consensus.cc:740] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.332275  2092 ts_tablet_manager.cc:1434] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.002s
I20260709 13:17:28.332149  2106 raft_consensus.cc:493] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.332891  2106 raft_consensus.cc:515] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.333599  2092 tablet_bootstrap.cc:492] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:28.337460  2106 leader_election.cc:290] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.337664  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:28.338433  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.338495  1948 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:28.339191  1875 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:28.340222  1766 leader_election.cc:304] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.341570  2106 raft_consensus.cc:2804] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.343242  2092 tablet_bootstrap.cc:654] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.353087  2090 ts_tablet_manager.cc:1434] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:17:28.341969  2106 raft_consensus.cc:493] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.354578  2106 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.369545  2092 tablet_bootstrap.cc:492] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.370291  2092 ts_tablet_manager.cc:1403] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.037s	user 0.023s	sys 0.011s
I20260709 13:17:28.338441  2091 consensus_queue.cc:260] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.374085  2092 raft_consensus.cc:359] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.374878  2092 raft_consensus.cc:385] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.375174  2092 raft_consensus.cc:740] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.376382  2092 consensus_queue.cc:260] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.377687  2091 ts_tablet_manager.cc:1434] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.003s
I20260709 13:17:28.379510  2091 tablet_bootstrap.cc:492] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:28.383073  2092 ts_tablet_manager.cc:1434] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:17:28.384552  2092 tablet_bootstrap.cc:492] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:28.388664  2091 tablet_bootstrap.cc:654] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.396247  2045 consensus_queue.cc:1048] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.406188  2106 raft_consensus.cc:515] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.408563  2091 tablet_bootstrap.cc:492] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:28.409272  2091 ts_tablet_manager.cc:1403] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.013s
I20260709 13:17:28.410660  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:28.411474  1876 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.412693  2091 raft_consensus.cc:359] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.413350  2092 tablet_bootstrap.cc:654] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.413489  2091 raft_consensus.cc:385] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.413949  2091 raft_consensus.cc:740] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.414769  2091 consensus_queue.cc:260] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.417459  2091 ts_tablet_manager.cc:1434] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:28.418929  2091 tablet_bootstrap.cc:492] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: Bootstrap starting.
I20260709 13:17:28.427279  2106 leader_election.cc:290] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.430917  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:28.431324  1876 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:28.431792  1948 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.433053  1766 leader_election.cc:304] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:28.435256  2091 tablet_bootstrap.cc:654] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.437927  2039 raft_consensus.cc:493] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.438649  2039 raft_consensus.cc:515] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.440477  2045 raft_consensus.cc:493] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.441306  2045 raft_consensus.cc:515] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.442986  2039 leader_election.cc:290] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.447572  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:28.448470  1802 raft_consensus.cc:2468] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.450474  1915 leader_election.cc:304] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.452011  2039 raft_consensus.cc:2804] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.452580  2039 raft_consensus.cc:493] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.453145  2039 raft_consensus.cc:3060] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.456066  2106 raft_consensus.cc:2804] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.461364  2091 tablet_bootstrap.cc:492] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: No bootstrap required, opened a new log
I20260709 13:17:28.462116  2091 ts_tablet_manager.cc:1403] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.010s
I20260709 13:17:28.461527  1877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.443858  2045 leader_election.cc:290] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.467674  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.467871  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:28.469738  1949 raft_consensus.cc:2393] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d2de778a80423c8ea7230963568a1c in current term 1: Already voted for candidate e91ba25a6a344f70aef0384659b3fd73 in this term.
I20260709 13:17:28.468709  1876 raft_consensus.cc:2468] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.481161  2091 raft_consensus.cc:359] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.482991  2091 raft_consensus.cc:385] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.483402  2091 raft_consensus.cc:740] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.484113  2092 tablet_bootstrap.cc:492] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.484812  2092 ts_tablet_manager.cc:1403] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.101s	user 0.010s	sys 0.006s
I20260709 13:17:28.484405  2091 consensus_queue.cc:260] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.487582  2039 raft_consensus.cc:515] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.491925  2091 ts_tablet_manager.cc:1434] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:17:28.479019  1948 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:28.488241  2092 raft_consensus.cc:359] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.490252  2106 raft_consensus.cc:697] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:28.494701  2092 raft_consensus.cc:385] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.495357  2092 raft_consensus.cc:740] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.496361  2092 consensus_queue.cc:260] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.497613  1766 leader_election.cc:304] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c; no voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952
W20260709 13:17:28.498332  1823 debug-util.cc:398] Leaking SignalData structure 0x7b080010eda0 after lost signal to thread 21639
I20260709 13:17:28.499269  2092 ts_tablet_manager.cc:1434] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
W20260709 13:17:28.499889  1823 debug-util.cc:398] Leaking SignalData structure 0x7b0800393de0 after lost signal to thread 1734
I20260709 13:17:28.500573  2092 tablet_bootstrap.cc:492] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
W20260709 13:17:28.503997  1823 debug-util.cc:398] Leaking SignalData structure 0x7b080034d800 after lost signal to thread 1826
I20260709 13:17:28.501915  2045 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.517118  2039 leader_election.cc:290] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.520582  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
W20260709 13:17:28.521131  1823 debug-util.cc:398] Leaking SignalData structure 0x7b08000edb60 after lost signal to thread 1901
I20260709 13:17:28.521535  1875 raft_consensus.cc:3060] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:28.522969  1823 debug-util.cc:398] Leaking SignalData structure 0x7b0800219000 after lost signal to thread 1975
I20260709 13:17:28.505487  2137 consensus_queue.cc:1048] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:17:28.520645  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:28.531276  1802 raft_consensus.cc:3060] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.528926  2106 consensus_queue.cc:237] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.556720  1802 raft_consensus.cc:2468] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.565027  1915 leader_election.cc:304] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.566154  2039 raft_consensus.cc:2804] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.567082  2039 raft_consensus.cc:697] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:28.566063  1875 raft_consensus.cc:2468] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.568475  2092 tablet_bootstrap.cc:654] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.568049  2039 consensus_queue.cc:237] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.577450  2045 raft_consensus.cc:2749] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.583732  2092 tablet_bootstrap.cc:492] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.584297  2092 ts_tablet_manager.cc:1403] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.002s
I20260709 13:17:28.587280  2092 raft_consensus.cc:359] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.588152  2092 raft_consensus.cc:385] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.588500  2092 raft_consensus.cc:740] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.589084  2093 consensus_queue.cc:1048] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.589257  2092 consensus_queue.cc:260] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.592219  2092 ts_tablet_manager.cc:1434] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:28.593981  2092 tablet_bootstrap.cc:492] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: Bootstrap starting.
I20260709 13:17:28.596730  1698 catalog_manager.cc:5697] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 1 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.603044  2092 tablet_bootstrap.cc:654] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.610793  2106 raft_consensus.cc:493] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.615852  2106 raft_consensus.cc:515] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.619712  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.620507  1875 raft_consensus.cc:2468] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:28.620771  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:28.621826  1948 raft_consensus.cc:2468] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:28.622324  1766 leader_election.cc:304] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:28.623600  2045 raft_consensus.cc:2804] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.624107  2045 raft_consensus.cc:493] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.624724  2045 raft_consensus.cc:3060] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.637795  2106 leader_election.cc:290] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.640076  2092 tablet_bootstrap.cc:492] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: No bootstrap required, opened a new log
I20260709 13:17:28.640712  2092 ts_tablet_manager.cc:1403] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.011s
I20260709 13:17:28.643994  2092 raft_consensus.cc:359] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.644858  2092 raft_consensus.cc:385] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.645193  2092 raft_consensus.cc:740] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.646464  2092 consensus_queue.cc:260] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.649410  2092 ts_tablet_manager.cc:1434] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:17:28.651415  2106 raft_consensus.cc:493] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.652058  2106 raft_consensus.cc:515] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.655357  2045 raft_consensus.cc:515] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.659638  2039 consensus_queue.cc:1048] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.660804  1698 catalog_manager.cc:5697] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.664162  2147 consensus_queue.cc:1048] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.667484  2106 leader_election.cc:290] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.669848  2045 leader_election.cc:290] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.674343  1874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:28.675473  1874 raft_consensus.cc:3060] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.678308  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.679173  1876 raft_consensus.cc:2468] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:28.693338  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952" is_pre_election: true
I20260709 13:17:28.689350  1875 raft_consensus.cc:3060] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.695142  1766 leader_election.cc:304] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:28.697952  1951 raft_consensus.cc:2468] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 0.
I20260709 13:17:28.698184  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:28.698964  1948 raft_consensus.cc:3060] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.717196  1948 raft_consensus.cc:2468] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:28.720021  1766 leader_election.cc:304] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.721095  2106 raft_consensus.cc:2804] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.721680  2106 raft_consensus.cc:697] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:28.722765  2106 consensus_queue.cc:237] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.726964  1874 raft_consensus.cc:2468] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:28.709748  2045 raft_consensus.cc:2804] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.733990  2045 raft_consensus.cc:493] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.734476  2045 raft_consensus.cc:3060] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.735121  2100 consensus_queue.cc:1048] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.752455  2045 raft_consensus.cc:515] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.755846  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:28.756542  1875 raft_consensus.cc:3060] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.757545  1948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:28.758348  1948 raft_consensus.cc:3060] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.755863  1699 catalog_manager.cc:5697] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 1 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.767175  1875 raft_consensus.cc:2468] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:28.770124  1766 leader_election.cc:304] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:28.771837  2137 raft_consensus.cc:2804] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.772430  2137 raft_consensus.cc:697] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:28.773415  2137 consensus_queue.cc:237] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.776050  2100 raft_consensus.cc:493] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.776664  2100 raft_consensus.cc:515] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.779026  2100 leader_election.cc:290] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.776481  1948 raft_consensus.cc:2468] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 1.
I20260709 13:17:28.781225  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:28.782053  1799 raft_consensus.cc:2468] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.783924  1915 leader_election.cc:304] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.785161  2100 raft_consensus.cc:2804] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.785725  2100 raft_consensus.cc:493] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.786127  2100 raft_consensus.cc:3060] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.786677  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.787367  1875 raft_consensus.cc:2468] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.773849  2045 leader_election.cc:290] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 1 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:28.796707  2100 raft_consensus.cc:515] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.800196  2100 leader_election.cc:290] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.802845  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:28.802944  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:28.803747  1875 raft_consensus.cc:3060] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.803864  1799 raft_consensus.cc:3060] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.805156  1698 catalog_manager.cc:5697] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 1 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.813623  1875 raft_consensus.cc:2468] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.814157  1799 raft_consensus.cc:2468] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.815434  1915 leader_election.cc:304] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.816313  2100 raft_consensus.cc:2804] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.817183  2100 raft_consensus.cc:697] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:28.818252  2100 consensus_queue.cc:237] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.828954  1698 catalog_manager.cc:5697] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.843755  2100 raft_consensus.cc:493] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.844558  2100 raft_consensus.cc:515] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.847276  2100 leader_election.cc:290] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.848229  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c" is_pre_election: true
I20260709 13:17:28.848421  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73" is_pre_election: true
I20260709 13:17:28.849290  1875 raft_consensus.cc:2468] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.849306  1799 raft_consensus.cc:2468] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 0.
I20260709 13:17:28.850719  1915 leader_election.cc:304] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.851532  2100 raft_consensus.cc:2804] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.851979  2100 raft_consensus.cc:493] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.852342  2100 raft_consensus.cc:3060] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.863794  2100 raft_consensus.cc:515] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.866276  2100 leader_election.cc:290] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:28.867323  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:28.867323  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:28.868779  1875 raft_consensus.cc:3060] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.869156  1799 raft_consensus.cc:3060] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.875551  1799 raft_consensus.cc:2468] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.876785  1915 leader_election.cc:304] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:28.877573  2100 raft_consensus.cc:2804] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.878079  2100 raft_consensus.cc:697] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:28.879001  2100 consensus_queue.cc:237] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:28.880494  1875 raft_consensus.cc:2468] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 1.
I20260709 13:17:28.889983  1698 catalog_manager.cc:5697] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 1 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:29.017155  2137 consensus_queue.cc:1048] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.035233  2106 consensus_queue.cc:1048] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.113417  1875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
mode: GRACEFUL
new_leader_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59108
I20260709 13:17:29.114176  1875 raft_consensus.cc:606] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Received request to transfer leadership to TS d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:29.122556  1951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:29.123494  1951 raft_consensus.cc:606] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:29.131235  1951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:29.132131  1951 raft_consensus.cc:606] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS d4d2de778a80423c8ea7230963568a1c
I20260709 13:17:29.142527  2100 consensus_queue.cc:1048] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.157627  2100 consensus_queue.cc:1048] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.170859  1799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbce222191f9408397faebf06a41ba57"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59660
I20260709 13:17:29.171768  1799 raft_consensus.cc:606] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:29.185173  2100 raft_consensus.cc:993] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: : Instructing follower d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:29.185784  2100 raft_consensus.cc:1081] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:29.195466  1802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59660
I20260709 13:17:29.196370  1802 raft_consensus.cc:606] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:29.196604  1801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59644
I20260709 13:17:29.197331  1801 raft_consensus.cc:493] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.197777  1801 raft_consensus.cc:3060] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.208113  1801 raft_consensus.cc:515] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.210641  1801 leader_election.cc:290] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.212064  1874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:29.212791  1874 raft_consensus.cc:3060] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.218768  1951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:29.219410  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361936d22bf24421b2af4d6c0510298f" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.219854  1951 raft_consensus.cc:606] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:29.220206  1949 raft_consensus.cc:3055] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.220623  1949 raft_consensus.cc:740] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:29.221614  1949 consensus_queue.cc:260] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.223201  1949 raft_consensus.cc:3060] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.224673  1874 raft_consensus.cc:2468] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:29.228809  2135 consensus_queue.cc:1048] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.230525  2135 consensus_queue.cc:1048] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.234048  1949 raft_consensus.cc:2468] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:29.238629  1766 leader_election.cc:304] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:29.239603  2135 raft_consensus.cc:2804] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.240622  1951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
mode: GRACEFUL
new_leader_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:39862
I20260709 13:17:29.243146  1951 raft_consensus.cc:606] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Received request to transfer leadership to TS e91ba25a6a344f70aef0384659b3fd73
I20260709 13:17:29.243377  2162 consensus_queue.cc:1048] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.242470  2135 raft_consensus.cc:697] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:29.245221  2135 consensus_queue.cc:237] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.265285  1697 catalog_manager.cc:5697] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.286152  2106 consensus_queue.cc:1048] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.315793  2100 consensus_queue.cc:1048] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.321025  2106 consensus_queue.cc:1048] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.325985  1801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
mode: GRACEFUL
new_leader_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:59660
I20260709 13:17:29.326812  1801 raft_consensus.cc:606] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Received request to transfer leadership to TS f9e8deca048d4039851f55fe03299952
I20260709 13:17:29.350329  2045 raft_consensus.cc:993] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.350867  2045 raft_consensus.cc:1081] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.357870  1874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59092
I20260709 13:17:29.358819  1874 raft_consensus.cc:493] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.359350  1874 raft_consensus.cc:3060] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.369738  1874 raft_consensus.cc:515] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.371488  2147 raft_consensus.cc:993] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: : Instructing follower d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:29.372138  2147 raft_consensus.cc:1081] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Signalling peer d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:29.372360  1874 leader_election.cc:290] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.376070  1802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59654
I20260709 13:17:29.376713  1802 raft_consensus.cc:493] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.377107  1802 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.376946  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.378031  2100 consensus_queue.cc:1048] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.379837  2100 raft_consensus.cc:993] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: : Instructing follower d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:29.380266  2100 raft_consensus.cc:1081] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer d4d2de778a80423c8ea7230963568a1c to start an election
I20260709 13:17:29.379930  1876 raft_consensus.cc:1240] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Rejecting Update request from peer d4d2de778a80423c8ea7230963568a1c for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:29.382166  2106 raft_consensus.cc:993] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: : Instructing follower f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:29.382712  2106 raft_consensus.cc:1081] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Signalling peer f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:29.382637  2172 consensus_queue.cc:1059] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.383706  2045 raft_consensus.cc:3055] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.384078  2045 raft_consensus.cc:740] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:29.384950  2045 consensus_queue.cc:260] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.386348  2045 raft_consensus.cc:3060] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.387948  1801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4bb4416aa6442e09813b3cf9e8237f8" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:29.389460  1800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
 from {username='slave'} at 127.0.0.1:59644
I20260709 13:17:29.390151  1800 raft_consensus.cc:493] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.390513  1800 raft_consensus.cc:3060] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.395042  1948 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:39848
I20260709 13:17:29.395778  1948 raft_consensus.cc:493] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.396183  1948 raft_consensus.cc:3060] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.400045  1800 raft_consensus.cc:515] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.402760  1800 leader_election.cc:290] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.403507  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:29.404263  1876 raft_consensus.cc:3060] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.404737  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3908698cf2654827abcfe6ae5933096c" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.405364  1951 raft_consensus.cc:3055] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.405709  1951 raft_consensus.cc:740] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:29.406594  1951 consensus_queue.cc:260] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.408030  1951 raft_consensus.cc:3060] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.412621  1876 raft_consensus.cc:2468] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:29.413936  1766 leader_election.cc:304] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:29.414734  2045 raft_consensus.cc:2804] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.415228  2045 raft_consensus.cc:697] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:29.416100  2045 consensus_queue.cc:237] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.424444  1951 raft_consensus.cc:2468] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:29.425362  1948 raft_consensus.cc:515] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.424052  1802 raft_consensus.cc:515] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.431671  1802 leader_election.cc:290] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Requested vote from peers e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.433816  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.434690  1949 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.437599  1801 raft_consensus.cc:2468] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.440021  1841 leader_election.cc:304] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: f9e8deca048d4039851f55fe03299952
I20260709 13:17:29.442803  2166 raft_consensus.cc:2804] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.443472  2166 raft_consensus.cc:697] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:29.444530  2166 consensus_queue.cc:237] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.446244  1876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "d4d2de778a80423c8ea7230963568a1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:29.447227  1876 raft_consensus.cc:3055] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.447618  1876 raft_consensus.cc:740] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:29.448872  1876 consensus_queue.cc:260] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.454751  1875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:29.463444  1875 raft_consensus.cc:3060] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.451608  1948 leader_election.cc:290] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:29.470398  1698 catalog_manager.cc:5697] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from d4d2de778a80423c8ea7230963568a1c (127.21.33.1) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.471571  1875 raft_consensus.cc:2468] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 2.
I20260709 13:17:29.471491  1949 raft_consensus.cc:2468] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:29.474222  1915 leader_election.cc:304] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:29.474310  1766 leader_election.cc:304] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:29.475207  2039 raft_consensus.cc:2804] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.475654  2135 raft_consensus.cc:2804] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.475759  2039 raft_consensus.cc:697] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 2 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:29.476245  2135 raft_consensus.cc:697] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Leader. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:29.477023  2039 consensus_queue.cc:237] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.462392  2162 consensus_queue.cc:1048] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:17:29.477279  2135 consensus_queue.cc:237] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.455415  1876 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.453944  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4123a7d6ed643d2a7213ca3f34e7b1d" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:29.484205  1800 raft_consensus.cc:3055] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.484570  1800 raft_consensus.cc:740] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:29.485591  1800 consensus_queue.cc:260] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.487172  1800 raft_consensus.cc:3060] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.492022  1699 catalog_manager.cc:5697] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 1 to 2, leader changed from d4d2de778a80423c8ea7230963568a1c (127.21.33.1) to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 2 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:29.511715  2045 raft_consensus.cc:993] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.512461  2045 raft_consensus.cc:1081] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.514568  1800 raft_consensus.cc:2468] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 2.
I20260709 13:17:29.519084  1874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbce222191f9408397faebf06a41ba57"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59092
I20260709 13:17:29.520061  1874 raft_consensus.cc:493] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.520272  1876 raft_consensus.cc:2468] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d2de778a80423c8ea7230963568a1c in term 2.
I20260709 13:17:29.520608  1874 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.525302  1697 catalog_manager.cc:5697] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.536578  2100 raft_consensus.cc:993] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.537310  2100 raft_consensus.cc:1081] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.539695  1876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59076
I20260709 13:17:29.540452  1876 raft_consensus.cc:493] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.541052  1876 raft_consensus.cc:3060] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.549559  1874 raft_consensus.cc:515] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.551524  1876 raft_consensus.cc:515] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.553077  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:29.553818  1800 raft_consensus.cc:3055] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.553853  1876 leader_election.cc:290] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.554380  1800 raft_consensus.cc:740] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:29.555419  1800 consensus_queue.cc:260] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.557137  1799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:29.557596  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63162ba797b434ca275ef7b6299520f" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.557832  1799 raft_consensus.cc:3060] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.558269  1949 raft_consensus.cc:3055] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.558653  1949 raft_consensus.cc:740] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:29.559984  1949 consensus_queue.cc:260] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.562013  1949 raft_consensus.cc:3060] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.566545  1799 raft_consensus.cc:2468] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.566033  1700 catalog_manager.cc:5697] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c reported cstate change: term changed from 1 to 2, leader changed from e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2) to d4d2de778a80423c8ea7230963568a1c (127.21.33.1). New cstate: current_term: 2 leader_uuid: "d4d2de778a80423c8ea7230963568a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.567831  1874 leader_election.cc:290] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.568048  1841 leader_election.cc:304] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, e91ba25a6a344f70aef0384659b3fd73; no voters: 
I20260709 13:17:29.570914  2166 raft_consensus.cc:2804] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.571410  2166 raft_consensus.cc:697] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:29.572562  2166 consensus_queue.cc:237] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.575723  1951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.569084  1800 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.577400  1949 raft_consensus.cc:2468] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.577662  1951 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.594758  1800 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.595044  1951 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.596591  1841 leader_election.cc:304] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:29.598228  2166 raft_consensus.cc:2804] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.598757  2166 raft_consensus.cc:697] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:29.599563  2166 consensus_queue.cc:237] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.609561  1697 catalog_manager.cc:5697] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.611752  1697 catalog_manager.cc:5697] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from d4d2de778a80423c8ea7230963568a1c (127.21.33.1) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.649101  1874 raft_consensus.cc:1275] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.650840  2135 consensus_queue.cc:1048] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:29.661196  1951 raft_consensus.cc:1275] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.666139  2135 consensus_queue.cc:1048] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:29.765885  2100 raft_consensus.cc:993] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: : Instructing follower e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.766592  2176 raft_consensus.cc:1081] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Signalling peer e91ba25a6a344f70aef0384659b3fd73 to start an election
I20260709 13:17:29.768486  1874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
 from {username='slave'} at 127.0.0.1:59076
I20260709 13:17:29.769258  1874 raft_consensus.cc:493] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.769735  1874 raft_consensus.cc:3060] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.779428  1874 raft_consensus.cc:515] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.781997  1874 leader_election.cc:290] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), f9e8deca048d4039851f55fe03299952 (127.21.33.3:41281)
I20260709 13:17:29.783373  1949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9e8deca048d4039851f55fe03299952"
I20260709 13:17:29.783962  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1888a2cd9a9245ef8ae30cebf3fc1401" candidate_uuid: "e91ba25a6a344f70aef0384659b3fd73" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:29.784166  1949 raft_consensus.cc:3055] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.784556  1949 raft_consensus.cc:740] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:29.784614  1800 raft_consensus.cc:3060] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.785616  1949 consensus_queue.cc:260] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.786906  1949 raft_consensus.cc:3060] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.795993  1800 raft_consensus.cc:2468] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.796988  1949 raft_consensus.cc:2468] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91ba25a6a344f70aef0384659b3fd73 in term 2.
I20260709 13:17:29.798167  1841 leader_election.cc:304] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:29.799362  2166 raft_consensus.cc:2804] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.799885  2166 raft_consensus.cc:697] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Leader. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:29.800756  2166 consensus_queue.cc:237] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:29.809127  1698 catalog_manager.cc:5697] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 reported cstate change: term changed from 1 to 2, leader changed from f9e8deca048d4039851f55fe03299952 (127.21.33.3) to e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2). New cstate: current_term: 2 leader_uuid: "e91ba25a6a344f70aef0384659b3fd73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.860528  1949 raft_consensus.cc:1275] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.862684  2135 consensus_queue.cc:1048] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:29.881466  1874 raft_consensus.cc:1275] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.892432  2106 consensus_queue.cc:1048] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:29.894865  1800 raft_consensus.cc:1275] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.901480  2166 consensus_queue.cc:1048] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:29.915722  1951 raft_consensus.cc:3060] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.921033  1874 raft_consensus.cc:1275] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.926702  2176 consensus_queue.cc:1048] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:29.926890  1951 raft_consensus.cc:1275] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.930043  2163 consensus_queue.cc:1048] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.943975  1800 raft_consensus.cc:1275] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.949913  2100 consensus_queue.cc:1048] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:29.992236  1951 raft_consensus.cc:1275] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:29.993680  2045 consensus_queue.cc:1048] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.003084  1951 raft_consensus.cc:1275] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.005277  1874 raft_consensus.cc:1275] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Refusing update from remote peer d4d2de778a80423c8ea7230963568a1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.006600  2166 consensus_queue.cc:1048] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:30.007443  2106 consensus_queue.cc:1048] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.021965  1800 raft_consensus.cc:1275] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.023921  2163 consensus_queue.cc:1048] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:30.028891  1799 raft_consensus.cc:1275] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.035130  2163 consensus_queue.cc:1048] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.066291  1951 raft_consensus.cc:1275] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.072467  2180 consensus_queue.cc:1048] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:30.378888  1951 raft_consensus.cc:1275] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.380686  2147 consensus_queue.cc:1048] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.393307  1800 raft_consensus.cc:1275] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Refusing update from remote peer e91ba25a6a344f70aef0384659b3fd73: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.400161  2163 consensus_queue.cc:1048] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:32.334399  1699 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41842:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:17:32.358907  1699 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41842:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:17:32.397167 21636 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:17:32.547902  2208 log.cc:927] Time spent T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:17:32.634702  2223 tablet.cc:1724] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:32.698179  2218 log.cc:927] Time spent T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 13:17:32.710533  1841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:41281, user_credentials={real_user=slave}} blocked reactor thread for 96628.7us
W20260709 13:17:32.754086  1766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:40885, user_credentials={real_user=slave}} blocked reactor thread for 77194.4us
I20260709 13:17:32.757637  2237 tablet.cc:1724] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:32.760073  1828 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:372e2af2548a46d39bb7577e4d7e56ff)
I20260709 13:17:32.780992  2245 tablet.cc:1724] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:32.787295  1828 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:372e2af2548a46d39bb7577e4d7e56ff)
W20260709 13:17:32.788465  1828 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:372e2af2548a46d39bb7577e4d7e56ff)
I20260709 13:17:32.797742  2246 tablet.cc:1724] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.827492  2238 tablet.cc:1724] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.839464  1699 catalog_manager.cc:5789] TS d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529) does not have the latest schema for tablet 3908698cf2654827abcfe6ae5933096c (table soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]). Expected version 1 got 0
I20260709 13:17:32.844795  1699 catalog_manager.cc:5789] TS d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529) does not have the latest schema for tablet 361936d22bf24421b2af4d6c0510298f (table soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]). Expected version 1 got 0
I20260709 13:17:32.845794  1699 catalog_manager.cc:5789] TS d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529) does not have the latest schema for tablet ccd0925475c14c1abdcc732703f9fbd6 (table soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d]). Expected version 1 got 0
I20260709 13:17:32.795734  2240 tablet.cc:1724] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.873238  2239 tablet.cc:1724] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.902283  2254 tablet.cc:1724] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.879220  2247 tablet.cc:1724] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.925983  2223 tablet.cc:1724] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:32.929795  2235 log.cc:927] Time spent T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: Append to log took a long time: real 0.200s	user 0.004s	sys 0.000s
I20260709 13:17:32.931480  2252 tablet.cc:1724] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.986462  2249 tablet.cc:1724] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.990067  2253 tablet.cc:1724] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.995191  2238 tablet.cc:1724] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.005725  2240 tablet.cc:1724] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:33.033135  1766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.2:40885, user_credentials={real_user=slave}} blocked reactor thread for 159372us
I20260709 13:17:33.034574  2245 tablet.cc:1724] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.961081  2239 tablet.cc:1724] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.040359  2246 tablet.cc:1724] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.046885  1797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
mode: GRACEFUL
new_leader_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:41016
I20260709 13:17:33.047685  1797 raft_consensus.cc:606] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Received request to transfer leadership to TS f9e8deca048d4039851f55fe03299952
I20260709 13:17:33.050081  2237 tablet.cc:1724] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.050454  2247 tablet.cc:1724] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.053953  2249 tablet.cc:1724] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.061221  2253 tablet.cc:1724] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.080255  2223 tablet.cc:1724] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.082892  2258 raft_consensus.cc:993] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: : Instructing follower f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.083717  2257 raft_consensus.cc:1081] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Signalling peer f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.085319  2252 tablet.cc:1724] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.094574  2238 tablet.cc:1724] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.100304  1949 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:39848
I20260709 13:17:33.101805  1949 raft_consensus.cc:493] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.102187  1949 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.113828  2240 tablet.cc:1724] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.123231  2254 tablet.cc:1724] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.168926  1874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbce222191f9408397faebf06a41ba57"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
mode: GRACEFUL
new_leader_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:37414
I20260709 13:17:33.169782  1874 raft_consensus.cc:606] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Received request to transfer leadership to TS f9e8deca048d4039851f55fe03299952
I20260709 13:17:33.171401 21636 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:17:33.172187 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:33.172370  1699 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be: soft_deleted_table [id=92164b39a4be44a3a7f9906467a8439d] alter complete (version 1)
I20260709 13:17:33.172079  1949 raft_consensus.cc:515] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.180781  1949 leader_election.cc:290] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:33.183123  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:33.183804  1802 raft_consensus.cc:3055] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:33.183782  1874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372e2af2548a46d39bb7577e4d7e56ff" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:33.184433  1802 raft_consensus.cc:740] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:33.184839  1874 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.185326  1802 consensus_queue.cc:260] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.187309  1802 raft_consensus.cc:3060] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.200094  1802 raft_consensus.cc:2468] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.201965  1915 leader_election.cc:304] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:33.203153  2230 raft_consensus.cc:2804] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:33.203683  2230 raft_consensus.cc:697] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:33.204629  2230 consensus_queue.cc:237] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.205375  1874 raft_consensus.cc:2468] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.218809  1697 catalog_manager.cc:5697] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 2 to 3, leader changed from d4d2de778a80423c8ea7230963568a1c (127.21.33.1) to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.236642  1874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f"
dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
mode: GRACEFUL
new_leader_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:37414
I20260709 13:17:33.237435  1874 raft_consensus.cc:606] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Received request to transfer leadership to TS f9e8deca048d4039851f55fe03299952
I20260709 13:17:33.242683  1802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "484797364d6049a68a06966fb4660242"
dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
mode: GRACEFUL
new_leader_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:41016
I20260709 13:17:33.243319  1802 raft_consensus.cc:606] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Received request to transfer leadership to TS f9e8deca048d4039851f55fe03299952
I20260709 13:17:33.244436 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:33.245124 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:33.265203  2258 raft_consensus.cc:993] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: : Instructing follower f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.265900  2258 raft_consensus.cc:1081] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Signalling peer f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.267984  1949 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "484797364d6049a68a06966fb4660242"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:39848
I20260709 13:17:33.268621  1949 raft_consensus.cc:493] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.269050  1949 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.283310  1949 raft_consensus.cc:515] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.285107  1949 leader_election.cc:290] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:33.286736  1874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:33.286850  1802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484797364d6049a68a06966fb4660242" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:33.287432  1874 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.287783  1802 raft_consensus.cc:3055] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:33.288367  1802 raft_consensus.cc:740] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d4d2de778a80423c8ea7230963568a1c, State: Running, Role: LEADER
I20260709 13:17:33.289443  1802 consensus_queue.cc:260] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.290856  1802 raft_consensus.cc:3060] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.302882  1874 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.304173  1915 leader_election.cc:304] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e91ba25a6a344f70aef0384659b3fd73, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:33.305090  2100 raft_consensus.cc:2804] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:33.305822  2100 raft_consensus.cc:697] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:33.306747  2100 consensus_queue.cc:237] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.307725  2224 raft_consensus.cc:993] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: : Instructing follower f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.308279  2224 raft_consensus.cc:1081] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Signalling peer f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.310245  1950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:39840
I20260709 13:17:33.310858  1950 raft_consensus.cc:493] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.311255  1950 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.334472  1802 raft_consensus.cc:2468] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.340651  1950 raft_consensus.cc:515] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.345172  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:33.346163  1800 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.361788  1950 leader_election.cc:290] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:33.362215  1874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dce46c69af941cc81ab6459e9e3424f" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:33.362912  1874 raft_consensus.cc:3055] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:33.363255  1874 raft_consensus.cc:740] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:33.364075  1874 consensus_queue.cc:260] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.364269  1800 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.365483  1874 raft_consensus.cc:3060] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.366686  1915 leader_election.cc:304] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:33.367578  2100 raft_consensus.cc:2804] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:33.368101  2100 raft_consensus.cc:697] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:33.368386  1700 catalog_manager.cc:5697] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 2 to 3, leader changed from d4d2de778a80423c8ea7230963568a1c (127.21.33.1) to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.369009  2100 consensus_queue.cc:237] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.408625  1874 raft_consensus.cc:2468] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.417035  1700 catalog_manager.cc:5697] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 2 to 3, leader changed from e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2) to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.619915  2224 raft_consensus.cc:993] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: : Instructing follower f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.620645  2185 raft_consensus.cc:1081] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Signalling peer f9e8deca048d4039851f55fe03299952 to start an election
I20260709 13:17:33.622731  1950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbce222191f9408397faebf06a41ba57"
dest_uuid: "f9e8deca048d4039851f55fe03299952"
 from {username='slave'} at 127.0.0.1:39840
I20260709 13:17:33.623355  1950 raft_consensus.cc:493] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.623723  1950 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.630461  1950 raft_consensus.cc:515] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.632800  1950 leader_election.cc:290] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Requested vote from peers d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529), e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885)
I20260709 13:17:33.633720  1800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d4d2de778a80423c8ea7230963568a1c"
I20260709 13:17:33.634552  1800 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.635856  1874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbce222191f9408397faebf06a41ba57" candidate_uuid: "f9e8deca048d4039851f55fe03299952" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e91ba25a6a344f70aef0384659b3fd73"
I20260709 13:17:33.636592  1874 raft_consensus.cc:3055] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:33.636931  1874 raft_consensus.cc:740] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e91ba25a6a344f70aef0384659b3fd73, State: Running, Role: LEADER
I20260709 13:17:33.637823  1874 consensus_queue.cc:260] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.639261  1874 raft_consensus.cc:3060] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.643533  1800 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.644982  1915 leader_election.cc:304] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4d2de778a80423c8ea7230963568a1c, f9e8deca048d4039851f55fe03299952; no voters: 
I20260709 13:17:33.645905  2100 raft_consensus.cc:2804] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:33.646517  2100 raft_consensus.cc:697] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Becoming Leader. State: Replica: f9e8deca048d4039851f55fe03299952, State: Running, Role: LEADER
I20260709 13:17:33.647531  1874 raft_consensus.cc:2468] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e8deca048d4039851f55fe03299952 in term 3.
I20260709 13:17:33.647576  2100 consensus_queue.cc:237] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } }
I20260709 13:17:33.657678  1700 catalog_manager.cc:5697] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 reported cstate change: term changed from 2 to 3, leader changed from e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2) to f9e8deca048d4039851f55fe03299952 (127.21.33.3). New cstate: current_term: 3 leader_uuid: "f9e8deca048d4039851f55fe03299952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e8deca048d4039851f55fe03299952" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 41281 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.688234  1800 raft_consensus.cc:1275] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:33.689785  2100 consensus_queue.cc:1048] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:33.706360  1874 raft_consensus.cc:1275] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:33.708781  2100 consensus_queue.cc:1048] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:33.748967  1800 raft_consensus.cc:1275] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:33.750790  2100 consensus_queue.cc:1048] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:33.761149  1874 raft_consensus.cc:1275] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:33.763343  2100 consensus_queue.cc:1048] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:33.956954  1802 raft_consensus.cc:1275] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:33.959550  2270 consensus_queue.cc:1048] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:33.983126  1875 raft_consensus.cc:1275] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:33.987387  2100 consensus_queue.cc:1048] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:34.066985  1875 raft_consensus.cc:1275] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:34.069629  2270 consensus_queue.cc:1048] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91ba25a6a344f70aef0384659b3fd73" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:34.090070  1802 raft_consensus.cc:1275] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Refusing update from remote peer f9e8deca048d4039851f55fe03299952: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:34.092468  2270 consensus_queue.cc:1048] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d2de778a80423c8ea7230963568a1c" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:34.321744 21636 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:17:34.329581 21636 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:17:34.330240 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:34.336194 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:17:34.336730 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:35.382772 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
W20260709 13:17:35.392212  1915 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 13:17:35.399525  1915 consensus_peers.cc:597] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.400326  1915 consensus_peers.cc:597] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.442436 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:35.443691 21636 tablet_replica.cc:333] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.445313 21636 raft_consensus.cc:2243] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:35.446950 21636 raft_consensus.cc:2272] T 3908698cf2654827abcfe6ae5933096c P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.451700 21636 tablet_replica.cc:333] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.452951 21636 raft_consensus.cc:2243] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.454026 21636 raft_consensus.cc:2272] T 1888a2cd9a9245ef8ae30cebf3fc1401 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:35.454268  1841 consensus_peers.cc:597] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.458021 21636 tablet_replica.cc:333] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.459261 21636 raft_consensus.cc:2243] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.460065 21636 raft_consensus.cc:2272] T 372e2af2548a46d39bb7577e4d7e56ff P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.464285 21636 tablet_replica.cc:333] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.465356 21636 raft_consensus.cc:2243] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.466367 21636 raft_consensus.cc:2272] T f63162ba797b434ca275ef7b6299520f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:35.470093  1841 consensus_peers.cc:597] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.470827 21636 tablet_replica.cc:333] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.471681 21636 raft_consensus.cc:2243] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:35.472856 21636 raft_consensus.cc:2272] T 361936d22bf24421b2af4d6c0510298f P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.476333 21636 tablet_replica.cc:333] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.477380 21636 raft_consensus.cc:2243] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:35.478932 21636 raft_consensus.cc:2272] T ccd0925475c14c1abdcc732703f9fbd6 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.482336 21636 tablet_replica.cc:333] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.483095 21636 raft_consensus.cc:2243] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.483755 21636 raft_consensus.cc:2272] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.487164 21636 tablet_replica.cc:333] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.487978 21636 raft_consensus.cc:2243] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.488497 21636 raft_consensus.cc:2272] T dbce222191f9408397faebf06a41ba57 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.492101 21636 tablet_replica.cc:333] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.493127 21636 raft_consensus.cc:2243] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:35.494665 21636 raft_consensus.cc:2272] T 025ec36b5ae54e11a43c4f78d870c2fc P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.498134 21636 tablet_replica.cc:333] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.499154 21636 raft_consensus.cc:2243] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:35.500725 21636 raft_consensus.cc:2272] T 7563b7d4737943d6987c8a3451d85a75 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.503770 21636 tablet_replica.cc:333] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.504643 21636 raft_consensus.cc:2243] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.505645 21636 raft_consensus.cc:2272] T ff66ac876d0546a19d3f5002d1f77848 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.508634 21636 tablet_replica.cc:333] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.509254 21636 raft_consensus.cc:2243] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.509781 21636 raft_consensus.cc:2272] T 3dce46c69af941cc81ab6459e9e3424f P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.512584 21636 tablet_replica.cc:333] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.513633 21636 raft_consensus.cc:2243] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:35.515151 21636 raft_consensus.cc:2272] T 5d190db5d02f418d80ee52a83ea60119 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:35.518253  1841 consensus_peers.cc:597] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.519965 21636 tablet_replica.cc:333] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.520725 21636 raft_consensus.cc:2243] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.521410 21636 raft_consensus.cc:2272] T 484797364d6049a68a06966fb4660242 P d4d2de778a80423c8ea7230963568a1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.524928 21636 tablet_replica.cc:333] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.526039 21636 raft_consensus.cc:2243] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.527388 21636 raft_consensus.cc:2272] T c4123a7d6ed643d2a7213ca3f34e7b1d P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:35.529287  1841 consensus_peers.cc:597] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.531118 21636 tablet_replica.cc:333] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.531970 21636 raft_consensus.cc:2243] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.532858 21636 raft_consensus.cc:2272] T b4bb4416aa6442e09813b3cf9e8237f8 P d4d2de778a80423c8ea7230963568a1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.536525 21636 tablet_replica.cc:333] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.537170 21636 raft_consensus.cc:2243] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.537882 21636 raft_consensus.cc:2272] T e1c65f83f85d44d1b7525a643c8fe3e8 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:35.541394 21636 tablet_replica.cc:333] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c: stopping tablet replica
I20260709 13:17:35.542062 21636 raft_consensus.cc:2243] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:35.542639 21636 raft_consensus.cc:2272] T 011ac8a199d24d52a117549237f0d442 P d4d2de778a80423c8ea7230963568a1c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:35.581054  1915 consensus_peers.cc:597] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.604207  1841 consensus_peers.cc:597] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.654976  1915 consensus_peers.cc:597] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.757627  1841 consensus_peers.cc:597] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.788791  1915 consensus_peers.cc:597] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.832357  1915 consensus_peers.cc:597] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 -> Peer d4d2de778a80423c8ea7230963568a1c (127.21.33.1:38529): Couldn't send request to peer d4d2de778a80423c8ea7230963568a1c. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:38529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.926550 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
I20260709 13:17:36.000550 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
W20260709 13:17:36.040666  1915 consensus_peers.cc:597] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 -> Peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Couldn't send request to peer e91ba25a6a344f70aef0384659b3fd73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.044960 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:36.046053 21636 tablet_replica.cc:333] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.047171 21636 raft_consensus.cc:2243] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.048331 21636 raft_consensus.cc:2272] T 1888a2cd9a9245ef8ae30cebf3fc1401 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.051302 21636 tablet_replica.cc:333] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.052014 21636 raft_consensus.cc:2243] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.052587 21636 raft_consensus.cc:2272] T 3908698cf2654827abcfe6ae5933096c P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.056375 21636 tablet_replica.cc:333] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.057646 21636 raft_consensus.cc:2243] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.058614 21636 raft_consensus.cc:2272] T 361936d22bf24421b2af4d6c0510298f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:36.060437  1915 consensus_peers.cc:597] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 -> Peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Couldn't send request to peer e91ba25a6a344f70aef0384659b3fd73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.062717 21636 tablet_replica.cc:333] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.063532 21636 raft_consensus.cc:2243] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.064522 21636 raft_consensus.cc:2272] T f63162ba797b434ca275ef7b6299520f P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.067520 21636 tablet_replica.cc:333] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.068671 21636 raft_consensus.cc:2243] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.070120 21636 raft_consensus.cc:2272] T ccd0925475c14c1abdcc732703f9fbd6 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:36.073611  1915 consensus_peers.cc:597] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 -> Peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Couldn't send request to peer e91ba25a6a344f70aef0384659b3fd73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.074457 21636 tablet_replica.cc:333] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.075227 21636 raft_consensus.cc:2243] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.075798 21636 raft_consensus.cc:2272] T 7563b7d4737943d6987c8a3451d85a75 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.079236 21636 tablet_replica.cc:333] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.079943 21636 raft_consensus.cc:2243] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.080485 21636 raft_consensus.cc:2272] T dbce222191f9408397faebf06a41ba57 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.082864 21636 tablet_replica.cc:333] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.083518 21636 raft_consensus.cc:2243] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.084075 21636 raft_consensus.cc:2272] T 025ec36b5ae54e11a43c4f78d870c2fc P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.087477 21636 tablet_replica.cc:333] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.088478 21636 raft_consensus.cc:2243] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.089239 21636 raft_consensus.cc:2272] T 3dce46c69af941cc81ab6459e9e3424f P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.092447 21636 tablet_replica.cc:333] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.093173 21636 raft_consensus.cc:2243] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.094403 21636 raft_consensus.cc:2272] T ff66ac876d0546a19d3f5002d1f77848 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.097326 21636 tablet_replica.cc:333] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.098059 21636 raft_consensus.cc:2243] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.098628 21636 raft_consensus.cc:2272] T 011ac8a199d24d52a117549237f0d442 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.102205 21636 tablet_replica.cc:333] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.102900 21636 raft_consensus.cc:2243] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.103857 21636 raft_consensus.cc:2272] T 372e2af2548a46d39bb7577e4d7e56ff P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:36.104406  1915 consensus_peers.cc:597] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 -> Peer e91ba25a6a344f70aef0384659b3fd73 (127.21.33.2:40885): Couldn't send request to peer e91ba25a6a344f70aef0384659b3fd73. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.107481 21636 tablet_replica.cc:333] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.108593 21636 raft_consensus.cc:2243] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.109304 21636 raft_consensus.cc:2272] T 5d190db5d02f418d80ee52a83ea60119 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.112994 21636 tablet_replica.cc:333] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.113826 21636 raft_consensus.cc:2243] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.114830 21636 raft_consensus.cc:2272] T b4bb4416aa6442e09813b3cf9e8237f8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.117663 21636 tablet_replica.cc:333] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.118264 21636 raft_consensus.cc:2243] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.118875 21636 raft_consensus.cc:2272] T c4123a7d6ed643d2a7213ca3f34e7b1d P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.121404 21636 tablet_replica.cc:333] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.122043 21636 raft_consensus.cc:2243] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.122977 21636 raft_consensus.cc:2272] T e1c65f83f85d44d1b7525a643c8fe3e8 P e91ba25a6a344f70aef0384659b3fd73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.125478 21636 tablet_replica.cc:333] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.126257 21636 raft_consensus.cc:2243] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.127177 21636 raft_consensus.cc:2272] T 484797364d6049a68a06966fb4660242 P e91ba25a6a344f70aef0384659b3fd73 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.131134 21636 tablet_replica.cc:333] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73: stopping tablet replica
I20260709 13:17:36.131788 21636 raft_consensus.cc:2243] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.132941 21636 raft_consensus.cc:2272] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P e91ba25a6a344f70aef0384659b3fd73 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.188647 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:17:36.237015 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
I20260709 13:17:36.276501 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:36.277382 21636 tablet_replica.cc:333] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.278581 21636 raft_consensus.cc:2243] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.279476 21636 raft_consensus.cc:2272] T 1888a2cd9a9245ef8ae30cebf3fc1401 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.282645 21636 tablet_replica.cc:333] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.283607 21636 raft_consensus.cc:2243] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.284507 21636 raft_consensus.cc:2272] T 3908698cf2654827abcfe6ae5933096c P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.288031 21636 tablet_replica.cc:333] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.289008 21636 raft_consensus.cc:2243] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.289783 21636 raft_consensus.cc:2272] T f63162ba797b434ca275ef7b6299520f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.293831 21636 tablet_replica.cc:333] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.295010 21636 raft_consensus.cc:2243] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.295878 21636 raft_consensus.cc:2272] T 361936d22bf24421b2af4d6c0510298f P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.299644 21636 tablet_replica.cc:333] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.300558 21636 raft_consensus.cc:2243] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.301355 21636 raft_consensus.cc:2272] T ccd0925475c14c1abdcc732703f9fbd6 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.304806 21636 tablet_replica.cc:333] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.305665 21636 raft_consensus.cc:2243] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.306874 21636 raft_consensus.cc:2272] T 3dce46c69af941cc81ab6459e9e3424f P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.310784 21636 tablet_replica.cc:333] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.311725 21636 raft_consensus.cc:2243] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.312436 21636 raft_consensus.cc:2272] T ff66ac876d0546a19d3f5002d1f77848 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.315419 21636 tablet_replica.cc:333] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.316025 21636 raft_consensus.cc:2243] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.317034 21636 raft_consensus.cc:2272] T 011ac8a199d24d52a117549237f0d442 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.319913 21636 tablet_replica.cc:333] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.320561 21636 raft_consensus.cc:2243] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.321044 21636 raft_consensus.cc:2272] T 0fbb6f2fcba04b6b9f78d2db58f523a8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.323844 21636 tablet_replica.cc:333] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.324440 21636 raft_consensus.cc:2243] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.324918 21636 raft_consensus.cc:2272] T b4bb4416aa6442e09813b3cf9e8237f8 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.327581 21636 tablet_replica.cc:333] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.328171 21636 raft_consensus.cc:2243] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.329455 21636 raft_consensus.cc:2272] T c4123a7d6ed643d2a7213ca3f34e7b1d P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.332122 21636 tablet_replica.cc:333] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.332679 21636 raft_consensus.cc:2243] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.333107 21636 raft_consensus.cc:2272] T e1c65f83f85d44d1b7525a643c8fe3e8 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.335635 21636 tablet_replica.cc:333] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.336231 21636 raft_consensus.cc:2243] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.337242 21636 raft_consensus.cc:2272] T 484797364d6049a68a06966fb4660242 P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.339651 21636 tablet_replica.cc:333] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.340325 21636 raft_consensus.cc:2243] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.340781 21636 raft_consensus.cc:2272] T 5d190db5d02f418d80ee52a83ea60119 P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.343690 21636 tablet_replica.cc:333] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.344259 21636 raft_consensus.cc:2243] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.344746 21636 raft_consensus.cc:2272] T 7563b7d4737943d6987c8a3451d85a75 P f9e8deca048d4039851f55fe03299952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.347486 21636 tablet_replica.cc:333] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.348029 21636 raft_consensus.cc:2243] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.348867 21636 raft_consensus.cc:2272] T dbce222191f9408397faebf06a41ba57 P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.352200 21636 tablet_replica.cc:333] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.353008 21636 raft_consensus.cc:2243] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.353626 21636 raft_consensus.cc:2272] T 025ec36b5ae54e11a43c4f78d870c2fc P f9e8deca048d4039851f55fe03299952 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.356482 21636 tablet_replica.cc:333] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952: stopping tablet replica
I20260709 13:17:36.357115 21636 raft_consensus.cc:2243] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.358166 21636 raft_consensus.cc:2272] T 372e2af2548a46d39bb7577e4d7e56ff P f9e8deca048d4039851f55fe03299952 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.407449 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:17:36.451646 21636 master.cc:562] Master@127.21.33.62:36851 shutting down...
I20260709 13:17:36.478343 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.479193 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.479598 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4f686b4594bf4d02a2983c96c17af8be: stopping tablet replica
I20260709 13:17:36.504068 21636 master.cc:584] Master@127.21.33.62:36851 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (15147 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:17:36.571252 21636 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.33.62:39217
I20260709 13:17:36.572691 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:36.580375  2288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.581040  2287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.582116 21636 server_base.cc:1061] running on GCE node
W20260709 13:17:36.582880  2290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.583839 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.584041 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.584281 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603056584246 us; error 0 us; skew 500 ppm
I20260709 13:17:36.584838 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.588230 21636 webserver.cc:533] Webserver started at http://127.21.33.62:40823/ using document root <none> and password file <none>
I20260709 13:17:36.589108 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.589416 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.589936 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.591413 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/master-0-root/instance:
uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-p8rg"
I20260709 13:17:36.596839 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:36.601361  2295 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.602775 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:36.603345 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/master-0-root
uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-p8rg"
I20260709 13:17:36.603722 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.620278 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.621769 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:36.671769 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.62:39217
I20260709 13:17:36.672015  2346 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.62:39217 every 8 connection(s)
I20260709 13:17:36.677209  2347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:36.692564  2347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca: Bootstrap starting.
I20260709 13:17:36.698863  2347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:36.704094  2347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca: No bootstrap required, opened a new log
I20260709 13:17:36.707218  2347 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" member_type: VOTER }
I20260709 13:17:36.707738  2347 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:36.708107  2347 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc5307b9ebb64c8d9aa0ddd0381fa2ca, State: Initialized, Role: FOLLOWER
I20260709 13:17:36.708859  2347 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" member_type: VOTER }
I20260709 13:17:36.709430  2347 raft_consensus.cc:399] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:36.709698  2347 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:36.710044  2347 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:36.716483  2347 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" member_type: VOTER }
I20260709 13:17:36.717214  2347 leader_election.cc:304] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc5307b9ebb64c8d9aa0ddd0381fa2ca; no voters: 
I20260709 13:17:36.718669  2347 leader_election.cc:290] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:36.719231  2350 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:36.720961  2350 raft_consensus.cc:697] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 1 LEADER]: Becoming Leader. State: Replica: fc5307b9ebb64c8d9aa0ddd0381fa2ca, State: Running, Role: LEADER
I20260709 13:17:36.721797  2350 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" member_type: VOTER }
I20260709 13:17:36.722532  2347 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:36.727113  2352 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc5307b9ebb64c8d9aa0ddd0381fa2ca. Latest consensus state: current_term: 1 leader_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" member_type: VOTER } }
I20260709 13:17:36.727114  2351 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5307b9ebb64c8d9aa0ddd0381fa2ca" member_type: VOTER } }
I20260709 13:17:36.728070  2352 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:36.728462  2351 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:36.731952  2355 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:36.742008  2355 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:36.745426 21636 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:36.756322  2355 catalog_manager.cc:1383] Generated new cluster ID: a7a1b43fe20b4d168a7cdd87e7ff5431
I20260709 13:17:36.756690  2355 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:36.772258  2355 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:36.773994  2355 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:36.798483  2355 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca: Generated new TSK 0
I20260709 13:17:36.799556  2355 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:36.813763 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:36.822247  2369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.821226  2368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.823766  2371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.824692 21636 server_base.cc:1061] running on GCE node
I20260709 13:17:36.825795 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.825980 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.826177 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603056826158 us; error 0 us; skew 500 ppm
I20260709 13:17:36.826809 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.830005 21636 webserver.cc:533] Webserver started at http://127.21.33.1:38221/ using document root <none> and password file <none>
I20260709 13:17:36.830595 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.830904 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.831337 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.832644 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-0-root/instance:
uuid: "641d982ff4c746e3bf32bbee02e44712"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-p8rg"
I20260709 13:17:36.837610 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:17:36.841216  2376 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.842077 21636 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:17:36.842388 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-0-root
uuid: "641d982ff4c746e3bf32bbee02e44712"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-p8rg"
I20260709 13:17:36.842706 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.856920 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.858402 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:36.860239 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:36.862735 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:36.862957 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.863204 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:36.863394 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.916116 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.1:44755
I20260709 13:17:36.916246  2438 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.1:44755 every 8 connection(s)
I20260709 13:17:36.921825 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:36.931248  2443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.932581  2444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.938427 21636 server_base.cc:1061] running on GCE node
W20260709 13:17:36.939913  2446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.941404 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.941828 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.942122 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603056942095 us; error 0 us; skew 500 ppm
I20260709 13:17:36.942765 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.945272  2439 heartbeater.cc:344] Connected to a master server at 127.21.33.62:39217
I20260709 13:17:36.945741  2439 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:36.946331 21636 webserver.cc:533] Webserver started at http://127.21.33.2:39635/ using document root <none> and password file <none>
I20260709 13:17:36.946568  2439 heartbeater.cc:507] Master 127.21.33.62:39217 requested a full tablet report, sending...
I20260709 13:17:36.947072 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.947391 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.947755 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.949095  2312 ts_manager.cc:194] Registered new tserver with Master: 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755)
I20260709 13:17:36.949273 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-1-root/instance:
uuid: "fd75afe960fe4640b52ac3328158343e"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-p8rg"
I20260709 13:17:36.951833  2312 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45940
I20260709 13:17:36.956105 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:36.961305  2451 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.962447 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:17:36.962854 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-1-root
uuid: "fd75afe960fe4640b52ac3328158343e"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-p8rg"
I20260709 13:17:36.963217 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.979836 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.981451 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:36.983589 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:36.986593 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:36.987082 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:17:36.987425 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:36.987620 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.043817 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.2:39539
I20260709 13:17:37.044015  2513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.2:39539 every 8 connection(s)
I20260709 13:17:37.049683 21636 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:37.059533  2517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:37.063575  2518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.067900  2514 heartbeater.cc:344] Connected to a master server at 127.21.33.62:39217
I20260709 13:17:37.068567  2514 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.069007 21636 server_base.cc:1061] running on GCE node
W20260709 13:17:37.069425  2520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.069793  2514 heartbeater.cc:507] Master 127.21.33.62:39217 requested a full tablet report, sending...
I20260709 13:17:37.070713 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:37.071026 21636 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:37.071249 21636 hybrid_clock.cc:648] HybridClock initialized: now 1783603057071230 us; error 0 us; skew 500 ppm
I20260709 13:17:37.072032 21636 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:37.072225  2312 ts_manager.cc:194] Registered new tserver with Master: fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:37.074157  2312 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45946
I20260709 13:17:37.075064 21636 webserver.cc:533] Webserver started at http://127.21.33.3:33069/ using document root <none> and password file <none>
I20260709 13:17:37.075847 21636 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:37.076200 21636 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:37.076552 21636 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:37.078054 21636 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-2-root/instance:
uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-p8rg"
I20260709 13:17:37.083765 21636 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:17:37.087646  2525 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.088694 21636 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:17:37.089051 21636 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-2-root
uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-p8rg"
I20260709 13:17:37.089370 21636 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskGb300l/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602629827198-21636-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:37.118865 21636 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:37.120617 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.122485 21636 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:37.125471 21636 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:37.125787 21636 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.126046 21636 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:37.126240 21636 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.183598 21636 rpc_server.cc:307] RPC server started. Bound to: 127.21.33.3:39227
I20260709 13:17:37.183744  2587 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.33.3:39227 every 8 connection(s)
I20260709 13:17:37.200716  2588 heartbeater.cc:344] Connected to a master server at 127.21.33.62:39217
I20260709 13:17:37.201261  2588 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.202181  2588 heartbeater.cc:507] Master 127.21.33.62:39217 requested a full tablet report, sending...
I20260709 13:17:37.204548  2312 ts_manager.cc:194] Registered new tserver with Master: 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:37.205403 21636 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018223136s
I20260709 13:17:37.206722  2312 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45950
I20260709 13:17:37.236752  2312 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45956:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:37.240267  2312 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:37.341392  2404 tablet_service.cc:1511] Processing CreateTablet for tablet 0b628f2eeb3c481fa33dc3951c686819 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:37.342267  2402 tablet_service.cc:1511] Processing CreateTablet for tablet 0d49b9acef904e9a9c0243c2880cedd4 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:37.343286  2404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b628f2eeb3c481fa33dc3951c686819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.345083  2403 tablet_service.cc:1511] Processing CreateTablet for tablet 5698faf31e5444d49b74dd662444cd26 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:37.346766  2403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5698faf31e5444d49b74dd662444cd26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.352313  2402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d49b9acef904e9a9c0243c2880cedd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.360353  2479 tablet_service.cc:1511] Processing CreateTablet for tablet 0b628f2eeb3c481fa33dc3951c686819 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:37.362275  2479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b628f2eeb3c481fa33dc3951c686819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.367295  2401 tablet_service.cc:1511] Processing CreateTablet for tablet ca726d8f91c54a5591b80fdeed720a42 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:37.370996  2478 tablet_service.cc:1511] Processing CreateTablet for tablet 5698faf31e5444d49b74dd662444cd26 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:37.372699  2478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5698faf31e5444d49b74dd662444cd26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.373304  2476 tablet_service.cc:1511] Processing CreateTablet for tablet ca726d8f91c54a5591b80fdeed720a42 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:37.375295  2476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca726d8f91c54a5591b80fdeed720a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.375104  2477 tablet_service.cc:1511] Processing CreateTablet for tablet 0d49b9acef904e9a9c0243c2880cedd4 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:37.376569  2477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d49b9acef904e9a9c0243c2880cedd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.369356  2401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca726d8f91c54a5591b80fdeed720a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.384199  2400 tablet_service.cc:1511] Processing CreateTablet for tablet a22a754bb6fd4bf5be2806cee04c1fef (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:37.389483  2475 tablet_service.cc:1511] Processing CreateTablet for tablet a22a754bb6fd4bf5be2806cee04c1fef (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:37.391637  2475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22a754bb6fd4bf5be2806cee04c1fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.395892  2400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22a754bb6fd4bf5be2806cee04c1fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.400665  2399 tablet_service.cc:1511] Processing CreateTablet for tablet 9c51c9dd7adc4c1d91f6820000bd1b19 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:37.402572  2399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c51c9dd7adc4c1d91f6820000bd1b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.408257  2474 tablet_service.cc:1511] Processing CreateTablet for tablet 9c51c9dd7adc4c1d91f6820000bd1b19 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:37.417754  2474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c51c9dd7adc4c1d91f6820000bd1b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.422957  2398 tablet_service.cc:1511] Processing CreateTablet for tablet 7b97664c7c30434098b33c6307e2a071 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:37.424688  2398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b97664c7c30434098b33c6307e2a071. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.425558  2473 tablet_service.cc:1511] Processing CreateTablet for tablet 7b97664c7c30434098b33c6307e2a071 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:37.427676  2473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b97664c7c30434098b33c6307e2a071. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.432765  2397 tablet_service.cc:1511] Processing CreateTablet for tablet 97fbbb8b200748dca15c416922ccc0f8 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:37.433820  2472 tablet_service.cc:1511] Processing CreateTablet for tablet 97fbbb8b200748dca15c416922ccc0f8 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:37.435144  2397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97fbbb8b200748dca15c416922ccc0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.435595  2472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97fbbb8b200748dca15c416922ccc0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.446709  2396 tablet_service.cc:1511] Processing CreateTablet for tablet 88ed6a6dd1f1495abc4d09ca92e0f4ab (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:37.448458  2396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ed6a6dd1f1495abc4d09ca92e0f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.448680  2471 tablet_service.cc:1511] Processing CreateTablet for tablet 88ed6a6dd1f1495abc4d09ca92e0f4ab (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:37.450620  2471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ed6a6dd1f1495abc4d09ca92e0f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.455127  2553 tablet_service.cc:1511] Processing CreateTablet for tablet 0b628f2eeb3c481fa33dc3951c686819 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:37.457048  2553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b628f2eeb3c481fa33dc3951c686819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.463699  2552 tablet_service.cc:1511] Processing CreateTablet for tablet 5698faf31e5444d49b74dd662444cd26 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:37.465754  2552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5698faf31e5444d49b74dd662444cd26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.486974  2604 tablet_bootstrap.cc:492] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:37.492267  2551 tablet_service.cc:1511] Processing CreateTablet for tablet 0d49b9acef904e9a9c0243c2880cedd4 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:37.503355  2551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d49b9acef904e9a9c0243c2880cedd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.495451  2550 tablet_service.cc:1511] Processing CreateTablet for tablet ca726d8f91c54a5591b80fdeed720a42 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:37.499236  2548 tablet_service.cc:1511] Processing CreateTablet for tablet 9c51c9dd7adc4c1d91f6820000bd1b19 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:37.501170  2549 tablet_service.cc:1511] Processing CreateTablet for tablet a22a754bb6fd4bf5be2806cee04c1fef (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:37.508412  2548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c51c9dd7adc4c1d91f6820000bd1b19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.510596  2549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22a754bb6fd4bf5be2806cee04c1fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.512423  2550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca726d8f91c54a5591b80fdeed720a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.548034  2603 tablet_bootstrap.cc:492] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:37.549841  2604 tablet_bootstrap.cc:654] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.581218  2547 tablet_service.cc:1511] Processing CreateTablet for tablet 7b97664c7c30434098b33c6307e2a071 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:37.581564  2546 tablet_service.cc:1511] Processing CreateTablet for tablet 97fbbb8b200748dca15c416922ccc0f8 (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:37.583083  2547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b97664c7c30434098b33c6307e2a071. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.583868  2546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97fbbb8b200748dca15c416922ccc0f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.598884  2604 tablet_bootstrap.cc:492] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:37.599614  2604 ts_tablet_manager.cc:1403] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.113s	user 0.016s	sys 0.022s
I20260709 13:17:37.600054  2603 tablet_bootstrap.cc:654] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.604935  2604 raft_consensus.cc:359] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.607400  2604 raft_consensus.cc:385] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.607894  2604 raft_consensus.cc:740] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.610041  2604 consensus_queue.cc:260] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.612540  2603 tablet_bootstrap.cc:492] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:37.611044  2606 tablet_bootstrap.cc:492] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:37.613245  2603 ts_tablet_manager.cc:1403] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.007s
I20260709 13:17:37.616726  2603 raft_consensus.cc:359] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.618054  2603 raft_consensus.cc:385] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.618412  2603 raft_consensus.cc:740] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.619555  2603 consensus_queue.cc:260] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.621234  2606 tablet_bootstrap.cc:654] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.622202  2604 ts_tablet_manager.cc:1434] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.022s	user 0.012s	sys 0.002s
I20260709 13:17:37.623270  2439 heartbeater.cc:499] Master 127.21.33.62:39217 was elected leader, sending a full tablet report...
I20260709 13:17:37.623859  2604 tablet_bootstrap.cc:492] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:37.626590  2547 tablet_service.cc:1511] Processing CreateTablet for tablet 88ed6a6dd1f1495abc4d09ca92e0f4ab (DEFAULT_TABLE table=test-workload [id=bb9159f58b4c4348840226278ed16fa8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:37.628265  2547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ed6a6dd1f1495abc4d09ca92e0f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.650852  2514 heartbeater.cc:499] Master 127.21.33.62:39217 was elected leader, sending a full tablet report...
I20260709 13:17:37.658638  2603 ts_tablet_manager.cc:1434] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.045s	user 0.015s	sys 0.001s
I20260709 13:17:37.659993  2603 tablet_bootstrap.cc:492] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
W20260709 13:17:37.671059  2440 tablet.cc:2406] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:37.684581  2603 tablet_bootstrap.cc:654] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.684581  2604 tablet_bootstrap.cc:654] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.715935  2606 tablet_bootstrap.cc:492] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:37.716610  2606 ts_tablet_manager.cc:1403] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.106s	user 0.012s	sys 0.013s
I20260709 13:17:37.719524  2604 tablet_bootstrap.cc:492] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:37.720285  2604 ts_tablet_manager.cc:1403] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.097s	user 0.008s	sys 0.014s
I20260709 13:17:37.720201  2606 raft_consensus.cc:359] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.721146  2606 raft_consensus.cc:385] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.721484  2606 raft_consensus.cc:740] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.722358  2606 consensus_queue.cc:260] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.723596  2604 raft_consensus.cc:359] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.724445  2604 raft_consensus.cc:385] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.724747  2604 raft_consensus.cc:740] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.724794  2588 heartbeater.cc:499] Master 127.21.33.62:39217 was elected leader, sending a full tablet report...
I20260709 13:17:37.725293  2606 ts_tablet_manager.cc:1434] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.005s
I20260709 13:17:37.726429  2606 tablet_bootstrap.cc:492] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:37.725685  2604 consensus_queue.cc:260] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.728755  2603 tablet_bootstrap.cc:492] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:37.729393  2603 ts_tablet_manager.cc:1403] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.018s
I20260709 13:17:37.731042  2604 ts_tablet_manager.cc:1434] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:17:37.732182  2604 tablet_bootstrap.cc:492] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:37.733008  2603 raft_consensus.cc:359] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.733819  2606 tablet_bootstrap.cc:654] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.733992  2603 raft_consensus.cc:385] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.734365  2603 raft_consensus.cc:740] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.735257  2603 consensus_queue.cc:260] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
W20260709 13:17:37.738443  2360 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:37.741117  2603 ts_tablet_manager.cc:1434] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:17:37.742996  2603 tablet_bootstrap.cc:492] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:37.748100  2604 tablet_bootstrap.cc:654] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.752782  2606 tablet_bootstrap.cc:492] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:37.757680  2606 ts_tablet_manager.cc:1403] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.004s
I20260709 13:17:37.761770  2606 raft_consensus.cc:359] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.762310  2603 tablet_bootstrap.cc:654] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.762748  2606 raft_consensus.cc:385] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.763141  2606 raft_consensus.cc:740] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.764536  2604 tablet_bootstrap.cc:492] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:37.764055  2606 consensus_queue.cc:260] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.765349  2604 ts_tablet_manager.cc:1403] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.003s
I20260709 13:17:37.770171  2604 raft_consensus.cc:359] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.771595  2604 raft_consensus.cc:385] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.772058  2604 raft_consensus.cc:740] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.773244  2604 consensus_queue.cc:260] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.775743  2606 ts_tablet_manager.cc:1434] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.005s
I20260709 13:17:37.776659  2604 ts_tablet_manager.cc:1434] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:17:37.776973  2606 tablet_bootstrap.cc:492] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:37.778343  2604 tablet_bootstrap.cc:492] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:37.786074  2606 tablet_bootstrap.cc:654] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.789062  2604 tablet_bootstrap.cc:654] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.805756  2603 tablet_bootstrap.cc:492] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:37.806582  2603 ts_tablet_manager.cc:1403] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.026s
I20260709 13:17:37.811501  2603 raft_consensus.cc:359] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.812613  2603 raft_consensus.cc:385] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.813469  2603 raft_consensus.cc:740] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.815611  2603 consensus_queue.cc:260] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.818955  2603 ts_tablet_manager.cc:1434] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 13:17:37.820497  2603 tablet_bootstrap.cc:492] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:37.821954  2608 raft_consensus.cc:493] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.822685  2608 raft_consensus.cc:515] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.831791  2603 tablet_bootstrap.cc:654] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.832361  2608 leader_election.cc:290] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:37.839963  2604 tablet_bootstrap.cc:492] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:37.842280  2606 tablet_bootstrap.cc:492] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:37.845008  2604 ts_tablet_manager.cc:1403] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.067s	user 0.023s	sys 0.010s
I20260709 13:17:37.845288  2606 ts_tablet_manager.cc:1403] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.033s
I20260709 13:17:37.849977  2604 raft_consensus.cc:359] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.851122  2604 raft_consensus.cc:385] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.851640  2604 raft_consensus.cc:740] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.852662  2604 consensus_queue.cc:260] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.850507  2606 raft_consensus.cc:359] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.854038  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca726d8f91c54a5591b80fdeed720a42" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:37.854317  2603 tablet_bootstrap.cc:492] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:37.854446  2606 raft_consensus.cc:385] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.855213  2606 raft_consensus.cc:740] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.855413  2603 ts_tablet_manager.cc:1403] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.004s
I20260709 13:17:37.856570  2604 ts_tablet_manager.cc:1434] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:17:37.856386  2606 consensus_queue.cc:260] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.858304  2604 tablet_bootstrap.cc:492] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
W20260709 13:17:37.863628  2379 leader_election.cc:343] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:37.857481  2613 raft_consensus.cc:493] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.864635  2613 raft_consensus.cc:515] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.870954  2604 tablet_bootstrap.cc:654] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.872381  2613 leader_election.cc:290] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:37.860055  2606 ts_tablet_manager.cc:1434] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:17:37.875773  2606 tablet_bootstrap.cc:492] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:37.859272  2603 raft_consensus.cc:359] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.877334  2603 raft_consensus.cc:385] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.877728  2603 raft_consensus.cc:740] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.878038  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca726d8f91c54a5591b80fdeed720a42" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:37.878823  2563 raft_consensus.cc:2468] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:37.878786  2603 consensus_queue.cc:260] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.882519  2603 ts_tablet_manager.cc:1434] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 13:17:37.883838  2603 tablet_bootstrap.cc:492] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:37.883710  2377 leader_election.cc:304] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:37.884946  2608 raft_consensus.cc:2804] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:37.885358  2608 raft_consensus.cc:493] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.885741  2608 raft_consensus.cc:3060] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.886896  2606 tablet_bootstrap.cc:654] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.892355  2603 tablet_bootstrap.cc:654] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.903867  2608 raft_consensus.cc:515] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.905862  2603 tablet_bootstrap.cc:492] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:37.906564  2603 ts_tablet_manager.cc:1403] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 13:17:37.910020  2603 raft_consensus.cc:359] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.910835  2603 raft_consensus.cc:385] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.911136  2603 raft_consensus.cc:740] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.912290  2603 consensus_queue.cc:260] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.917367  2609 raft_consensus.cc:493] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.920053  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca726d8f91c54a5591b80fdeed720a42" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
W20260709 13:17:37.921880  2379 leader_election.cc:343] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:37.922495  2604 tablet_bootstrap.cc:492] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:37.923166  2604 ts_tablet_manager.cc:1403] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.010s
I20260709 13:17:37.923538  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca726d8f91c54a5591b80fdeed720a42" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:37.924276  2563 raft_consensus.cc:3060] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.924443  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca726d8f91c54a5591b80fdeed720a42" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:37.918962  2609 raft_consensus.cc:515] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
W20260709 13:17:37.926673  2528 leader_election.cc:343] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:37.927389  2606 tablet_bootstrap.cc:492] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:37.928032  2606 ts_tablet_manager.cc:1403] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.010s
I20260709 13:17:37.930827  2608 leader_election.cc:290] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:37.933800  2609 leader_election.cc:290] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:37.944818  2604 raft_consensus.cc:359] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.945124  2606 raft_consensus.cc:359] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.945871  2604 raft_consensus.cc:385] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.945909  2606 raft_consensus.cc:385] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.946352  2604 raft_consensus.cc:740] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.946393  2606 raft_consensus.cc:740] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.947414  2604 consensus_queue.cc:260] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.949779  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca726d8f91c54a5591b80fdeed720a42" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:37.950830  2604 ts_tablet_manager.cc:1434] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 13:17:37.951287  2414 raft_consensus.cc:2393] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17ac24c81e5d47f1a15abae5f11e5cce in current term 1: Already voted for candidate 641d982ff4c746e3bf32bbee02e44712 in this term.
I20260709 13:17:37.952534  2563 raft_consensus.cc:2468] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:37.952855  2604 tablet_bootstrap.cc:492] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:37.953878  2527 leader_election.cc:304] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:37.955075  2603 ts_tablet_manager.cc:1434] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:17:37.955559  2613 raft_consensus.cc:2749] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:37.947414  2606 consensus_queue.cc:260] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.956455  2603 tablet_bootstrap.cc:492] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:37.959131  2609 raft_consensus.cc:493] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.960011  2609 raft_consensus.cc:515] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.963196  2377 leader_election.cc:304] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:37.964264  2609 leader_election.cc:290] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:37.967643  2603 tablet_bootstrap.cc:654] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.967447  2606 ts_tablet_manager.cc:1434] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.001s
I20260709 13:17:37.969237  2606 tablet_bootstrap.cc:492] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:37.965045  2608 raft_consensus.cc:2804] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:37.979318  2604 tablet_bootstrap.cc:654] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.991792  2609 raft_consensus.cc:493] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.993064  2608 raft_consensus.cc:697] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:37.999179  2608 consensus_queue.cc:237] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:37.999538  2629 raft_consensus.cc:493] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.000427  2629 raft_consensus.cc:515] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.003906  2629 leader_election.cc:290] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:37.997596  2609 raft_consensus.cc:515] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.015352  2603 tablet_bootstrap.cc:492] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:38.017664  2603 ts_tablet_manager.cc:1403] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.009s
I20260709 13:17:38.019060  2609 leader_election.cc:290] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.020100  2604 tablet_bootstrap.cc:492] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:38.020893  2604 ts_tablet_manager.cc:1403] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.068s	user 0.015s	sys 0.016s
I20260709 13:17:38.021485  2633 raft_consensus.cc:493] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.022069  2603 raft_consensus.cc:359] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.025779  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.029505  2606 tablet_bootstrap.cc:654] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.030263  2414 raft_consensus.cc:2468] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:38.029994  2633 raft_consensus.cc:515] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.031284  2603 raft_consensus.cc:385] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.031689  2603 raft_consensus.cc:740] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.031517  2310 catalog_manager.cc:5697] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.024887  2604 raft_consensus.cc:359] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.026273  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.027657  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.028146  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c51c9dd7adc4c1d91f6820000bd1b19" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.033733  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.034417  2412 raft_consensus.cc:2468] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:38.035308  2604 raft_consensus.cc:385] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.035655  2604 raft_consensus.cc:740] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
W20260709 13:17:38.035992  2379 leader_election.cc:343] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.034461  2411 raft_consensus.cc:2468] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:38.036454  2604 consensus_queue.cc:260] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.032869  2603 consensus_queue.cc:260] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.040053  2603 ts_tablet_manager.cc:1434] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.022s	user 0.000s	sys 0.007s
I20260709 13:17:38.041263  2603 tablet_bootstrap.cc:492] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:38.041288  2604 ts_tablet_manager.cc:1434] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.005s
I20260709 13:17:38.043284  2453 leader_election.cc:304] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
W20260709 13:17:38.044539  2453 leader_election.cc:343] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.044963  2604 tablet_bootstrap.cc:492] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:38.045324  2453 leader_election.cc:304] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.045070  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.046411  2629 raft_consensus.cc:2804] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.046516  2453 leader_election.cc:304] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.074849  2629 raft_consensus.cc:493] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.075676  2629 raft_consensus.cc:3060] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.047326  2609 raft_consensus.cc:2804] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.076483  2609 raft_consensus.cc:493] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.076920  2609 raft_consensus.cc:3060] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.058629  2603 tablet_bootstrap.cc:654] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:38.048123  2377 leader_election.cc:343] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.081950  2377 leader_election.cc:304] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712; no voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.083969  2640 raft_consensus.cc:2804] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.084048  2608 raft_consensus.cc:2749] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.057318  2561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
W20260709 13:17:38.086987  2452 leader_election.cc:343] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.058530  2560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c51c9dd7adc4c1d91f6820000bd1b19" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.088259  2606 tablet_bootstrap.cc:492] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:38.088932  2560 raft_consensus.cc:2468] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:38.056543  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.065419  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.091751  2640 raft_consensus.cc:493] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.092592  2640 raft_consensus.cc:3060] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:38.093490  2452 leader_election.cc:343] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:38.096750  2452 leader_election.cc:343] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.097256  2452 leader_election.cc:304] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:38.089390  2606 ts_tablet_manager.cc:1403] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.120s	user 0.017s	sys 0.010s
I20260709 13:17:38.098454  2633 leader_election.cc:290] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.101819  2629 raft_consensus.cc:515] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.108162  2606 raft_consensus.cc:359] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.109107  2606 raft_consensus.cc:385] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.109457  2606 raft_consensus.cc:740] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.110976  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.112152  2643 raft_consensus.cc:2749] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.112488  2411 raft_consensus.cc:3060] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.109762  2640 raft_consensus.cc:515] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.115095  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.118085  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c51c9dd7adc4c1d91f6820000bd1b19" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.118821  2412 raft_consensus.cc:3060] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.112533  2629 leader_election.cc:290] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.111161  2606 consensus_queue.cc:260] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.123240  2609 raft_consensus.cc:515] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.124078  2606 ts_tablet_manager.cc:1434] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:17:38.119498  2640 leader_election.cc:290] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.126096  2604 tablet_bootstrap.cc:654] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.127552  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.127866  2606 tablet_bootstrap.cc:492] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:38.128330  2413 raft_consensus.cc:3060] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.131443  2609 leader_election.cc:290] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.137255  2413 raft_consensus.cc:2468] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:38.138811  2606 tablet_bootstrap.cc:654] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.138957  2453 leader_election.cc:304] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.140098  2609 raft_consensus.cc:2804] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:38.118722  2452 leader_election.cc:343] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.140810  2609 raft_consensus.cc:697] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.141808  2609 consensus_queue.cc:237] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.143004  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c51c9dd7adc4c1d91f6820000bd1b19" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.143473  2603 tablet_bootstrap.cc:492] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:38.143976  2562 raft_consensus.cc:3060] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.146052  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
W20260709 13:17:38.148911  2452 leader_election.cc:343] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.150254  2412 raft_consensus.cc:2468] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:38.151911  2453 leader_election.cc:304] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.153409  2640 raft_consensus.cc:2804] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.154071  2640 raft_consensus.cc:697] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.155318  2640 consensus_queue.cc:237] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.144078  2603 ts_tablet_manager.cc:1403] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.103s	user 0.030s	sys 0.003s
I20260709 13:17:38.162253  2603 raft_consensus.cc:359] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.163136  2603 raft_consensus.cc:385] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.163448  2603 raft_consensus.cc:740] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.163821  2562 raft_consensus.cc:2468] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:38.164361  2603 consensus_queue.cc:260] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.175309  2603 ts_tablet_manager.cc:1434] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:17:38.177923  2603 tablet_bootstrap.cc:492] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:38.175944  2606 tablet_bootstrap.cc:492] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:38.182401  2606 ts_tablet_manager.cc:1403] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.055s	user 0.029s	sys 0.019s
I20260709 13:17:38.184790  2411 raft_consensus.cc:2468] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:38.186390  2606 raft_consensus.cc:359] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.187320  2453 leader_election.cc:304] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:38.191318  2606 raft_consensus.cc:385] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.191870  2606 raft_consensus.cc:740] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.191862  2604 tablet_bootstrap.cc:492] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:38.192124  2640 raft_consensus.cc:2804] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.192880  2640 raft_consensus.cc:697] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.192915  2604 ts_tablet_manager.cc:1403] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.148s	user 0.025s	sys 0.015s
I20260709 13:17:38.193075  2606 consensus_queue.cc:260] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.193982  2640 consensus_queue.cc:237] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.195547  2606 ts_tablet_manager.cc:1434] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:17:38.196830  2606 tablet_bootstrap.cc:492] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:38.197664  2604 raft_consensus.cc:359] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.198592  2604 raft_consensus.cc:385] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.199278  2604 raft_consensus.cc:740] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.200700  2604 consensus_queue.cc:260] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.190713  2603 tablet_bootstrap.cc:654] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.204246  2604 ts_tablet_manager.cc:1434] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:38.206071  2606 tablet_bootstrap.cc:654] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.209911  2604 tablet_bootstrap.cc:492] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:38.222158  2604 tablet_bootstrap.cc:654] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.229478  2640 raft_consensus.cc:493] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.230438  2640 raft_consensus.cc:515] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.231997  2633 raft_consensus.cc:493] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.232739  2633 raft_consensus.cc:515] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.234536  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.235033  2613 raft_consensus.cc:493] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.235816  2613 raft_consensus.cc:515] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
W20260709 13:17:38.236331  2453 leader_election.cc:343] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.236671  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b628f2eeb3c481fa33dc3951c686819" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.237107  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b628f2eeb3c481fa33dc3951c686819" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.237450  2562 raft_consensus.cc:2468] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:38.237708  2640 leader_election.cc:290] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.239148  2377 leader_election.cc:304] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:38.239471  2633 leader_election.cc:290] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.240186  2608 raft_consensus.cc:2804] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.240336  2613 leader_election.cc:290] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.240813  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.240715  2608 raft_consensus.cc:493] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.241842  2608 raft_consensus.cc:3060] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.241340  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.242395  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
W20260709 13:17:38.241022  2379 leader_election.cc:343] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.243283  2411 raft_consensus.cc:2393] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17ac24c81e5d47f1a15abae5f11e5cce in current term 1: Already voted for candidate fd75afe960fe4640b52ac3328158343e in this term.
I20260709 13:17:38.244840  2527 leader_election.cc:304] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.242650  2562 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:38.245162  2311 catalog_manager.cc:5697] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 0 to 1, leader changed from <none> to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.246205  2613 raft_consensus.cc:3060] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.246770  2452 leader_election.cc:304] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:38.247938  2640 raft_consensus.cc:2804] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.248342  2640 raft_consensus.cc:493] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.248766  2640 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.252561  2311 catalog_manager.cc:5697] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 0 to 1, leader changed from <none> to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.254043  2608 raft_consensus.cc:515] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.258553  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b628f2eeb3c481fa33dc3951c686819" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.259214  2562 raft_consensus.cc:3060] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.259757  2613 raft_consensus.cc:2749] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.264477  2604 tablet_bootstrap.cc:492] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:38.265516  2604 ts_tablet_manager.cc:1403] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.060s	user 0.020s	sys 0.007s
I20260709 13:17:38.268381  2562 raft_consensus.cc:2468] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:38.270308  2377 leader_election.cc:304] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:38.270485  2604 raft_consensus.cc:359] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.271373  2604 raft_consensus.cc:385] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.271441  2633 raft_consensus.cc:2804] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.271762  2604 raft_consensus.cc:740] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.272229  2633 raft_consensus.cc:697] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:38.264936  2640 raft_consensus.cc:515] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.272589  2604 consensus_queue.cc:260] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.273178  2633 consensus_queue.cc:237] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.274921  2604 ts_tablet_manager.cc:1434] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:17:38.276008  2604 tablet_bootstrap.cc:492] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:38.279337  2603 tablet_bootstrap.cc:492] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:38.281705  2603 ts_tablet_manager.cc:1403] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.104s	user 0.025s	sys 0.036s
I20260709 13:17:38.282275  2604 tablet_bootstrap.cc:654] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.284874  2603 raft_consensus.cc:359] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.286137  2603 raft_consensus.cc:385] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.286504  2603 raft_consensus.cc:740] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.289080  2608 leader_election.cc:290] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.288959  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b628f2eeb3c481fa33dc3951c686819" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:38.287735  2603 consensus_queue.cc:260] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:38.291446  2489 raft_consensus.cc:3060] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.291397  2310 catalog_manager.cc:5697] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 0 to 1, leader changed from <none> to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.293296  2603 ts_tablet_manager.cc:1434] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:17:38.294611  2603 tablet_bootstrap.cc:492] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:38.296532  2606 tablet_bootstrap.cc:492] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:38.296831  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.297655  2411 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.297794  2606 ts_tablet_manager.cc:1403] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.101s	user 0.023s	sys 0.025s
I20260709 13:17:38.300248  2613 raft_consensus.cc:493] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.300149  2640 leader_election.cc:290] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.300943  2613 raft_consensus.cc:515] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.302870  2606 raft_consensus.cc:359] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.303452  2613 leader_election.cc:290] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.303890  2606 raft_consensus.cc:385] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.304271  2606 raft_consensus.cc:740] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.305155  2606 consensus_queue.cc:260] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.307448  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.308092  2562 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.309960  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.310812  2488 raft_consensus.cc:2468] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:38.307967  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.313544  2528 leader_election.cc:304] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.314095  2412 raft_consensus.cc:2468] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:38.314589  2616 raft_consensus.cc:2804] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.315382  2616 raft_consensus.cc:493] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.315447  2489 raft_consensus.cc:2468] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:38.315798  2616 raft_consensus.cc:3060] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.319424  2606 ts_tablet_manager.cc:1434] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:17:38.323467  2606 tablet_bootstrap.cc:492] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:38.326398  2616 raft_consensus.cc:515] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.330099  2604 tablet_bootstrap.cc:492] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:38.330729  2604 ts_tablet_manager.cc:1403] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.000s
I20260709 13:17:38.331961  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.332765  2412 raft_consensus.cc:3060] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.333384  2311 catalog_manager.cc:5697] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.336714  2606 tablet_bootstrap.cc:654] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.339232  2604 raft_consensus.cc:359] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.340032  2604 raft_consensus.cc:385] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.340320  2604 raft_consensus.cc:740] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.343048  2603 tablet_bootstrap.cc:654] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.344328  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:38.345197  2489 raft_consensus.cc:3060] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.341045  2604 consensus_queue.cc:260] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.348268  2604 ts_tablet_manager.cc:1434] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:17:38.348630  2616 leader_election.cc:290] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.327919  2411 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:38.326511  2562 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:38.349715  2606 tablet_bootstrap.cc:492] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:38.350421  2606 ts_tablet_manager.cc:1403] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.003s
I20260709 13:17:38.351815  2452 leader_election.cc:304] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.353605  2640 raft_consensus.cc:2804] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.354362  2640 raft_consensus.cc:697] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.355413  2640 consensus_queue.cc:237] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.355685  2606 raft_consensus.cc:359] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.356585  2603 tablet_bootstrap.cc:492] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:38.356699  2606 raft_consensus.cc:385] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.357009  2412 raft_consensus.cc:2468] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:38.357234  2603 ts_tablet_manager.cc:1403] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.063s	user 0.009s	sys 0.003s
I20260709 13:17:38.357286  2606 raft_consensus.cc:740] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.358759  2606 consensus_queue.cc:260] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.360877  2603 raft_consensus.cc:359] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.361546  2606 ts_tablet_manager.cc:1434] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:38.361752  2603 raft_consensus.cc:385] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.362133  2603 raft_consensus.cc:740] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.363005  2603 consensus_queue.cc:260] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.365476  2603 ts_tablet_manager.cc:1434] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:17:38.367733  2527 leader_election.cc:304] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:38.368045  2606 tablet_bootstrap.cc:492] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:38.368733  2616 raft_consensus.cc:2804] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.369555  2616 raft_consensus.cc:697] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:38.370364  2616 consensus_queue.cc:237] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.374976  2633 raft_consensus.cc:493] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.375939  2489 raft_consensus.cc:2468] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:38.375934  2633 raft_consensus.cc:515] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.381330  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.381701  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:38.382576  2489 raft_consensus.cc:2468] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:38.384155  2379 leader_election.cc:304] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
W20260709 13:17:38.385046  2377 leader_election.cc:343] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.381615  2633 leader_election.cc:290] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.385171  2608 raft_consensus.cc:2804] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.385977  2608 raft_consensus.cc:493] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.386591  2608 raft_consensus.cc:3060] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.388962  2613 raft_consensus.cc:493] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.390319  2613 raft_consensus.cc:515] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.395622  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.396736  2412 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:38.398886  2527 leader_election.cc:304] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:38.402513  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.403647  2489 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:38.404762  2311 catalog_manager.cc:5697] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 0 to 1, leader changed from <none> to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.406539  2608 raft_consensus.cc:515] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.407789  2613 leader_election.cc:290] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.403975  2606 tablet_bootstrap.cc:654] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:38.408669  2613 raft_consensus.cc:2804] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:38.409152  2613 raft_consensus.cc:493] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.409471  2613 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.414367  2633 consensus_queue.cc:1048] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.416561  2608 leader_election.cc:290] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.419695  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
W20260709 13:17:38.421382  2377 leader_election.cc:343] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:38.422477  2309 catalog_manager.cc:5697] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 1 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.424552  2606 tablet_bootstrap.cc:492] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:38.422472  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:38.425220  2606 ts_tablet_manager.cc:1403] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.057s	user 0.007s	sys 0.008s
I20260709 13:17:38.425697  2489 raft_consensus.cc:3060] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.429072  2606 raft_consensus.cc:359] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.430150  2606 raft_consensus.cc:385] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:38.430538  2606 raft_consensus.cc:740] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:38.431600  2606 consensus_queue.cc:260] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.433390  2489 raft_consensus.cc:2468] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:38.434602  2606 ts_tablet_manager.cc:1434] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:17:38.435423  2379 leader_election.cc:304] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:38.437199  2608 raft_consensus.cc:2804] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.437728  2608 raft_consensus.cc:697] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:38.438737  2608 consensus_queue.cc:237] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.442325  2613 raft_consensus.cc:515] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.445155  2613 leader_election.cc:290] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.447237  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.447572  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:38.448120  2412 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.448225  2489 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.456061  2488 raft_consensus.cc:3060] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.458117  2309 catalog_manager.cc:5697] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.464563  2489 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:38.466274  2528 leader_election.cc:304] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.467324  2613 raft_consensus.cc:2804] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:38.467856  2613 raft_consensus.cc:697] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:38.468750  2613 consensus_queue.cc:237] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.469610  2633 consensus_queue.cc:1048] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.472661  2412 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:38.483664  2309 catalog_manager.cc:5697] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 1 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.567723  2640 consensus_queue.cc:1048] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.578948  2613 raft_consensus.cc:493] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.579671  2613 raft_consensus.cc:515] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.581368  2643 consensus_queue.cc:1048] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.582106  2613 leader_election.cc:290] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.584192  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.585518  2412 raft_consensus.cc:2393] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17ac24c81e5d47f1a15abae5f11e5cce in current term 1: Already voted for candidate fd75afe960fe4640b52ac3328158343e in this term.
I20260709 13:17:38.590649  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.592624  2528 leader_election.cc:304] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.594435  2613 raft_consensus.cc:3060] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.603229  2613 raft_consensus.cc:2749] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.652040  2613 raft_consensus.cc:493] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.653012  2613 raft_consensus.cc:515] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.655229  2613 leader_election.cc:290] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:38.656558  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:38.656558  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:38.657925  2488 raft_consensus.cc:2393] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17ac24c81e5d47f1a15abae5f11e5cce in current term 1: Already voted for candidate 641d982ff4c746e3bf32bbee02e44712 in this term.
I20260709 13:17:38.659075  2528 leader_election.cc:304] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.659864  2613 raft_consensus.cc:3060] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:38.665961  2613 raft_consensus.cc:2749] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:38.670662  2643 consensus_queue.cc:1048] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.680677  2629 consensus_queue.cc:1048] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.683306  2643 consensus_queue.cc:1048] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.695649  2629 consensus_queue.cc:1048] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.729493  2488 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
mode: GRACEFUL
new_leader_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:60438
I20260709 13:17:38.730576  2488 raft_consensus.cc:606] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Received request to transfer leadership to TS 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:38.731433  2633 consensus_queue.cc:1048] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.738083  2489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
mode: GRACEFUL
new_leader_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:60438
I20260709 13:17:38.738775  2489 raft_consensus.cc:606] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Received request to transfer leadership to TS 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:38.747984  2608 consensus_queue.cc:1048] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.774938  2613 consensus_queue.cc:1048] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.775336  2411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:33232
I20260709 13:17:38.776165  2411 raft_consensus.cc:606] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.785735  2613 consensus_queue.cc:1048] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.817003  2562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:17:38.817756  2562 raft_consensus.cc:606] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.823930  2562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:17:38.824573  2562 raft_consensus.cc:606] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:38.830717  2488 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
mode: GRACEFUL
new_leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:60438
I20260709 13:17:38.831293  2488 raft_consensus.cc:606] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Received request to transfer leadership to TS 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:38.843251  2640 consensus_queue.cc:1048] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.849993  2608 consensus_queue.cc:1048] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.856979  2659 consensus_queue.cc:1048] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.858402  2640 raft_consensus.cc:993] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: : Instructing follower 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:38.859038  2640 raft_consensus.cc:1081] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Signalling peer 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:38.862066  2413 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:33226
I20260709 13:17:38.862759  2413 raft_consensus.cc:493] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.863191  2413 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.865262  2629 consensus_queue.cc:1048] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.878453  2659 consensus_queue.cc:1048] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.890679  2633 consensus_queue.cc:1048] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.897336  2413 raft_consensus.cc:515] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.902426  2411 raft_consensus.cc:1240] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Rejecting Update request from peer fd75afe960fe4640b52ac3328158343e for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:38.909121  2640 consensus_queue.cc:1059] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.909545  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:38.910125  2640 raft_consensus.cc:3055] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.910516  2640 raft_consensus.cc:740] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.911579  2640 consensus_queue.cc:260] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.912375  2413 leader_election.cc:290] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.915140  2640 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.914939  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.915678  2636 raft_consensus.cc:993] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:38.916079  2563 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.916244  2636 raft_consensus.cc:1081] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:38.918951  2659 raft_consensus.cc:993] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:38.919497  2659 raft_consensus.cc:1081] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:38.927446  2563 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:38.929539  2377 leader_election.cc:304] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:38.930282  2488 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60428
I20260709 13:17:38.930660  2633 raft_consensus.cc:2804] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.931061  2488 raft_consensus.cc:493] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.931242  2633 raft_consensus.cc:697] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:38.931499  2488 raft_consensus.cc:3060] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.932242  2633 consensus_queue.cc:237] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.933771  2489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60434
I20260709 13:17:38.934507  2489 raft_consensus.cc:493] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.935099  2489 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.941720  2488 raft_consensus.cc:515] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.944329  2488 leader_election.cc:290] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.944754  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.945602  2413 raft_consensus.cc:3055] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.945958  2413 raft_consensus.cc:740] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:38.946866  2413 consensus_queue.cc:260] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.948342  2413 raft_consensus.cc:3060] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.950582  2489 raft_consensus.cc:515] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.951920  2487 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:38.954531  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.954551  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:38.955281  2562 raft_consensus.cc:3055] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.955369  2411 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.955699  2562 raft_consensus.cc:740] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:38.956769  2413 raft_consensus.cc:2468] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:38.956684  2562 consensus_queue.cc:260] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.957943  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a22a754bb6fd4bf5be2806cee04c1fef" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:38.958005  2562 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.958621  2563 raft_consensus.cc:3060] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.959157  2312 catalog_manager.cc:5697] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 1 to 2, leader changed from fd75afe960fe4640b52ac3328158343e (127.21.33.2) to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.962419  2453 leader_election.cc:304] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.963493  2629 raft_consensus.cc:2804] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.963994  2629 raft_consensus.cc:697] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.967249  2563 raft_consensus.cc:2468] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:38.964870  2629 consensus_queue.cc:237] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.973059  2489 leader_election.cc:290] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:38.974179  2562 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:38.975955  2452 leader_election.cc:304] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:38.976997  2609 raft_consensus.cc:2804] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.977576  2609 raft_consensus.cc:697] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:38.978518  2609 consensus_queue.cc:237] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:38.980947  2411 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:38.982577  2312 catalog_manager.cc:5697] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.997906  2312 catalog_manager.cc:5697] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.104528  2629 raft_consensus.cc:993] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: : Instructing follower 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:39.105100  2609 raft_consensus.cc:1081] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Signalling peer 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:39.106850  2562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:46200
I20260709 13:17:39.107472  2562 raft_consensus.cc:493] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.107793  2562 raft_consensus.cc:3060] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.113569  2562 raft_consensus.cc:515] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.115260  2562 leader_election.cc:290] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:39.116483  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:39.117152  2411 raft_consensus.cc:3060] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.116470  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88ed6a6dd1f1495abc4d09ca92e0f4ab" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:39.117939  2487 raft_consensus.cc:3055] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.118229  2487 raft_consensus.cc:740] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:39.119037  2487 consensus_queue.cc:260] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.120203  2487 raft_consensus.cc:3060] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.123369  2411 raft_consensus.cc:2468] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 2.
I20260709 13:17:39.124830  2527 leader_election.cc:304] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:39.125788  2685 raft_consensus.cc:2804] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.126320  2487 raft_consensus.cc:2468] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 2.
I20260709 13:17:39.126320  2685 raft_consensus.cc:697] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:39.127779  2685 consensus_queue.cc:237] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.133105  2629 raft_consensus.cc:993] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: : Instructing follower 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:39.133623  2609 raft_consensus.cc:1081] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Signalling peer 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:39.135344  2411 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:33226
I20260709 13:17:39.135960  2411 raft_consensus.cc:493] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.135650  2312 catalog_manager.cc:5697] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 1 to 2, leader changed from fd75afe960fe4640b52ac3328158343e (127.21.33.2) to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 2 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:39.136348  2411 raft_consensus.cc:3060] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.144686  2411 raft_consensus.cc:515] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.146776  2411 leader_election.cc:290] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:39.147686  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:39.147958  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d49b9acef904e9a9c0243c2880cedd4" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:39.148447  2487 raft_consensus.cc:3055] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.148555  2562 raft_consensus.cc:3060] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.148794  2487 raft_consensus.cc:740] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:39.149755  2487 consensus_queue.cc:260] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.150801  2487 raft_consensus.cc:3060] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.154836  2562 raft_consensus.cc:2468] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:39.156167  2377 leader_election.cc:304] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:39.156450  2487 raft_consensus.cc:2468] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:39.157073  2633 raft_consensus.cc:2804] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.157833  2633 raft_consensus.cc:697] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:39.158784  2633 consensus_queue.cc:237] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.165977  2312 catalog_manager.cc:5697] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 1 to 2, leader changed from fd75afe960fe4640b52ac3328158343e (127.21.33.2) to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.338780  2562 raft_consensus.cc:1275] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.338781  2487 raft_consensus.cc:1275] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.340796  2608 consensus_queue.cc:1048] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.341809  2633 consensus_queue.cc:1048] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.362536  2659 raft_consensus.cc:993] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:39.363163  2659 raft_consensus.cc:1081] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:39.365703  2487 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60434
I20260709 13:17:39.366490  2487 raft_consensus.cc:493] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.366995  2487 raft_consensus.cc:3060] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.375452  2487 raft_consensus.cc:515] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.377729  2487 leader_election.cc:290] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:39.378654  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:39.378854  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b97664c7c30434098b33c6307e2a071" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:39.379568  2411 raft_consensus.cc:3060] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.379613  2562 raft_consensus.cc:3055] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:39.380071  2562 raft_consensus.cc:740] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:39.380820  2562 consensus_queue.cc:260] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.381992  2562 raft_consensus.cc:3060] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:39.386925  2411 raft_consensus.cc:2468] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:39.388509  2453 leader_election.cc:304] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:39.388808  2562 raft_consensus.cc:2468] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:39.389613  2609 raft_consensus.cc:2804] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:39.390179  2609 raft_consensus.cc:697] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:39.391067  2609 consensus_queue.cc:237] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:39.399035  2310 catalog_manager.cc:5697] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.415194  2562 raft_consensus.cc:1275] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.417119  2609 consensus_queue.cc:1048] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.426440  2411 raft_consensus.cc:1275] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.428753  2609 consensus_queue.cc:1048] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:39.529342  2487 raft_consensus.cc:1275] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.530867  2685 consensus_queue.cc:1048] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.540701  2411 raft_consensus.cc:1275] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.543620  2689 consensus_queue.cc:1048] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:39.553192  2411 raft_consensus.cc:1275] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.555099  2609 consensus_queue.cc:1048] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.567039  2562 raft_consensus.cc:1275] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.568693  2609 consensus_queue.cc:1048] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:39.575119  2563 raft_consensus.cc:1275] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.577724  2608 consensus_queue.cc:1048] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.578680  2487 raft_consensus.cc:1275] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.580828  2608 consensus_queue.cc:1048] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.816941  2563 raft_consensus.cc:1275] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.818580  2609 consensus_queue.cc:1048] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.830631  2411 raft_consensus.cc:1275] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.832886  2643 consensus_queue.cc:1048] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:41.862165  2311 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50838:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:41.865124  2311 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:41.929607  2400 tablet_service.cc:1511] Processing CreateTablet for tablet b0d188cc51bf48d389a66c8fb4be7d70 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:41.930827  2472 tablet_service.cc:1511] Processing CreateTablet for tablet b0d188cc51bf48d389a66c8fb4be7d70 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:41.931505  2400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d188cc51bf48d389a66c8fb4be7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.932704  2472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d188cc51bf48d389a66c8fb4be7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.935101  2403 tablet_service.cc:1511] Processing CreateTablet for tablet 00d9e86a230e4a71a92ddac9e6b8df5c (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:41.936594  2479 tablet_service.cc:1511] Processing CreateTablet for tablet 00d9e86a230e4a71a92ddac9e6b8df5c (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:41.935202  2549 tablet_service.cc:1511] Processing CreateTablet for tablet b0d188cc51bf48d389a66c8fb4be7d70 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:41.938115  2479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d9e86a230e4a71a92ddac9e6b8df5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.938901  2549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d188cc51bf48d389a66c8fb4be7d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.937029  2403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d9e86a230e4a71a92ddac9e6b8df5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.946666  2546 tablet_service.cc:1511] Processing CreateTablet for tablet 00d9e86a230e4a71a92ddac9e6b8df5c (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:41.948293  2546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00d9e86a230e4a71a92ddac9e6b8df5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.951164  2398 tablet_service.cc:1511] Processing CreateTablet for tablet f7e1090e1c1c4b8b9ab62a00dbe87658 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:41.952675  2398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e1090e1c1c4b8b9ab62a00dbe87658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.979538  2473 tablet_service.cc:1511] Processing CreateTablet for tablet f7e1090e1c1c4b8b9ab62a00dbe87658 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:41.981194  2473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e1090e1c1c4b8b9ab62a00dbe87658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.981660  2551 tablet_service.cc:1511] Processing CreateTablet for tablet f7e1090e1c1c4b8b9ab62a00dbe87658 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:41.983165  2551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e1090e1c1c4b8b9ab62a00dbe87658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.986701  2404 tablet_service.cc:1511] Processing CreateTablet for tablet 52217a35c864463a89ea9456ef84d469 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:41.988289  2404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52217a35c864463a89ea9456ef84d469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.990046  2705 tablet_bootstrap.cc:492] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.005162  2704 tablet_bootstrap.cc:492] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.006220  2472 tablet_service.cc:1511] Processing CreateTablet for tablet 52217a35c864463a89ea9456ef84d469 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:42.008337  2472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52217a35c864463a89ea9456ef84d469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.021307  2704 tablet_bootstrap.cc:654] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.021688  2549 tablet_service.cc:1511] Processing CreateTablet for tablet 52217a35c864463a89ea9456ef84d469 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:42.028934  2549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52217a35c864463a89ea9456ef84d469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.030530  2705 tablet_bootstrap.cc:654] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.039184  2400 tablet_service.cc:1511] Processing CreateTablet for tablet 421c81aafc1d43a084fce36623a90d28 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:42.040850  2400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 421c81aafc1d43a084fce36623a90d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.046737  2476 tablet_service.cc:1511] Processing CreateTablet for tablet 421c81aafc1d43a084fce36623a90d28 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:42.048331  2476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 421c81aafc1d43a084fce36623a90d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.062500  2706 tablet_bootstrap.cc:492] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.065569  2404 tablet_service.cc:1511] Processing CreateTablet for tablet e718fcee53b14a67907e9102ac235c88 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:42.070417  2404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e718fcee53b14a67907e9102ac235c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.071943  2479 tablet_service.cc:1511] Processing CreateTablet for tablet e718fcee53b14a67907e9102ac235c88 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:42.073357  2479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e718fcee53b14a67907e9102ac235c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.086071  2398 tablet_service.cc:1511] Processing CreateTablet for tablet 69144e3ea31a47d8a6d835bc04709159 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:42.085726  2551 tablet_service.cc:1511] Processing CreateTablet for tablet e718fcee53b14a67907e9102ac235c88 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:42.087718  2398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69144e3ea31a47d8a6d835bc04709159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.091516  2705 tablet_bootstrap.cc:492] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.081859  2546 tablet_service.cc:1511] Processing CreateTablet for tablet 421c81aafc1d43a084fce36623a90d28 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:42.100536  2705 ts_tablet_manager.cc:1403] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.111s	user 0.014s	sys 0.005s
I20260709 13:17:42.088649  2551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e718fcee53b14a67907e9102ac235c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.107923  2473 tablet_service.cc:1511] Processing CreateTablet for tablet 69144e3ea31a47d8a6d835bc04709159 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:42.110162  2473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69144e3ea31a47d8a6d835bc04709159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.113585  2705 raft_consensus.cc:359] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.114630  2705 raft_consensus.cc:385] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.114986  2705 raft_consensus.cc:740] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.110028  2549 tablet_service.cc:1511] Processing CreateTablet for tablet 69144e3ea31a47d8a6d835bc04709159 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:42.115908  2705 consensus_queue.cc:260] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.124410  2704 tablet_bootstrap.cc:492] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.118633  2546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 421c81aafc1d43a084fce36623a90d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.129278  2704 ts_tablet_manager.cc:1403] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.124s	user 0.029s	sys 0.052s
I20260709 13:17:42.133399  2706 tablet_bootstrap.cc:654] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.133023  2704 raft_consensus.cc:359] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.146112  2704 raft_consensus.cc:385] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.146520  2704 raft_consensus.cc:740] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.147329  2704 consensus_queue.cc:260] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.160604  2704 ts_tablet_manager.cc:1434] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.002s
I20260709 13:17:42.161903  2704 tablet_bootstrap.cc:492] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.162031  2549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69144e3ea31a47d8a6d835bc04709159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.169406  2398 tablet_service.cc:1511] Processing CreateTablet for tablet dc8cd7e640e94674a822c2ace3fdb0ba (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:42.172005  2398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc8cd7e640e94674a822c2ace3fdb0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.177776  2705 ts_tablet_manager.cc:1434] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.000s
I20260709 13:17:42.181423  2705 tablet_bootstrap.cc:492] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.184254  2704 tablet_bootstrap.cc:654] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.186988  2546 tablet_service.cc:1511] Processing CreateTablet for tablet dc8cd7e640e94674a822c2ace3fdb0ba (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:42.188956  2706 tablet_bootstrap.cc:492] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.188947  2546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc8cd7e640e94674a822c2ace3fdb0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.189899  2706 ts_tablet_manager.cc:1403] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.128s	user 0.019s	sys 0.000s
I20260709 13:17:42.202880  2400 tablet_service.cc:1511] Processing CreateTablet for tablet eaba0a4408ee4af4a78ef5354911ebd7 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:42.205020  2400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaba0a4408ee4af4a78ef5354911ebd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.214411  2479 tablet_service.cc:1511] Processing CreateTablet for tablet dc8cd7e640e94674a822c2ace3fdb0ba (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:42.222724  2692 raft_consensus.cc:493] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.227269  2706 raft_consensus.cc:359] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.231700  2692 raft_consensus.cc:515] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.233924  2476 tablet_service.cc:1511] Processing CreateTablet for tablet eaba0a4408ee4af4a78ef5354911ebd7 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:42.234297  2692 leader_election.cc:290] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.235258  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d188cc51bf48d389a66c8fb4be7d70" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:42.238059  2476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaba0a4408ee4af4a78ef5354911ebd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.238611  2487 raft_consensus.cc:2468] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:42.240923  2379 leader_election.cc:304] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:42.244602  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d188cc51bf48d389a66c8fb4be7d70" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:42.230648  2706 raft_consensus.cc:385] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.248214  2706 raft_consensus.cc:740] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.249370  2706 consensus_queue.cc:260] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.249891  2547 tablet_service.cc:1511] Processing CreateTablet for tablet eaba0a4408ee4af4a78ef5354911ebd7 (DEFAULT_TABLE table=soft_deleted_table [id=70baf6033554420899d4d95b1054407e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:42.251735  2547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaba0a4408ee4af4a78ef5354911ebd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.252774  2706 ts_tablet_manager.cc:1434] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.057s	user 0.015s	sys 0.007s
I20260709 13:17:42.254233  2706 tablet_bootstrap.cc:492] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.257807  2692 raft_consensus.cc:2804] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.258826  2692 raft_consensus.cc:493] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.262162  2692 raft_consensus.cc:3060] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.276896  2706 tablet_bootstrap.cc:654] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.280130  2704 tablet_bootstrap.cc:492] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.280999  2704 ts_tablet_manager.cc:1403] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.119s	user 0.030s	sys 0.013s
I20260709 13:17:42.278422  2705 tablet_bootstrap.cc:654] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.282258  2479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc8cd7e640e94674a822c2ace3fdb0ba. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:42.290351  2377 leader_election.cc:343] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.292737  2704 raft_consensus.cc:359] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.294916  2704 raft_consensus.cc:385] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.295274  2704 raft_consensus.cc:740] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.296034  2704 consensus_queue.cc:260] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.298761  2704 ts_tablet_manager.cc:1434] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:17:42.306619  2704 tablet_bootstrap.cc:492] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.298292  2692 raft_consensus.cc:515] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.308866  2692 leader_election.cc:290] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.311750  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d188cc51bf48d389a66c8fb4be7d70" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.312809  2487 raft_consensus.cc:3060] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.323237  2704 tablet_bootstrap.cc:654] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.326438  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d188cc51bf48d389a66c8fb4be7d70" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:42.333205  2487 raft_consensus.cc:2468] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
W20260709 13:17:42.329128  2377 leader_election.cc:343] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.336146  2379 leader_election.cc:304] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:42.338459  2712 raft_consensus.cc:2804] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.339006  2712 raft_consensus.cc:697] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:42.339749  2706 tablet_bootstrap.cc:492] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.340521  2706 ts_tablet_manager.cc:1403] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.087s	user 0.031s	sys 0.021s
I20260709 13:17:42.339900  2712 consensus_queue.cc:237] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.341667  2705 tablet_bootstrap.cc:492] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.342314  2705 ts_tablet_manager.cc:1403] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.161s	user 0.023s	sys 0.010s
I20260709 13:17:42.344205  2706 raft_consensus.cc:359] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.345803  2706 raft_consensus.cc:385] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.346374  2706 raft_consensus.cc:740] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.346575  2705 raft_consensus.cc:359] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.347045  2704 tablet_bootstrap.cc:492] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.347608  2705 raft_consensus.cc:385] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.347432  2706 consensus_queue.cc:260] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.348528  2704 ts_tablet_manager.cc:1403] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.001s
I20260709 13:17:42.356694  2706 ts_tablet_manager.cc:1434] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.002s
I20260709 13:17:42.352694  2705 raft_consensus.cc:740] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.358182  2706 tablet_bootstrap.cc:492] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.361640  2689 raft_consensus.cc:493] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.361651  2705 consensus_queue.cc:260] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.362484  2689 raft_consensus.cc:515] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.364809  2705 ts_tablet_manager.cc:1434] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:17:42.366230  2705 tablet_bootstrap.cc:492] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.367221  2706 tablet_bootstrap.cc:654] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.366935  2704 raft_consensus.cc:359] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.367877  2704 raft_consensus.cc:385] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.368248  2704 raft_consensus.cc:740] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.369436  2704 consensus_queue.cc:260] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.381889  2689 leader_election.cc:290] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.382313  2704 ts_tablet_manager.cc:1434] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.006s
I20260709 13:17:42.382575  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.383306  2411 raft_consensus.cc:2468] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.383596  2704 tablet_bootstrap.cc:492] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.384912  2527 leader_election.cc:304] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:42.385061  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
W20260709 13:17:42.386758  2528 leader_election.cc:343] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.386790  2312 catalog_manager.cc:5697] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.377380  2705 tablet_bootstrap.cc:654] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.391059  2689 raft_consensus.cc:2804] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.393324  2704 tablet_bootstrap.cc:654] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.396008  2689 raft_consensus.cc:493] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.397097  2689 raft_consensus.cc:3060] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.408910  2689 raft_consensus.cc:515] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.414070  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:42.414906  2411 raft_consensus.cc:3060] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.416179  2706 tablet_bootstrap.cc:492] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.416838  2706 ts_tablet_manager.cc:1403] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.025s
I20260709 13:17:42.421402  2706 raft_consensus.cc:359] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.422753  2706 raft_consensus.cc:385] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.423235  2706 raft_consensus.cc:740] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.424247  2706 consensus_queue.cc:260] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.426944  2705 tablet_bootstrap.cc:492] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.430349  2705 ts_tablet_manager.cc:1403] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.025s
I20260709 13:17:42.435379  2706 ts_tablet_manager.cc:1434] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.018s	user 0.018s	sys 0.000s
I20260709 13:17:42.436321  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.437515  2706 tablet_bootstrap.cc:492] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
W20260709 13:17:42.438102  2528 leader_election.cc:343] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.439023  2689 leader_election.cc:290] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.440099  2411 raft_consensus.cc:2468] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:42.441896  2527 leader_election.cc:304] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.443212  2689 raft_consensus.cc:2804] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.443779  2689 raft_consensus.cc:697] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:42.444811  2689 consensus_queue.cc:237] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.449280  2706 tablet_bootstrap.cc:654] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.451640  2705 raft_consensus.cc:359] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.452679  2705 raft_consensus.cc:385] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.453024  2705 raft_consensus.cc:740] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.453958  2705 consensus_queue.cc:260] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.456352  2704 tablet_bootstrap.cc:492] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.457370  2704 ts_tablet_manager.cc:1403] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.074s	user 0.015s	sys 0.025s
I20260709 13:17:42.460387  2705 ts_tablet_manager.cc:1434] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:17:42.461602  2704 raft_consensus.cc:359] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.462540  2704 raft_consensus.cc:385] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.462918  2704 raft_consensus.cc:740] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.463158  2706 tablet_bootstrap.cc:492] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.463797  2706 ts_tablet_manager.cc:1403] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.006s
I20260709 13:17:42.463900  2704 consensus_queue.cc:260] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.467330  2704 ts_tablet_manager.cc:1434] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:42.467828  2706 raft_consensus.cc:359] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.468673  2704 tablet_bootstrap.cc:492] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.468753  2706 raft_consensus.cc:385] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.469319  2706 raft_consensus.cc:740] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.470386  2706 consensus_queue.cc:260] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.473397  2706 ts_tablet_manager.cc:1434] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.008s
I20260709 13:17:42.474773  2706 tablet_bootstrap.cc:492] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.475956  2310 catalog_manager.cc:5697] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 1 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.477849  2704 tablet_bootstrap.cc:654] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.480255  2705 tablet_bootstrap.cc:492] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.483162  2706 tablet_bootstrap.cc:654] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.488999  2705 tablet_bootstrap.cc:654] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.500756  2706 tablet_bootstrap.cc:492] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.500756  2704 tablet_bootstrap.cc:492] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.501660  2704 ts_tablet_manager.cc:1403] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.011s
I20260709 13:17:42.501660  2706 ts_tablet_manager.cc:1403] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 13:17:42.505383  2706 raft_consensus.cc:359] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.505702  2704 raft_consensus.cc:359] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.506343  2706 raft_consensus.cc:385] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.506522  2704 raft_consensus.cc:385] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.506844  2706 raft_consensus.cc:740] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.506877  2704 raft_consensus.cc:740] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.507875  2706 consensus_queue.cc:260] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.507896  2704 consensus_queue.cc:260] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.510866  2706 ts_tablet_manager.cc:1434] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.006s
I20260709 13:17:42.511009  2704 ts_tablet_manager.cc:1434] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:42.512117  2706 tablet_bootstrap.cc:492] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.512117  2704 tablet_bootstrap.cc:492] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.521083  2704 tablet_bootstrap.cc:654] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.521900  2706 tablet_bootstrap.cc:654] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.528287  2705 tablet_bootstrap.cc:492] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.528925  2705 ts_tablet_manager.cc:1403] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.009s
I20260709 13:17:42.532582  2705 raft_consensus.cc:359] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.533212  2705 raft_consensus.cc:385] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.533238  2706 tablet_bootstrap.cc:492] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.533586  2705 raft_consensus.cc:740] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.533968  2706 ts_tablet_manager.cc:1403] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.003s
I20260709 13:17:42.534646  2705 consensus_queue.cc:260] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.536083  2704 tablet_bootstrap.cc:492] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.536731  2704 ts_tablet_manager.cc:1403] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.005s
I20260709 13:17:42.537401  2705 ts_tablet_manager.cc:1434] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:17:42.538105  2706 raft_consensus.cc:359] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.539052  2705 tablet_bootstrap.cc:492] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.540604  2704 raft_consensus.cc:359] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.541541  2704 raft_consensus.cc:385] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.541981  2704 raft_consensus.cc:740] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.539072  2706 raft_consensus.cc:385] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.543263  2706 raft_consensus.cc:740] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.542894  2704 consensus_queue.cc:260] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.544123  2706 consensus_queue.cc:260] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.544982  2640 raft_consensus.cc:493] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.545656  2640 raft_consensus.cc:515] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.547362  2706 ts_tablet_manager.cc:1434] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:17:42.547979  2640 leader_election.cc:290] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.548609  2706 tablet_bootstrap.cc:492] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.550239  2704 ts_tablet_manager.cc:1434] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:17:42.551640  2704 tablet_bootstrap.cc:492] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.552397  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52217a35c864463a89ea9456ef84d469" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.559492  2704 tablet_bootstrap.cc:654] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.559265  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52217a35c864463a89ea9456ef84d469" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:42.567202  2562 raft_consensus.cc:2468] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:42.561978  2689 raft_consensus.cc:493] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.569067  2689 raft_consensus.cc:515] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.570866  2411 raft_consensus.cc:2468] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 0.
I20260709 13:17:42.562384  2705 tablet_bootstrap.cc:654] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.573917  2453 leader_election.cc:304] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:42.557381  2706 tablet_bootstrap.cc:654] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.577651  2640 raft_consensus.cc:2804] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.580965  2640 raft_consensus.cc:493] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.588286  2640 raft_consensus.cc:3060] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.590334  2720 raft_consensus.cc:493] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.583608  2689 leader_election.cc:290] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.585764  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:42.591176  2720 raft_consensus.cc:515] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.592160  2487 raft_consensus.cc:2468] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.595530  2720 leader_election.cc:290] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.582059  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.596903  2411 raft_consensus.cc:2468] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.595618  2528 leader_election.cc:304] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:42.599092  2706 tablet_bootstrap.cc:492] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.608088  2706 ts_tablet_manager.cc:1403] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.006s
I20260709 13:17:42.610042  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d188cc51bf48d389a66c8fb4be7d70" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:42.610695  2720 raft_consensus.cc:2804] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.611029  2487 raft_consensus.cc:2393] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 17ac24c81e5d47f1a15abae5f11e5cce in current term 1: Already voted for candidate 641d982ff4c746e3bf32bbee02e44712 in this term.
I20260709 13:17:42.598757  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d188cc51bf48d389a66c8fb4be7d70" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.602035  2712 raft_consensus.cc:493] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.611167  2720 raft_consensus.cc:493] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.612773  2706 raft_consensus.cc:359] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.612776  2712 raft_consensus.cc:515] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.614073  2706 raft_consensus.cc:385] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.612927  2720 raft_consensus.cc:3060] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.614569  2706 raft_consensus.cc:740] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.615775  2706 consensus_queue.cc:260] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.617070  2712 leader_election.cc:290] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.605052  2705 tablet_bootstrap.cc:492] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.619175  2705 ts_tablet_manager.cc:1403] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.004s
I20260709 13:17:42.619897  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
W20260709 13:17:42.627097  2379 leader_election.cc:343] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.631057  2640 raft_consensus.cc:515] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.632303  2720 raft_consensus.cc:515] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.634073  2528 leader_election.cc:304] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.619235  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:42.635823  2562 raft_consensus.cc:2468] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:42.639048  2705 raft_consensus.cc:359] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.639375  2377 leader_election.cc:304] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.639930  2705 raft_consensus.cc:385] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.640260  2705 raft_consensus.cc:740] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.640733  2712 raft_consensus.cc:2804] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.641229  2712 raft_consensus.cc:493] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.642890  2712 raft_consensus.cc:3060] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.645159  2705 consensus_queue.cc:260] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.646029  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:42.646432  2720 leader_election.cc:290] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.646670  2411 raft_consensus.cc:3060] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.648078  2720 raft_consensus.cc:3060] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.654013  2705 ts_tablet_manager.cc:1434] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:17:42.666033  2705 tablet_bootstrap.cc:492] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.676254  2720 raft_consensus.cc:2749] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:42.676067  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52217a35c864463a89ea9456ef84d469" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:42.677213  2731 raft_consensus.cc:493] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.677425  2413 raft_consensus.cc:3060] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.677918  2704 tablet_bootstrap.cc:492] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.677863  2731 raft_consensus.cc:515] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.678668  2704 ts_tablet_manager.cc:1403] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.127s	user 0.063s	sys 0.041s
I20260709 13:17:42.681391  2706 ts_tablet_manager.cc:1434] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.073s	user 0.032s	sys 0.029s
I20260709 13:17:42.682269  2412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.682732  2705 tablet_bootstrap.cc:654] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.682874  2706 tablet_bootstrap.cc:492] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.682989  2412 raft_consensus.cc:2468] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.684607  2527 leader_election.cc:304] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:42.685560  2689 raft_consensus.cc:2804] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.685956  2689 raft_consensus.cc:493] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.686322  2689 raft_consensus.cc:3060] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.688323  2704 raft_consensus.cc:359] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.689528  2704 raft_consensus.cc:385] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.689872  2704 raft_consensus.cc:740] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.690748  2704 consensus_queue.cc:260] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.693543  2704 ts_tablet_manager.cc:1434] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:17:42.695101  2706 tablet_bootstrap.cc:654] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.695468  2704 tablet_bootstrap.cc:492] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.707286  2731 leader_election.cc:290] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.704885  2689 raft_consensus.cc:515] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.717738  2413 raft_consensus.cc:2468] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:42.718115  2704 tablet_bootstrap.cc:654] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.719290  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:42.732951  2414 raft_consensus.cc:3060] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.734057  2453 leader_election.cc:304] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:42.735941  2693 raft_consensus.cc:2804] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.719846  2640 leader_election.cc:290] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.741293  2414 raft_consensus.cc:2468] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:42.723157  2689 leader_election.cc:290] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:42.727347  2528 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.33.2:39539, user_credentials={real_user=slave}} blocked reactor thread for 93380.4us
I20260709 13:17:42.742914  2527 leader_election.cc:304] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:42.744537  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.745270  2487 raft_consensus.cc:3060] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.745406  2489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:42.746163  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.746390  2489 raft_consensus.cc:2468] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.747067  2689 raft_consensus.cc:2804] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.747879  2689 raft_consensus.cc:697] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:42.748991  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52217a35c864463a89ea9456ef84d469" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:42.750036  2563 raft_consensus.cc:3060] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.749188  2689 consensus_queue.cc:237] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.728310  2706 tablet_bootstrap.cc:492] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.751766  2706 ts_tablet_manager.cc:1403] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.069s	user 0.023s	sys 0.014s
I20260709 13:17:42.757005  2706 raft_consensus.cc:359] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.758093  2704 tablet_bootstrap.cc:492] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.758963  2704 ts_tablet_manager.cc:1403] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.006s
I20260709 13:17:42.730940  2411 raft_consensus.cc:2468] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:42.721853  2712 raft_consensus.cc:515] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.758163  2706 raft_consensus.cc:385] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.761238  2527 leader_election.cc:304] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:42.762243  2706 raft_consensus.cc:740] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.762466  2487 raft_consensus.cc:2468] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:42.762416  2704 raft_consensus.cc:359] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.762974  2732 raft_consensus.cc:2804] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.763563  2704 raft_consensus.cc:385] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.763151  2706 consensus_queue.cc:260] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.764014  2732 raft_consensus.cc:697] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:42.764226  2704 raft_consensus.cc:740] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.765240  2704 consensus_queue.cc:260] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.765439  2732 consensus_queue.cc:237] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.768018  2704 ts_tablet_manager.cc:1434] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:42.769546  2704 tablet_bootstrap.cc:492] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: Bootstrap starting.
I20260709 13:17:42.773634  2706 ts_tablet_manager.cc:1434] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:17:42.775224  2706 tablet_bootstrap.cc:492] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: Bootstrap starting.
I20260709 13:17:42.777798  2704 tablet_bootstrap.cc:654] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.778661  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.782384  2706 tablet_bootstrap.cc:654] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:42.782992  2379 leader_election.cc:343] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.791827  2704 tablet_bootstrap.cc:492] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: No bootstrap required, opened a new log
I20260709 13:17:42.793008  2704 ts_tablet_manager.cc:1403] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 13:17:42.798414  2704 raft_consensus.cc:359] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.799237  2704 raft_consensus.cc:385] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.799634  2704 raft_consensus.cc:740] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.800333  2704 consensus_queue.cc:260] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.800645  2309 catalog_manager.cc:5697] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 1 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.804812  2704 ts_tablet_manager.cc:1434] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:17:42.805311  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:42.806214  2562 raft_consensus.cc:3060] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.810806  2692 raft_consensus.cc:493] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.811318  2706 tablet_bootstrap.cc:492] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: No bootstrap required, opened a new log
I20260709 13:17:42.811434  2692 raft_consensus.cc:515] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.811919  2706 ts_tablet_manager.cc:1403] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.010s
I20260709 13:17:42.816067  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:42.816792  2706 raft_consensus.cc:359] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.818233  2706 raft_consensus.cc:385] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:42.818096  2379 leader_election.cc:343] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.818715  2706 raft_consensus.cc:740] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.816982  2712 leader_election.cc:290] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.819536  2706 consensus_queue.cc:260] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.821008  2692 leader_election.cc:290] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
W20260709 13:17:42.821888  2360 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:42.822189  2706 ts_tablet_manager.cc:1434] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:42.822441  2693 raft_consensus.cc:697] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:42.823009  2560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:42.826357  2560 raft_consensus.cc:2468] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:42.825286  2705 tablet_bootstrap.cc:492] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.826357  2693 consensus_queue.cc:237] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.828433  2377 leader_election.cc:304] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.829629  2705 ts_tablet_manager.cc:1403] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.162s	user 0.019s	sys 0.019s
I20260709 13:17:42.830024  2692 raft_consensus.cc:2804] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.830492  2692 raft_consensus.cc:493] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.830838  2692 raft_consensus.cc:3060] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.833031  2705 raft_consensus.cc:359] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.833885  2705 raft_consensus.cc:385] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.834241  2705 raft_consensus.cc:740] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.816854  2562 raft_consensus.cc:2468] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:42.836714  2377 leader_election.cc:304] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.837205  2705 consensus_queue.cc:260] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.840173  2705 ts_tablet_manager.cc:1434] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:42.841755  2705 tablet_bootstrap.cc:492] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.844962  2563 raft_consensus.cc:2468] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 1.
I20260709 13:17:42.845803  2712 raft_consensus.cc:2804] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.847090  2692 raft_consensus.cc:515] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.848351  2689 raft_consensus.cc:493] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.851154  2689 raft_consensus.cc:515] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.848332  2712 raft_consensus.cc:697] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:42.850417  2743 raft_consensus.cc:493] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.854933  2689 leader_election.cc:290] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.857812  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.857988  2712 consensus_queue.cc:237] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.858727  2411 raft_consensus.cc:2468] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.858749  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
I20260709 13:17:42.860450  2488 raft_consensus.cc:2468] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 0.
I20260709 13:17:42.853564  2743 raft_consensus.cc:515] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.868549  2528 leader_election.cc:304] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:42.870285  2689 raft_consensus.cc:2804] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.870749  2689 raft_consensus.cc:493] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.870941  2705 tablet_bootstrap.cc:654] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.871172  2689 raft_consensus.cc:3060] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.871419  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.873587  2736 raft_consensus.cc:493] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:42.873971  2379 leader_election.cc:343] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.875061  2736 raft_consensus.cc:515] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.876879  2743 leader_election.cc:290] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.877197  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e" is_pre_election: true
W20260709 13:17:42.879046  2379 leader_election.cc:343] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.879189  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:42.880026  2562 raft_consensus.cc:2468] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 0.
I20260709 13:17:42.882396  2377 leader_election.cc:304] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.884291  2744 raft_consensus.cc:2804] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:42.884744  2744 raft_consensus.cc:493] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:42.885093  2744 raft_consensus.cc:3060] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.885627  2309 catalog_manager.cc:5697] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 0 to 1, leader changed from <none> to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 1 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.862450  2310 catalog_manager.cc:5697] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 1 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.889127  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:42.890399  2411 raft_consensus.cc:2393] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd75afe960fe4640b52ac3328158343e in current term 1: Already voted for candidate 17ac24c81e5d47f1a15abae5f11e5cce in this term.
I20260709 13:17:42.890736  2689 raft_consensus.cc:515] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.892444  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:42.894757  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:42.895506  2411 raft_consensus.cc:3060] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.895447  2453 leader_election.cc:304] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:42.889218  2736 leader_election.cc:290] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.906925  2488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:42.907670  2488 raft_consensus.cc:3060] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.908488  2640 raft_consensus.cc:3060] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.909037  2705 tablet_bootstrap.cc:492] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.909906  2705 ts_tablet_manager.cc:1403] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.008s
I20260709 13:17:42.916194  2744 raft_consensus.cc:515] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.917146  2640 raft_consensus.cc:2749] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:42.920449  2705 raft_consensus.cc:359] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.925334  2705 raft_consensus.cc:385] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.925837  2705 raft_consensus.cc:740] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.926964  2705 consensus_queue.cc:260] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.929052  2689 leader_election.cc:290] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:42.929823  2705 ts_tablet_manager.cc:1434] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 13:17:42.930981  2705 tablet_bootstrap.cc:492] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.931316  2692 leader_election.cc:290] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.932730  2562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:42.933430  2562 raft_consensus.cc:3060] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.934129  2744 leader_election.cc:290] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:42.935664  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd75afe960fe4640b52ac3328158343e"
W20260709 13:17:42.937408  2379 leader_election.cc:343] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.940433  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:42.941228  2563 raft_consensus.cc:3060] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:42.941776  2705 tablet_bootstrap.cc:654] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.943209  2311 catalog_manager.cc:5697] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.952793  2744 consensus_queue.cc:1048] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:42.953480  2705 tablet_bootstrap.cc:492] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.954185  2705 ts_tablet_manager.cc:1403] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.015s
I20260709 13:17:42.959403  2705 raft_consensus.cc:359] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.960284  2705 raft_consensus.cc:385] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.960567  2705 raft_consensus.cc:740] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.961299  2705 consensus_queue.cc:260] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.961455  2744 consensus_queue.cc:1048] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:42.963789  2705 ts_tablet_manager.cc:1434] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:42.964665  2705 tablet_bootstrap.cc:492] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: Bootstrap starting.
I20260709 13:17:42.968374  2488 raft_consensus.cc:2468] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:42.970291  2528 leader_election.cc:304] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:42.971783  2705 tablet_bootstrap.cc:654] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.971753  2689 raft_consensus.cc:2804] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.974622  2689 raft_consensus.cc:697] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:42.975999  2689 consensus_queue.cc:237] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.979251  2411 raft_consensus.cc:2468] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 1.
I20260709 13:17:42.981758  2563 raft_consensus.cc:2468] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:42.983392  2377 leader_election.cc:304] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.983568  2562 raft_consensus.cc:2468] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 1.
I20260709 13:17:42.984150  2705 tablet_bootstrap.cc:492] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: No bootstrap required, opened a new log
I20260709 13:17:42.984740  2705 ts_tablet_manager.cc:1403] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:17:42.984865  2377 leader_election.cc:304] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:42.986177  2692 raft_consensus.cc:2804] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.987006  2692 raft_consensus.cc:697] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:42.987162  2712 raft_consensus.cc:2804] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:42.987931  2712 raft_consensus.cc:697] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:42.987964  2705 raft_consensus.cc:359] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.988996  2705 raft_consensus.cc:385] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.988333  2692 consensus_queue.cc:237] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.989310  2705 raft_consensus.cc:740] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.990187  2705 consensus_queue.cc:260] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.989132  2712 consensus_queue.cc:237] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:42.992856  2705 ts_tablet_manager.cc:1434] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:43.000435  2732 consensus_queue.cc:1048] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.041203  2310 catalog_manager.cc:5697] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.041451  2311 catalog_manager.cc:5697] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 0 to 1, leader changed from <none> to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 1 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:43.044044  2310 catalog_manager.cc:5697] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 1 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.047221  2732 consensus_queue.cc:1048] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.110936  2560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
mode: GRACEFUL
new_leader_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:17:43.111743  2560 raft_consensus.cc:606] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Received request to transfer leadership to TS 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.117621  2560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
mode: GRACEFUL
new_leader_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:17:43.118274  2560 raft_consensus.cc:606] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Received request to transfer leadership to TS 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.123639  2489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52217a35c864463a89ea9456ef84d469"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
mode: GRACEFUL
new_leader_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:60438
I20260709 13:17:43.124305  2489 raft_consensus.cc:606] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Received request to transfer leadership to TS 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.130255  2560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:17:43.130910  2560 raft_consensus.cc:606] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:43.136289  2560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:17:43.136932  2560 raft_consensus.cc:606] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:43.142333  2411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:33232
I20260709 13:17:43.142943  2411 raft_consensus.cc:606] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:43.148758  2411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
mode: GRACEFUL
new_leader_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:33232
I20260709 13:17:43.149381  2411 raft_consensus.cc:606] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Received request to transfer leadership to TS fd75afe960fe4640b52ac3328158343e
I20260709 13:17:43.155411  2411 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
mode: GRACEFUL
new_leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:33232
I20260709 13:17:43.155961  2411 raft_consensus.cc:606] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Received request to transfer leadership to TS 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:43.219023  2693 consensus_queue.cc:1048] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:43.232074  2736 consensus_queue.cc:1048] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:43.244266  2736 raft_consensus.cc:993] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: : Instructing follower 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:43.244974  2736 raft_consensus.cc:1081] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Signalling peer 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:43.245824  2732 consensus_queue.cc:1048] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.246670  2411 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "52217a35c864463a89ea9456ef84d469"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:33226
I20260709 13:17:43.247139  2411 raft_consensus.cc:493] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.247424  2411 raft_consensus.cc:3060] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.251533  2736 raft_consensus.cc:493] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.252388  2736 raft_consensus.cc:515] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.255882  2736 leader_election.cc:290] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.256826  2560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:43.258463  2489 raft_consensus.cc:3060] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:43.260305  2560 raft_consensus.cc:2393] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd75afe960fe4640b52ac3328158343e in current term 1: Already voted for candidate 641d982ff4c746e3bf32bbee02e44712 in this term.
I20260709 13:17:43.264321  2488 raft_consensus.cc:3060] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:43.258299  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:43.258872  2411 raft_consensus.cc:515] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.270493  2732 consensus_queue.cc:1048] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:43.271132  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52217a35c864463a89ea9456ef84d469" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:43.271728  2560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52217a35c864463a89ea9456ef84d469" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.271901  2487 raft_consensus.cc:3055] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.272802  2487 raft_consensus.cc:740] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:43.272287  2411 leader_election.cc:290] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.273463  2736 raft_consensus.cc:493] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.272895  2560 raft_consensus.cc:3060] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.273813  2487 consensus_queue.cc:260] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.274065  2736 raft_consensus.cc:515] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.275182  2487 raft_consensus.cc:3060] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.278831  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:43.279759  2563 raft_consensus.cc:2393] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd75afe960fe4640b52ac3328158343e in current term 1: Already voted for candidate 641d982ff4c746e3bf32bbee02e44712 in this term.
I20260709 13:17:43.282189  2453 leader_election.cc:304] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.283524  2736 leader_election.cc:290] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.284332  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:43.285863  2453 leader_election.cc:304] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.286141  2760 raft_consensus.cc:2749] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:43.287045  2736 raft_consensus.cc:3060] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:43.289768  2692 consensus_queue.cc:1048] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.294230  2732 raft_consensus.cc:993] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.294904  2732 raft_consensus.cc:1081] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.297022  2486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60434
I20260709 13:17:43.297711  2486 raft_consensus.cc:493] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.298098  2486 raft_consensus.cc:3060] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.298224  2560 raft_consensus.cc:2468] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:43.299733  2377 leader_election.cc:304] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:43.300776  2692 raft_consensus.cc:2804] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.300666  2736 raft_consensus.cc:2749] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:43.301472  2692 raft_consensus.cc:697] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:43.302379  2692 consensus_queue.cc:237] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.321621  2486 raft_consensus.cc:515] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.322084  2736 raft_consensus.cc:493] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.322846  2736 raft_consensus.cc:515] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.325214  2486 leader_election.cc:290] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.325210  2736 leader_election.cc:290] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.325994  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:43.326725  2414 raft_consensus.cc:3060] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.328594  2560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e718fcee53b14a67907e9102ac235c88" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.329397  2560 raft_consensus.cc:3055] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.328733  2309 catalog_manager.cc:5697] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 1 to 2, leader changed from fd75afe960fe4640b52ac3328158343e (127.21.33.2) to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.329828  2560 raft_consensus.cc:740] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:43.330355  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" is_pre_election: true
I20260709 13:17:43.331239  2563 raft_consensus.cc:2393] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd75afe960fe4640b52ac3328158343e in current term 1: Already voted for candidate 641d982ff4c746e3bf32bbee02e44712 in this term.
I20260709 13:17:43.331082  2560 consensus_queue.cc:260] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.332540  2560 raft_consensus.cc:3060] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.334833  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "641d982ff4c746e3bf32bbee02e44712" is_pre_election: true
I20260709 13:17:43.336568  2453 leader_election.cc:304] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd75afe960fe4640b52ac3328158343e; no voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.337936  2744 raft_consensus.cc:993] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.338474  2744 raft_consensus.cc:1081] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.337440  2736 raft_consensus.cc:3060] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:43.342206  2489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60428
I20260709 13:17:43.342878  2489 raft_consensus.cc:493] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.343262  2489 raft_consensus.cc:3060] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.343470  2744 consensus_queue.cc:1048] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.342223  2720 consensus_queue.cc:1048] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:17:43.350050  2736 raft_consensus.cc:2749] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:43.352551  2560 raft_consensus.cc:2468] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.354019  2452 leader_election.cc:304] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:43.355345  2736 raft_consensus.cc:2804] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.356069  2736 raft_consensus.cc:697] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:43.356945  2736 consensus_queue.cc:237] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.363979  2732 raft_consensus.cc:993] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: : Instructing follower 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:43.364606  2732 raft_consensus.cc:1081] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Signalling peer 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:43.367002  2411 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:33212
I20260709 13:17:43.367589  2411 raft_consensus.cc:493] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.368317  2411 raft_consensus.cc:3060] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.369640  2414 raft_consensus.cc:2468] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.375041  2489 raft_consensus.cc:515] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.376308  2486 raft_consensus.cc:1240] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Rejecting Update request from peer 641d982ff4c746e3bf32bbee02e44712 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:43.381162  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:43.381691  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69144e3ea31a47d8a6d835bc04709159" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.381932  2413 raft_consensus.cc:3055] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.382293  2563 raft_consensus.cc:3060] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.382843  2720 consensus_queue.cc:1048] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:43.382302  2413 raft_consensus.cc:740] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:43.384783  2712 consensus_queue.cc:1059] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:43.379114  2489 leader_election.cc:290] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.376643  2487 raft_consensus.cc:2468] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:43.376042  2411 raft_consensus.cc:515] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.388976  2413 consensus_queue.cc:260] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.390410  2414 raft_consensus.cc:1240] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Rejecting Update request from peer 17ac24c81e5d47f1a15abae5f11e5cce for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:43.391981  2411 leader_election.cc:290] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.392802  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:43.393025  2563 raft_consensus.cc:2468] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.393872  2560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e1090e1c1c4b8b9ab62a00dbe87658" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.394680  2560 raft_consensus.cc:3055] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.395100  2560 raft_consensus.cc:740] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:43.395505  2452 leader_election.cc:304] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:43.396394  2736 raft_consensus.cc:2804] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.396275  2560 consensus_queue.cc:260] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.397217  2736 raft_consensus.cc:697] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:43.397706  2720 consensus_queue.cc:1048] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.398262  2736 consensus_queue.cc:237] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.398310  2560 raft_consensus.cc:3060] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.405098  2309 catalog_manager.cc:5697] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.390682  2413 raft_consensus.cc:3060] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.413733  2731 raft_consensus.cc:993] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.414309  2731 raft_consensus.cc:1081] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.416867  2486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60434
I20260709 13:17:43.417544  2486 raft_consensus.cc:493] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.417949  2486 raft_consensus.cc:3060] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.425436  2720 consensus_queue.cc:1048] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.427008  2560 raft_consensus.cc:2468] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:43.428611  2486 raft_consensus.cc:515] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.429059  2377 leader_election.cc:304] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: fd75afe960fe4640b52ac3328158343e
I20260709 13:17:43.430663  2744 raft_consensus.cc:2804] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.431154  2744 raft_consensus.cc:697] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:43.431774  2487 raft_consensus.cc:1240] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Rejecting Update request from peer 17ac24c81e5d47f1a15abae5f11e5cce for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:43.432098  2744 consensus_queue.cc:237] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.433774  2413 raft_consensus.cc:2468] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.433882  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:43.436053  2720 consensus_queue.cc:1059] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:43.436268  2486 leader_election.cc:290] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.436957  2720 raft_consensus.cc:3055] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.437318  2720 raft_consensus.cc:740] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:43.438236  2720 consensus_queue.cc:260] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.439481  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421c81aafc1d43a084fce36623a90d28" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.440572  2720 raft_consensus.cc:3060] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.459010  2712 consensus_queue.cc:1048] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.459515  2309 catalog_manager.cc:5697] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.461231  2712 consensus_queue.cc:1048] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.464155  2563 raft_consensus.cc:2468] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.466966  2452 leader_election.cc:304] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 641d982ff4c746e3bf32bbee02e44712
I20260709 13:17:43.467881  2736 raft_consensus.cc:2804] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.468384  2736 raft_consensus.cc:697] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:43.469270  2736 consensus_queue.cc:237] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.482215  2692 raft_consensus.cc:993] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: : Instructing follower fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.483124  2692 raft_consensus.cc:1081] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Signalling peer fd75afe960fe4640b52ac3328158343e to start an election
I20260709 13:17:43.481181  2312 catalog_manager.cc:5697] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 1 to 2, leader changed from 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3) to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.484975  2486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
 from {username='slave'} at 127.0.0.1:60428
I20260709 13:17:43.486387  2486 raft_consensus.cc:493] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.486801  2486 raft_consensus.cc:3060] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.496666  2486 raft_consensus.cc:515] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.496666  2310 catalog_manager.cc:5697] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.500195  2414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:43.500550  2486 leader_election.cc:290] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.500837  2414 raft_consensus.cc:3055] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.501308  2414 raft_consensus.cc:740] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:43.502349  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc8cd7e640e94674a822c2ace3fdb0ba" candidate_uuid: "fd75afe960fe4640b52ac3328158343e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.502215  2414 consensus_queue.cc:260] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.503021  2563 raft_consensus.cc:3060] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.503675  2414 raft_consensus.cc:3060] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.507113  2743 consensus_queue.cc:1048] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.511953  2563 raft_consensus.cc:2468] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.512673  2414 raft_consensus.cc:2468] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd75afe960fe4640b52ac3328158343e in term 2.
I20260709 13:17:43.513893  2452 leader_election.cc:304] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:43.516240  2736 raft_consensus.cc:2804] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.516717  2736 raft_consensus.cc:697] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Leader. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:43.517590  2736 consensus_queue.cc:237] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.518603  2743 raft_consensus.cc:993] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: : Instructing follower 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:43.519148  2743 raft_consensus.cc:1081] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Signalling peer 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:43.522199  2563 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:46190
I20260709 13:17:43.523882  2712 consensus_queue.cc:1048] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:43.525429  2563 raft_consensus.cc:493] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.525982  2563 raft_consensus.cc:3060] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.532915  2732 raft_consensus.cc:993] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: : Instructing follower 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:43.533489  2731 raft_consensus.cc:1081] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Signalling peer 641d982ff4c746e3bf32bbee02e44712 to start an election
I20260709 13:17:43.533964  2563 raft_consensus.cc:515] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.535663  2414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
 from {username='slave'} at 127.0.0.1:33212
I20260709 13:17:43.536443  2414 raft_consensus.cc:493] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:43.536803  2414 raft_consensus.cc:3060] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.538830  2563 leader_election.cc:290] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:43.538576  2310 catalog_manager.cc:5697] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e reported cstate change: term changed from 1 to 2, leader changed from 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1) to fd75afe960fe4640b52ac3328158343e (127.21.33.2). New cstate: current_term: 2 leader_uuid: "fd75afe960fe4640b52ac3328158343e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.539806  2411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:43.540102  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaba0a4408ee4af4a78ef5354911ebd7" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:43.540369  2411 raft_consensus.cc:3055] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.540727  2411 raft_consensus.cc:740] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:43.540737  2487 raft_consensus.cc:3060] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.541603  2411 consensus_queue.cc:260] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.542821  2411 raft_consensus.cc:3060] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.550645  2411 raft_consensus.cc:2468] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 2.
I20260709 13:17:43.553822  2487 raft_consensus.cc:2468] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 2.
I20260709 13:17:43.554644  2527 leader_election.cc:304] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:43.555627  2731 raft_consensus.cc:2804] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.556130  2731 raft_consensus.cc:697] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:43.557011  2731 consensus_queue.cc:237] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.559446  2414 raft_consensus.cc:515] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.562234  2414 leader_election.cc:290] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Requested vote from peers fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539), 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3:39227)
I20260709 13:17:43.562975  2487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:43.563295  2563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00d9e86a230e4a71a92ddac9e6b8df5c" candidate_uuid: "641d982ff4c746e3bf32bbee02e44712" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
I20260709 13:17:43.564210  2563 raft_consensus.cc:3055] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:43.564517  2563 raft_consensus.cc:740] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:43.563755  2487 raft_consensus.cc:3060] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.565454  2563 consensus_queue.cc:260] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.567046  2563 raft_consensus.cc:3060] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.570609  2309 catalog_manager.cc:5697] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 1 to 2, leader changed from 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1) to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 2 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:43.574213  2487 raft_consensus.cc:2468] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:43.575433  2379 leader_election.cc:304] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e; no voters: 
I20260709 13:17:43.577061  2563 raft_consensus.cc:2468] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641d982ff4c746e3bf32bbee02e44712 in term 2.
I20260709 13:17:43.577711  2712 raft_consensus.cc:2804] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:43.578276  2712 raft_consensus.cc:697] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Becoming Leader. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:43.579028  2712 consensus_queue.cc:237] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:43.587332  2309 catalog_manager.cc:5697] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 reported cstate change: term changed from 1 to 2, leader changed from 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3) to 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1). New cstate: current_term: 2 leader_uuid: "641d982ff4c746e3bf32bbee02e44712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:43.780851  2414 raft_consensus.cc:1275] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.782407  2736 consensus_queue.cc:1048] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.793099  2563 raft_consensus.cc:1275] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.795406  2760 consensus_queue.cc:1048] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.848834  2563 raft_consensus.cc:1275] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.850323  2712 consensus_queue.cc:1048] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.860706  2487 raft_consensus.cc:3060] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.869561  2487 raft_consensus.cc:1275] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.870970  2692 consensus_queue.cc:1048] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.871752  2563 raft_consensus.cc:1275] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.873631  2760 consensus_queue.cc:1048] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.882728  2414 raft_consensus.cc:3060] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:43.885129  2563 raft_consensus.cc:1275] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.886838  2692 consensus_queue.cc:1048] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.892614  2414 raft_consensus.cc:1275] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.896236  2760 consensus_queue.cc:1048] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:43.899622  2560 raft_consensus.cc:1275] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.901736  2760 consensus_queue.cc:1048] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:43.913154  2487 raft_consensus.cc:1275] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.982627  2692 consensus_queue.cc:1048] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:17:43.984552  2414 raft_consensus.cc:1275] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.985329  2563 raft_consensus.cc:1275] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.986660  2411 raft_consensus.cc:1275] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer fd75afe960fe4640b52ac3328158343e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:43.987273  2377 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:39227, user_credentials={real_user=slave}} blocked reactor thread for 78117.2us
I20260709 13:17:43.991015  2775 consensus_queue.cc:1048] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:43.991577  2487 raft_consensus.cc:1275] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:43.996964  2731 consensus_queue.cc:1048] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:43.997265  2775 consensus_queue.cc:1048] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:44.006367  2736 consensus_queue.cc:1048] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:44.048290  2560 raft_consensus.cc:1275] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.061451  2411 raft_consensus.cc:1275] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.061878  2741 consensus_queue.cc:1048] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:17:44.071338  2778 consensus_queue.cc:1048] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:44.117986  2486 raft_consensus.cc:1275] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Refusing update from remote peer 641d982ff4c746e3bf32bbee02e44712: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:44.120460  2712 consensus_queue.cc:1048] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:46.795907  2310 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50838:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:17:46.887364  2310 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50838:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:17:47.039973 21636 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:47.208590  2827 tablet.cc:1724] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.194383  2824 tablet.cc:1724] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:47.214735  2816 log.cc:927] Time spent T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:17:47.206558  2377 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.33.3:39227, user_credentials={real_user=slave}} blocked reactor thread for 98890.9us
W20260709 13:17:47.290853  2828 log.cc:927] Time spent T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: Append to log took a long time: real 0.082s	user 0.003s	sys 0.000s
W20260709 13:17:47.295938  2823 log.cc:927] Time spent T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
W20260709 13:17:47.296775  2831 log.cc:927] Time spent T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 13:17:47.297969  2833 log.cc:927] Time spent T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:17:47.302127  2830 log.cc:927] Time spent T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
W20260709 13:17:47.304072  2825 log.cc:927] Time spent T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: Append to log took a long time: real 0.122s	user 0.006s	sys 0.000s
I20260709 13:17:47.316565  2835 tablet.cc:1724] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.333709  2845 tablet.cc:1724] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.338340  2838 tablet.cc:1724] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.365134  2840 tablet.cc:1724] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.364929  2837 tablet.cc:1724] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.370548  2844 tablet.cc:1724] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.393213  2827 tablet.cc:1724] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.422003  2846 tablet.cc:1724] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.429615  2842 tablet.cc:1724] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.447495  2824 tablet.cc:1724] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:47.454599  2843 log.cc:927] Time spent T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: Append to log took a long time: real 0.109s	user 0.003s	sys 0.000s
I20260709 13:17:47.475625  2849 tablet.cc:1724] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.503399  2851 tablet.cc:1724] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.513453  2310 catalog_manager.cc:5789] TS 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755) does not have the latest schema for tablet 52217a35c864463a89ea9456ef84d469 (table soft_deleted_table [id=70baf6033554420899d4d95b1054407e]). Expected version 1 got 0
I20260709 13:17:47.512518  2838 tablet.cc:1724] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.528766  2842 tablet.cc:1724] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.506038  2840 tablet.cc:1724] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.535382  2838 tablet.cc:1724] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.536022  2844 tablet.cc:1724] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.541234  2309 catalog_manager.cc:5789] TS fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539) does not have the latest schema for tablet dc8cd7e640e94674a822c2ace3fdb0ba (table soft_deleted_table [id=70baf6033554420899d4d95b1054407e]). Expected version 1 got 0
I20260709 13:17:47.544921  2309 catalog_manager.cc:5789] TS fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539) does not have the latest schema for tablet 69144e3ea31a47d8a6d835bc04709159 (table soft_deleted_table [id=70baf6033554420899d4d95b1054407e]). Expected version 1 got 0
I20260709 13:17:47.546156  2309 catalog_manager.cc:5789] TS fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539) does not have the latest schema for tablet 421c81aafc1d43a084fce36623a90d28 (table soft_deleted_table [id=70baf6033554420899d4d95b1054407e]). Expected version 1 got 0
I20260709 13:17:47.541599  2837 tablet.cc:1724] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.557969  2413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26"
dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
mode: GRACEFUL
new_leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:38414
I20260709 13:17:47.558914  2413 raft_consensus.cc:606] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Received request to transfer leadership to TS 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:47.572445  2846 tablet.cc:1724] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.579586  2842 tablet.cc:1724] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.605715  2845 tablet.cc:1724] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.622913  2835 tablet.cc:1724] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.633031  2849 tablet.cc:1724] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.636129  2845 tablet.cc:1724] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.647564  2849 tablet.cc:1724] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:47.670418  2486 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8"
dest_uuid: "fd75afe960fe4640b52ac3328158343e"
mode: GRACEFUL
new_leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:48162
I20260709 13:17:47.671135  2486 raft_consensus.cc:606] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Received request to transfer leadership to TS 17ac24c81e5d47f1a15abae5f11e5cce
I20260709 13:17:47.673816 21636 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:47.674520 21636 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:47.683185  2310 catalog_manager.cc:6242] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca: soft_deleted_table [id=70baf6033554420899d4d95b1054407e] alter complete (version 1)
W20260709 13:17:47.768611  2510 debug-util.cc:398] Leaking SignalData structure 0x7b08001201e0 after lost signal to thread 21639
W20260709 13:17:47.769951  2510 debug-util.cc:398] Leaking SignalData structure 0x7b080022ae00 after lost signal to thread 2346
W20260709 13:17:47.772957  2510 debug-util.cc:398] Leaking SignalData structure 0x7b0800057700 after lost signal to thread 2587
I20260709 13:17:47.923691  2783 raft_consensus.cc:993] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: : Instructing follower 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:47.924806  2783 raft_consensus.cc:1081] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Signalling peer 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:47.926972  2560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:46190
I20260709 13:17:47.927726  2560 raft_consensus.cc:493] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:47.928182  2560 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:47.939625  2560 raft_consensus.cc:515] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:47.943370  2560 leader_election.cc:290] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:47.943989  2485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:47.944912  2485 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:47.946659  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5698faf31e5444d49b74dd662444cd26" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:47.947590  2413 raft_consensus.cc:3055] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:47.948029  2413 raft_consensus.cc:740] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 641d982ff4c746e3bf32bbee02e44712, State: Running, Role: LEADER
I20260709 13:17:47.949200  2413 consensus_queue.cc:260] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:47.951506  2413 raft_consensus.cc:3060] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:47.960295  2413 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 3.
I20260709 13:17:47.961229  2485 raft_consensus.cc:2468] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 3.
I20260709 13:17:47.962030  2527 leader_election.cc:304] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:47.963176  2791 raft_consensus.cc:2804] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:47.964289  2791 raft_consensus.cc:697] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:47.965417  2791 consensus_queue.cc:237] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:47.979212  2311 catalog_manager.cc:5697] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 2 to 3, leader changed from 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1) to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 3 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:48.189116  2788 raft_consensus.cc:993] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: : Instructing follower 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:48.189750  2852 raft_consensus.cc:1081] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Signalling peer 17ac24c81e5d47f1a15abae5f11e5cce to start an election
I20260709 13:17:48.191447  2560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8"
dest_uuid: "17ac24c81e5d47f1a15abae5f11e5cce"
 from {username='slave'} at 127.0.0.1:46200
I20260709 13:17:48.192005  2560 raft_consensus.cc:493] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:48.192278  2560 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:48.199757  2560 raft_consensus.cc:515] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:48.202769  2560 leader_election.cc:290] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 election: Requested vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:48.203275  2413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "641d982ff4c746e3bf32bbee02e44712"
I20260709 13:17:48.204257  2413 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:48.203518  2485 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97fbbb8b200748dca15c416922ccc0f8" candidate_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd75afe960fe4640b52ac3328158343e"
I20260709 13:17:48.206099  2485 raft_consensus.cc:3055] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:48.206475  2485 raft_consensus.cc:740] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fd75afe960fe4640b52ac3328158343e, State: Running, Role: LEADER
I20260709 13:17:48.207463  2485 consensus_queue.cc:260] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:48.209232  2485 raft_consensus.cc:3060] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:48.213090  2413 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 3.
I20260709 13:17:48.214735  2527 leader_election.cc:304] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce, 641d982ff4c746e3bf32bbee02e44712; no voters: 
I20260709 13:17:48.215672  2857 raft_consensus.cc:2804] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:48.216158  2857 raft_consensus.cc:697] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 LEADER]: Becoming Leader. State: Replica: 17ac24c81e5d47f1a15abae5f11e5cce, State: Running, Role: LEADER
I20260709 13:17:48.217151  2857 consensus_queue.cc:237] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:48.218433  2485 raft_consensus.cc:2468] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17ac24c81e5d47f1a15abae5f11e5cce in term 3.
I20260709 13:17:48.228232  2311 catalog_manager.cc:5697] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce reported cstate change: term changed from 2 to 3, leader changed from fd75afe960fe4640b52ac3328158343e (127.21.33.2) to 17ac24c81e5d47f1a15abae5f11e5cce (127.21.33.3). New cstate: current_term: 3 leader_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:48.468542  2413 raft_consensus.cc:1275] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:48.470700  2857 consensus_queue.cc:1048] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:48.484488  2486 raft_consensus.cc:1275] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:48.486299  2857 consensus_queue.cc:1048] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:48.638098  2413 raft_consensus.cc:1275] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:48.639930  2857 consensus_queue.cc:1048] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:48.654821  2485 raft_consensus.cc:1275] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Refusing update from remote peer 17ac24c81e5d47f1a15abae5f11e5cce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:48.658787  2857 consensus_queue.cc:1048] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:48.749580 21636 tablet_server.cc:179] TabletServer@127.21.33.1:0 shutting down...
I20260709 13:17:48.804083 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:48.806650 21636 tablet_replica.cc:333] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.807794 21636 raft_consensus.cc:2243] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.808418 21636 raft_consensus.cc:2272] T eaba0a4408ee4af4a78ef5354911ebd7 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.811244 21636 tablet_replica.cc:333] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.812000 21636 raft_consensus.cc:2243] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.812486 21636 raft_consensus.cc:2272] T 421c81aafc1d43a084fce36623a90d28 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.815198 21636 tablet_replica.cc:333] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.815891 21636 raft_consensus.cc:2243] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.816406 21636 raft_consensus.cc:2272] T 69144e3ea31a47d8a6d835bc04709159 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.818835 21636 tablet_replica.cc:333] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.819495 21636 raft_consensus.cc:2243] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.820526 21636 raft_consensus.cc:2272] T 00d9e86a230e4a71a92ddac9e6b8df5c P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.823092 21636 tablet_replica.cc:333] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.823684 21636 raft_consensus.cc:2243] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.824697 21636 raft_consensus.cc:2272] T 52217a35c864463a89ea9456ef84d469 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.827896 21636 tablet_replica.cc:333] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.828907 21636 raft_consensus.cc:2243] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.830274 21636 raft_consensus.cc:2272] T 0b628f2eeb3c481fa33dc3951c686819 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.833280 21636 tablet_replica.cc:333] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.834116 21636 raft_consensus.cc:2243] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.835052 21636 raft_consensus.cc:2272] T b0d188cc51bf48d389a66c8fb4be7d70 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.837636 21636 tablet_replica.cc:333] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.838284 21636 raft_consensus.cc:2243] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.839272 21636 raft_consensus.cc:2272] T 0d49b9acef904e9a9c0243c2880cedd4 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.842253 21636 tablet_replica.cc:333] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.842932 21636 raft_consensus.cc:2243] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.843873 21636 raft_consensus.cc:2272] T a22a754bb6fd4bf5be2806cee04c1fef P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.846683 21636 tablet_replica.cc:333] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.847420 21636 raft_consensus.cc:2243] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.847957 21636 raft_consensus.cc:2272] T 97fbbb8b200748dca15c416922ccc0f8 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.851686 21636 tablet_replica.cc:333] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.852360 21636 raft_consensus.cc:2243] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.852857 21636 raft_consensus.cc:2272] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.855492 21636 tablet_replica.cc:333] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.856096 21636 raft_consensus.cc:2243] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.856551 21636 raft_consensus.cc:2272] T e718fcee53b14a67907e9102ac235c88 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.858892 21636 tablet_replica.cc:333] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.859489 21636 raft_consensus.cc:2243] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.860440 21636 raft_consensus.cc:2272] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.862824 21636 tablet_replica.cc:333] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.863484 21636 raft_consensus.cc:2243] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.864071 21636 raft_consensus.cc:2272] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:48.866878  2453 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 25 similar messages]
I20260709 13:17:48.867851 21636 tablet_replica.cc:333] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.869122 21636 raft_consensus.cc:2243] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.869942 21636 raft_consensus.cc:2272] T 5698faf31e5444d49b74dd662444cd26 P 641d982ff4c746e3bf32bbee02e44712 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:48.871176  2453 consensus_peers.cc:597] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:48.873591 21636 tablet_replica.cc:333] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.874363 21636 raft_consensus.cc:2243] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.874910 21636 raft_consensus.cc:2272] T 7b97664c7c30434098b33c6307e2a071 P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.877792 21636 tablet_replica.cc:333] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.878530 21636 raft_consensus.cc:2243] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:48.879048 21636 raft_consensus.cc:2272] T dc8cd7e640e94674a822c2ace3fdb0ba P 641d982ff4c746e3bf32bbee02e44712 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:48.881938 21636 tablet_replica.cc:333] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712: stopping tablet replica
I20260709 13:17:48.882510 21636 raft_consensus.cc:2243] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:48.883488 21636 raft_consensus.cc:2272] T ca726d8f91c54a5591b80fdeed720a42 P 641d982ff4c746e3bf32bbee02e44712 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:48.904948  2527 consensus_peers.cc:597] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:48.934851 21636 tablet_server.cc:196] TabletServer@127.21.33.1:0 shutdown complete.
W20260709 13:17:48.992400  2453 consensus_peers.cc:597] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:48.997874  2527 consensus_peers.cc:597] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:48.998754  2527 consensus_peers.cc:597] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:49.003098  2453 consensus_peers.cc:597] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:49.008737 21636 tablet_server.cc:179] TabletServer@127.21.33.2:0 shutting down...
W20260709 13:17:49.016808  2528 consensus_peers.cc:597] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Couldn't send request to peer fd75afe960fe4640b52ac3328158343e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:49.054419  2528 consensus_peers.cc:597] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Couldn't send request to peer fd75afe960fe4640b52ac3328158343e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:49.060778 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:49.062021 21636 tablet_replica.cc:333] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.063288 21636 raft_consensus.cc:2243] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.064002 21636 raft_consensus.cc:2272] T eaba0a4408ee4af4a78ef5354911ebd7 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.066712 21636 tablet_replica.cc:333] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.067427 21636 raft_consensus.cc:2243] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.068512 21636 raft_consensus.cc:2272] T 69144e3ea31a47d8a6d835bc04709159 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.071267 21636 tablet_replica.cc:333] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.071972 21636 raft_consensus.cc:2243] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.072894 21636 raft_consensus.cc:2272] T 421c81aafc1d43a084fce36623a90d28 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.075462 21636 tablet_replica.cc:333] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.076154 21636 raft_consensus.cc:2243] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.076653 21636 raft_consensus.cc:2272] T 00d9e86a230e4a71a92ddac9e6b8df5c P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.079100 21636 tablet_replica.cc:333] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.079788 21636 raft_consensus.cc:2243] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.080463 21636 raft_consensus.cc:2272] T 52217a35c864463a89ea9456ef84d469 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.083323 21636 tablet_replica.cc:333] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.084393 21636 raft_consensus.cc:2243] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.085065 21636 raft_consensus.cc:2272] T 0b628f2eeb3c481fa33dc3951c686819 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.088609 21636 tablet_replica.cc:333] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.089709 21636 raft_consensus.cc:2243] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.090372 21636 raft_consensus.cc:2272] T b0d188cc51bf48d389a66c8fb4be7d70 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:49.091315  2453 consensus_peers.cc:597] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:49.093333 21636 tablet_replica.cc:333] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.094120 21636 raft_consensus.cc:2243] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.095086 21636 raft_consensus.cc:2272] T dc8cd7e640e94674a822c2ace3fdb0ba P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.097846 21636 tablet_replica.cc:333] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.098457 21636 raft_consensus.cc:2243] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.098912 21636 raft_consensus.cc:2272] T ca726d8f91c54a5591b80fdeed720a42 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.101434 21636 tablet_replica.cc:333] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.102102 21636 raft_consensus.cc:2243] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.102660 21636 raft_consensus.cc:2272] T 0d49b9acef904e9a9c0243c2880cedd4 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.105201 21636 tablet_replica.cc:333] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.105839 21636 raft_consensus.cc:2243] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.106876 21636 raft_consensus.cc:2272] T a22a754bb6fd4bf5be2806cee04c1fef P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.110116 21636 tablet_replica.cc:333] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.110971 21636 raft_consensus.cc:2243] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.112038 21636 raft_consensus.cc:2272] T e718fcee53b14a67907e9102ac235c88 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:49.114288  2453 consensus_peers.cc:597] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:49.115199 21636 tablet_replica.cc:333] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.115867 21636 raft_consensus.cc:2243] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.116361 21636 raft_consensus.cc:2272] T f7e1090e1c1c4b8b9ab62a00dbe87658 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.118970 21636 tablet_replica.cc:333] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.119635 21636 raft_consensus.cc:2243] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.120574 21636 raft_consensus.cc:2272] T 9c51c9dd7adc4c1d91f6820000bd1b19 P fd75afe960fe4640b52ac3328158343e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.123394 21636 tablet_replica.cc:333] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.123986 21636 raft_consensus.cc:2243] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:49.124890 21636 raft_consensus.cc:2272] T 7b97664c7c30434098b33c6307e2a071 P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.127331 21636 tablet_replica.cc:333] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.127897 21636 raft_consensus.cc:2243] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.128438 21636 raft_consensus.cc:2272] T 5698faf31e5444d49b74dd662444cd26 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.131685 21636 tablet_replica.cc:333] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.132268 21636 raft_consensus.cc:2243] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.132705 21636 raft_consensus.cc:2272] T 97fbbb8b200748dca15c416922ccc0f8 P fd75afe960fe4640b52ac3328158343e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.135833 21636 tablet_replica.cc:333] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e: stopping tablet replica
I20260709 13:17:49.136336 21636 raft_consensus.cc:2243] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.136870 21636 raft_consensus.cc:2272] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P fd75afe960fe4640b52ac3328158343e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:49.244182  2527 consensus_peers.cc:597] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:49.272294  2528 consensus_peers.cc:597] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Couldn't send request to peer fd75afe960fe4640b52ac3328158343e. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:49.397300  2528 consensus_peers.cc:597] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Couldn't send request to peer fd75afe960fe4640b52ac3328158343e. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:49.888326  2857 raft_consensus.cc:493] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:49.889176  2857 raft_consensus.cc:515] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:49.891644  2857 leader_election.cc:290] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:49.903311  2528 leader_election.cc:336] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:49.903872  2527 leader_election.cc:336] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:49.904388  2527 leader_election.cc:304] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:49.905324  2872 raft_consensus.cc:2749] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.920894  2872 raft_consensus.cc:493] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:49.921809  2872 raft_consensus.cc:515] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:49.924638  2872 leader_election.cc:290] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:49.925851  2872 raft_consensus.cc:493] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:49.926465  2872 raft_consensus.cc:515] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } }
I20260709 13:17:49.929304  2872 leader_election.cc:290] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:49.930703  2528 leader_election.cc:336] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:49.932233  2527 leader_election.cc:336] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:49.932708  2527 leader_election.cc:304] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:49.933470  2872 raft_consensus.cc:2749] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:49.933591  2528 leader_election.cc:336] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:49.936756  2527 leader_election.cc:336] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:49.937227  2527 leader_election.cc:304] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:49.938033  2872 raft_consensus.cc:2749] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:50.044853  2872 raft_consensus.cc:493] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
I20260709 13:17:50.045657  2872 raft_consensus.cc:515] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.047485  2857 raft_consensus.cc:493] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:50.048429  2857 raft_consensus.cc:515] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.051298  2857 leader_election.cc:290] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:50.052134  2872 leader_election.cc:290] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.057045  2528 leader_election.cc:336] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:50.057750  2528 leader_election.cc:336] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:50.057915  2527 leader_election.cc:336] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.058529  2527 leader_election.cc:304] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.059237  2872 raft_consensus.cc:2749] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:50.059368  2527 leader_election.cc:336] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.060072  2527 leader_election.cc:304] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.061194  2872 raft_consensus.cc:2749] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:50.147409  2872 raft_consensus.cc:493] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:50.148305  2872 raft_consensus.cc:515] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.151101  2872 leader_election.cc:290] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.155262  2528 leader_election.cc:336] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:50.156302  2527 leader_election.cc:336] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.156781  2527 leader_election.cc:304] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.157696  2872 raft_consensus.cc:2749] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:50.180054 21636 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 2510)
I20260709 13:17:50.217625  2872 raft_consensus.cc:493] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:50.218317  2872 raft_consensus.cc:515] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.220278  2872 leader_election.cc:290] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:50.222241  2872 raft_consensus.cc:493] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 641d982ff4c746e3bf32bbee02e44712)
I20260709 13:17:50.222864  2872 raft_consensus.cc:515] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.225646  2872 leader_election.cc:290] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.226387  2527 leader_election.cc:336] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
W20260709 13:17:50.227626  2528 leader_election.cc:336] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:50.229983  2527 leader_election.cc:336] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.230029  2528 leader_election.cc:304] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.231243  2872 raft_consensus.cc:2749] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:50.233038  2528 leader_election.cc:336] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
I20260709 13:17:50.233613  2528 leader_election.cc:304] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.234509  2872 raft_consensus.cc:2749] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:50.285607  2872 raft_consensus.cc:493] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
I20260709 13:17:50.286211  2872 raft_consensus.cc:515] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.288394  2872 leader_election.cc:290] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.295002  2528 leader_election.cc:336] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:50.300390  2527 leader_election.cc:336] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.301062  2527 leader_election.cc:304] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.302448  2872 raft_consensus.cc:2749] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:50.305459  2872 raft_consensus.cc:493] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
I20260709 13:17:50.306159  2872 raft_consensus.cc:515] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.309592  2872 leader_election.cc:290] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.313719  2527 leader_election.cc:336] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
W20260709 13:17:50.315760  2528 leader_election.cc:336] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
I20260709 13:17:50.316213  2528 leader_election.cc:304] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.317024  2872 raft_consensus.cc:2749] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:50.390993  2872 raft_consensus.cc:493] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
I20260709 13:17:50.391687  2872 raft_consensus.cc:515] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.396342  2872 leader_election.cc:290] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.400213  2527 leader_election.cc:336] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
W20260709 13:17:50.401094  2528 leader_election.cc:336] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
I20260709 13:17:50.401824  2528 leader_election.cc:304] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.402595  2872 raft_consensus.cc:2749] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:50.413908  2872 raft_consensus.cc:493] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
I20260709 13:17:50.414492  2872 raft_consensus.cc:515] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.416518  2872 leader_election.cc:290] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
I20260709 13:17:50.420888  2872 raft_consensus.cc:493] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
W20260709 13:17:50.421103  2528 leader_election.cc:336] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
I20260709 13:17:50.421614  2872 raft_consensus.cc:515] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
W20260709 13:17:50.422057  2527 leader_election.cc:336] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.422576  2527 leader_election.cc:304] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.423297  2857 raft_consensus.cc:2749] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:50.424276  2872 leader_election.cc:290] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.428339  2527 leader_election.cc:336] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
W20260709 13:17:50.429123  2528 leader_election.cc:336] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
I20260709 13:17:50.429694  2528 leader_election.cc:304] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.430361  2872 raft_consensus.cc:2749] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:50.583775  2872 raft_consensus.cc:493] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fd75afe960fe4640b52ac3328158343e)
I20260709 13:17:50.584432  2872 raft_consensus.cc:515] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "641d982ff4c746e3bf32bbee02e44712" member_type: VOTER last_known_addr { host: "127.21.33.1" port: 44755 } } peers { permanent_uuid: "fd75afe960fe4640b52ac3328158343e" member_type: VOTER last_known_addr { host: "127.21.33.2" port: 39539 } } peers { permanent_uuid: "17ac24c81e5d47f1a15abae5f11e5cce" member_type: VOTER last_known_addr { host: "127.21.33.3" port: 39227 } }
I20260709 13:17:50.586913  2872 leader_election.cc:290] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755), fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539)
W20260709 13:17:50.591979  2528 leader_election.cc:336] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fd75afe960fe4640b52ac3328158343e (127.21.33.2:39539): Network error: Client connection negotiation failed: client connection to 127.21.33.2:39539: connect: Connection refused (error 111)
W20260709 13:17:50.592819  2527 leader_election.cc:336] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111)
I20260709 13:17:50.593304  2527 leader_election.cc:304] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17ac24c81e5d47f1a15abae5f11e5cce; no voters: 641d982ff4c746e3bf32bbee02e44712, fd75afe960fe4640b52ac3328158343e
I20260709 13:17:50.594116  2872 raft_consensus.cc:2749] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:51.143110 21636 tablet_server.cc:196] TabletServer@127.21.33.2:0 shutdown complete.
I20260709 13:17:51.188221 21636 tablet_server.cc:179] TabletServer@127.21.33.3:0 shutting down...
W20260709 13:17:51.199756  2527 consensus_peers.cc:597] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce -> Peer 641d982ff4c746e3bf32bbee02e44712 (127.21.33.1:44755): Couldn't send request to peer 641d982ff4c746e3bf32bbee02e44712. Status: Network error: Client connection negotiation failed: client connection to 127.21.33.1:44755: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:51.225849 21636 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:51.226766 21636 tablet_replica.cc:333] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.227589 21636 raft_consensus.cc:2243] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:51.228623 21636 raft_consensus.cc:2272] T eaba0a4408ee4af4a78ef5354911ebd7 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.231112 21636 tablet_replica.cc:333] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.231833 21636 raft_consensus.cc:2243] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.232607 21636 raft_consensus.cc:2272] T 69144e3ea31a47d8a6d835bc04709159 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.235334 21636 tablet_replica.cc:333] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.236034 21636 raft_consensus.cc:2243] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.236539 21636 raft_consensus.cc:2272] T 421c81aafc1d43a084fce36623a90d28 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.238772 21636 tablet_replica.cc:333] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.239348 21636 raft_consensus.cc:2243] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.239847 21636 raft_consensus.cc:2272] T 52217a35c864463a89ea9456ef84d469 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.242152 21636 tablet_replica.cc:333] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.242740 21636 raft_consensus.cc:2243] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.243191 21636 raft_consensus.cc:2272] T 0b628f2eeb3c481fa33dc3951c686819 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.245388 21636 tablet_replica.cc:333] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.246057 21636 raft_consensus.cc:2243] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.246546 21636 raft_consensus.cc:2272] T b0d188cc51bf48d389a66c8fb4be7d70 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.248582 21636 tablet_replica.cc:333] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.249194 21636 raft_consensus.cc:2243] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.249717 21636 raft_consensus.cc:2272] T 00d9e86a230e4a71a92ddac9e6b8df5c P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.252285 21636 tablet_replica.cc:333] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.253098 21636 raft_consensus.cc:2243] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.253723 21636 raft_consensus.cc:2272] T 0d49b9acef904e9a9c0243c2880cedd4 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.255945 21636 tablet_replica.cc:333] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.256564 21636 raft_consensus.cc:2243] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.257092 21636 raft_consensus.cc:2272] T a22a754bb6fd4bf5be2806cee04c1fef P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.259435 21636 tablet_replica.cc:333] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.260138 21636 raft_consensus.cc:2243] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:51.261350 21636 raft_consensus.cc:2272] T 97fbbb8b200748dca15c416922ccc0f8 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.263737 21636 tablet_replica.cc:333] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.264434 21636 raft_consensus.cc:2243] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:51.265364 21636 raft_consensus.cc:2272] T 88ed6a6dd1f1495abc4d09ca92e0f4ab P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.267808 21636 tablet_replica.cc:333] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.268543 21636 raft_consensus.cc:2243] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.269094 21636 raft_consensus.cc:2272] T 7b97664c7c30434098b33c6307e2a071 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.271660 21636 tablet_replica.cc:333] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.272301 21636 raft_consensus.cc:2243] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:51.273221 21636 raft_consensus.cc:2272] T 5698faf31e5444d49b74dd662444cd26 P 17ac24c81e5d47f1a15abae5f11e5cce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.275653 21636 tablet_replica.cc:333] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.276260 21636 raft_consensus.cc:2243] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.276772 21636 raft_consensus.cc:2272] T e718fcee53b14a67907e9102ac235c88 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.279047 21636 tablet_replica.cc:333] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.279645 21636 raft_consensus.cc:2243] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.280172 21636 raft_consensus.cc:2272] T f7e1090e1c1c4b8b9ab62a00dbe87658 P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.282415 21636 tablet_replica.cc:333] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.282989 21636 raft_consensus.cc:2243] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.283470 21636 raft_consensus.cc:2272] T 9c51c9dd7adc4c1d91f6820000bd1b19 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.285853 21636 tablet_replica.cc:333] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.286422 21636 raft_consensus.cc:2243] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.286907 21636 raft_consensus.cc:2272] T dc8cd7e640e94674a822c2ace3fdb0ba P 17ac24c81e5d47f1a15abae5f11e5cce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.289058 21636 tablet_replica.cc:333] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce: stopping tablet replica
I20260709 13:17:51.289676 21636 raft_consensus.cc:2243] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:51.290151 21636 raft_consensus.cc:2272] T ca726d8f91c54a5591b80fdeed720a42 P 17ac24c81e5d47f1a15abae5f11e5cce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.337082 21636 tablet_server.cc:196] TabletServer@127.21.33.3:0 shutdown complete.
I20260709 13:17:51.378955 21636 master.cc:562] Master@127.21.33.62:39217 shutting down...
I20260709 13:17:51.412930 21636 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:51.413779 21636 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:51.414152 21636 tablet_replica.cc:333] T 00000000000000000000000000000000 P fc5307b9ebb64c8d9aa0ddd0381fa2ca: stopping tablet replica
I20260709 13:17:51.437918 21636 master.cc:584] Master@127.21.33.62:39217 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (14935 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (30082 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (441484 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:17:51.509364 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 523 messages since previous log ~18 seconds ago
I20260709 13:17:51.509922 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 523 messages since previous log ~18 seconds ago
I20260709 13:17:51.510161 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 517 messages since previous log ~18 seconds ago
I20260709 13:17:51.510412 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:17:51.510952 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:17:51.511339 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~192 seconds ago
I20260709 13:17:51.511605 21636 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 1 messages since previous log ~234 seconds ago
I20260709 13:17:51.511788 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 1 messages since previous log ~234 seconds ago
I20260709 13:17:51.512117 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 70 messages since previous log ~2 seconds ago
I20260709 13:17:51.512491 21636 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~13 seconds ago
